From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 10 06:01:11 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BFA8DBD3; Mon, 10 Feb 2014 06:01:11 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 73D34167B; Mon, 10 Feb 2014 06:01:10 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: X-IronPort-AV: E=Sophos;i="4.95,816,1384318800"; d="scan'208";a="95161725" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 10 Feb 2014 01:01:10 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 2B842B3F85; Mon, 10 Feb 2014 01:01:10 -0500 (EST) Date: Mon, 10 Feb 2014 01:01:10 -0500 (EST) From: Rick Macklem To: Freebsd hackers list Message-ID: <1714012824.3160193.1392012070166.JavaMail.root@uoguelph.ca> Subject: threads getting stuck in vmem_bt_alloc() at "btalloc"? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: Garrett Wollman X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Feb 2014 06:01:11 -0000 Hi, I've been doing some testing using pagesize clusters (4K) for NFS instead of mclbytes (2K) on a single core i386. Sometimes I get threads stuck sleeping on "btalloc", which seems to happen in vmem_bt_alloc(). The comment in vmem_bt_alloc() basically says: out of address space or lost a fill race Since this is persistent, I suspect it is the first case? So, does anyone know what is going on here or what I should look at to try and resolve this? Btw, when I am testing, I don't see the pagesize cluster allocation exceed 400, so it doesn't seem to be a leak or excessive allocation. Thanks in advance for any help, rick ps: Garrett, I cc'd you in part because I remember you mentioning something about mclbyte clusters being pre-allocated vs ?? Maybe, you could explain what you were talking about, since I didn't follow it the last time. From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 10 10:52:26 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6E46BBB3 for ; Mon, 10 Feb 2014 10:52:26 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C99B81D78 for ; Mon, 10 Feb 2014 10:52:25 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.7/8.14.7) with ESMTP id s1AAgacb001080; Mon, 10 Feb 2014 11:42:36 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.7/8.14.7/Submit) with ESMTP id s1AAgahR001077; Mon, 10 Feb 2014 11:42:36 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Mon, 10 Feb 2014 11:42:36 +0100 (CET) From: Wojciech Puchar To: Dieter BSD Subject: Re: opteron a1100 arm In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.4.3 (wojtek.tensor.gdynia.pl [127.0.0.1]); Mon, 10 Feb 2014 11:42:36 +0100 (CET) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Feb 2014 10:52:26 -0000 > > The bad news is that it has only 8 PCIe lanes. Are expansion cards > considered uncool these days, or what? How are you supposed to add all > the essential stuff they leave off the mainboard? anyway - it already have 10Gb/s ethernet, USB and 8 SATA ports. no need for high speed I/O expansions. and 8 single lane PCIe ports seems enough for me. From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 11 18:13:10 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 09C71774 for ; Tue, 11 Feb 2014 18:13:10 +0000 (UTC) Received: from cu01078b.smtpx.saremail.com (cu01078b.smtpx.saremail.com [195.16.151.53]) by mx1.freebsd.org (Postfix) with ESMTP id 831F81FD3 for ; Tue, 11 Feb 2014 18:13:09 +0000 (UTC) Received: from [172.16.2.46] (izaro.sarenet.es [192.148.167.11]) by proxypop04.sare.net (Postfix) with ESMTPSA id 72AC09DC70C for ; Tue, 11 Feb 2014 19:05:29 +0100 (CET) From: Egoitz Aurrekoetxea Subject: mbuf_tag memory freeing issues with LRO enabled on the XENHVM driver Message-Id: <1AECA876-08BC-48F8-B356-42CE35100805@ramattack.net> Date: Tue, 11 Feb 2014 19:05:26 +0100 To: freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) X-Mailer: Apple Mail (2.1827) Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 18:13:10 -0000 Good afternoon, It seems the LRO code inside the FreeBSD's kernel is causing the first = mbuf chain=92s anymbuf->m_hdr->mh_flags due to some failure not to = contain the M_PKTHDR flag.=20 Have you ever seen this bad behavior with any of these drivers having = LRO enabled :=20 cxgb cxgbe e1000 ixgbe ixgbe mxge oce qlxgb qlxge virtio vxge The no presence of this flag, seems to be causing the kernel not to be = able to enter :=20 static void mb_dtor_pack(void *mem, int size, void *arg) { struct mbuf *m; m =3D (struct mbuf *)mem; if ((m->m_flags & M_PKTHDR) !=3D 0) m_tag_delete_chain(m, NULL); <=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D to this function. causing the wired memory to increase a lot due mbuf_tags usage memory to = be pretty high. I have noticed about this issue using the XENHVM network driver (xn), = but taking a look at some other drivers using the same code as this one=85= have found the commented before ones=85. Has anyone too suffered the issue?. Best regards, From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 11 18:36:23 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 806DAFDB for ; Tue, 11 Feb 2014 18:36:23 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D7DF21203 for ; Tue, 11 Feb 2014 18:36:22 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.7/8.14.7) with ESMTP id s1BIa4xO008685 for ; Tue, 11 Feb 2014 19:36:04 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.7/8.14.7/Submit) with ESMTP id s1BIa4mR008682 for ; Tue, 11 Feb 2014 19:36:04 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Tue, 11 Feb 2014 19:36:04 +0100 (CET) From: Wojciech Puchar To: freebsd-hackers@freebsd.org Subject: FreeBSD UDF support Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.4.3 (wojtek.tensor.gdynia.pl [127.0.0.1]); Tue, 11 Feb 2014 19:36:04 +0100 (CET) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 18:36:23 -0000 what is the status? i see mount_udf and UDF filesystem in kernel options. man mount_udf doesn't state it's read only - so i assume RW support is there. but no newfs_udf tried to compile udfclient from ports - doesn't compile, both clang and gcc FreeBSD 10 about month old from cvs. any ideas? From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 11 19:49:42 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96FAFF2 for ; Tue, 11 Feb 2014 19:49:42 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6FE82199C for ; Tue, 11 Feb 2014 19:49:42 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 81049B94B; Tue, 11 Feb 2014 14:49:41 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Subject: Re: Atom Board ACPI API MOPNV10J failing since 9.1 Date: Tue, 11 Feb 2014 13:19:14 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <52CF850A.9060906@erdgeist.org> <201401211215.22021.jhb@freebsd.org> <52DED060.4070800@yahoo.com> In-Reply-To: <52DED060.4070800@yahoo.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201402111319.14546.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 11 Feb 2014 14:49:41 -0500 (EST) Cc: David Shane Holden X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 19:49:42 -0000 On Tuesday, January 21, 2014 2:54:08 pm David Shane Holden wrote: > On 01/21/14 12:15, John Baldwin wrote: > > > > Hmm, I think I see the issue, and I might have a fix for it in the > > works already. The problem is that we haven't reserved pcib1's > > windows when agp probes, so agp0 steals a resource that is already > > in use. The change I have that might fix this isn't trivial though, > > so I don't have a patch I can just give you to test right now. :( > > Also, this isn't a BIOS issue per se. > > > > I came to the same conclusion a few days ago when I started digging into > it more. The agp driver requests some memory and the resource manager > gives it a chunk which should be reserved for the bridge. I couldn't > find an elegant solution; the best I could come up with was to add the > bridge devices first in pci_add_children() to ensure they got attached > before anything else. Which worked, but seems like a hack. Feel free > to ping me if you have something that needs testing. The patch I have in mind is a more general version of this that would have the same effect. I haven't tested it in quite a while, but if you are brave you can test it. http://people.freebsd.org/~jhb/patches/multipass.patch -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 11 19:49:48 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D7A9C104; Tue, 11 Feb 2014 19:49:48 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AD6CE19A5; Tue, 11 Feb 2014 19:49:48 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id A6242B98A; Tue, 11 Feb 2014 14:49:47 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Subject: Re: The sonewconn listen queue overflow issue Date: Tue, 11 Feb 2014 13:30:54 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <52EFEEF7.5010704@xs4all.nl> In-Reply-To: <52EFEEF7.5010704@xs4all.nl> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201402111330.54758.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 11 Feb 2014 14:49:47 -0500 (EST) Cc: hackers@freebsd.org, Photo stuff X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 19:49:48 -0000 On Monday, February 03, 2014 2:33:11 pm Photo stuff wrote: > L.S. > > I came across a lot of messages in the log since, I think, about 9.1, > that sometimes fill it up so much that all else is made invisible, of > the type: > > sonewconn: pcb 0xyyyyyyyyyyyyyyyy: Listen queue overflow: 8 already in > queue awaiting acceptance > > I searched a bit on the web and came across recommendations to try > netstat -nAa to find out which program this came from. > > Well, running netstat -nAa |grep pcb 0xyyyyyyyyyyyyyyyy in a loop didn't > work, it didn't give any output even though the messages kept coming in > the log during that time. > > I forgot about it then recently searched a bit more, and I found this page: > > http://lawrencechen.net/2014/sonewconn-pcb-0xfffffe006acd9310-listen-queue > > So a listen queue overflow of 8 is caused by a listen value of 5 (QLEN > > 3 * (QLIM / 2)) > > Doing: > netstat -LaAn | grep '/5 ' > > resulted in > ffffzzzzzzzzzzzz tcp4 0/0/5 127.0.0.1:8000 > > The ffffzzz value was not the value in the message log, > but the connection means in my case it had to be the junkbuster proxy > (which I still use, still works well :) ), as I didn't run anything else > locally. > > So I looked into junkbuster's cource and in bind.c I changed 2 instances of: > > while (listen(fd, 5) == -1) { > > to > > while (listen(fd, 10) == -1) { /* 10 instead of 5, this fixes dmesg > spamming of the type 'sonewconn: pcb 0xyyyyyyyyyyyyyyyy: Listen queue > overflow: 8 already in queue awaiting acceptance'? */ > > This improved the situation, but still gave the issue of the log filling > up too much. So then I changed it to 20, which gave me silence :) > > Checking the latest log I couldn't find anything in the last 10 days. > > So for people who have this issue I recommend calculating which value > the listen-value that the overflow-value corresponds to, then checking > as above and then it should be possible to find the daemon causing the > issue. And modify that program... > > But while this removes the errors, what do these messages really > signify? I mean which didn't this happen before in earlier versions of > FreeBSD? Maybe earlier versions just dropped the connections without logging a message? The message means that connections are arriving faster than the userland program can accept them. There is a stat for that in 'netstat -s -p tcp' and you can see if it was increasing on older versions of FreeBSD (if you still have a machine with that around) to make sure the only change is the additional log message. > Btw., at the moment (running 10.0 RC2) my message log now gets filled up > with something else :) > > hwpstate0: set freq failed, err 6 > > This happened long ago already in 9.1 with my AMD3500+ and still now on > my AMD FX6100... I am not familiar with how hwpstate works, sorry. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 11 19:49:51 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 16385268 for ; Tue, 11 Feb 2014 19:49:51 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E1D5919AB for ; Tue, 11 Feb 2014 19:49:50 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id E6AC2B981; Tue, 11 Feb 2014 14:49:49 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD UDF support Date: Tue, 11 Feb 2014 13:38:41 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201402111338.41451.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 11 Feb 2014 14:49:50 -0500 (EST) Cc: Wojciech Puchar X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 19:49:51 -0000 On Tuesday, February 11, 2014 1:36:04 pm Wojciech Puchar wrote: > what is the status? > > i see mount_udf and UDF filesystem in kernel options. man mount_udf > doesn't state it's read only - so i assume RW support is there. No, it is read-only. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 11 19:49:44 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CE962F9; Tue, 11 Feb 2014 19:49:44 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A83A519A0; Tue, 11 Feb 2014 19:49:44 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 9BF7BB981; Tue, 11 Feb 2014 14:49:43 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Subject: Re: core dump vs kern.ipc.shm_use_phys Date: Tue, 11 Feb 2014 13:21:02 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <52DFAC31.6030905@FreeBSD.org> In-Reply-To: <52DFAC31.6030905@FreeBSD.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201402111321.02294.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 11 Feb 2014 14:49:43 -0500 (EST) Cc: Andriy Gapon , freebsd-arch@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 19:49:44 -0000 On Wednesday, January 22, 2014 6:32:01 am Andriy Gapon wrote: > > I seems that if kern.ipc.shm_use_phys is enabled then shared memory regions are > not included into a coredump. Seems that each_writable_segment() in > sys/kern/imgact_elf.c skips OBJT_PHYS objects. Hmm, that may be a feature. I often map large shared memory segments with MAP_NOCORE on purpose. Is it to tell if a given OBJT_PHYS object is a SYSV SHM object? (I assume it isn't.) -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 11 19:49:47 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EA48DFF for ; Tue, 11 Feb 2014 19:49:46 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A944E19A3 for ; Tue, 11 Feb 2014 19:49:46 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 9E9C8B9B9; Tue, 11 Feb 2014 14:49:45 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Subject: Re: ULE locking mechanism Date: Tue, 11 Feb 2014 13:25:24 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="windows-1252" Content-Transfer-Encoding: quoted-printable Message-Id: <201402111325.24523.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 11 Feb 2014 14:49:45 -0500 (EST) Cc: Jens Krieg X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 19:49:47 -0000 On Tuesday, January 28, 2014 8:07:08 am Jens Krieg wrote: > Hello, >=20 > we are currently working on project for our university. Our goal is to=20 implement a simple round robin scheduler for FreeBSD 9.2 on a single core=20 machine. > So far we removed most of the functionality of the ULE scheduler except t= he=20 functions that are called from outside. The system successfully boots to us= er=20 land with our RR scheduler managing thread in a list based run queue. Furth= er,=20 it is possible to interact with the system using the shell. >=20 > The next step is to replace the locking mechanism of the ULE scheduler.=20 Therefore, we replaced the scheduling dependent thread_lock/thread_unlock=20 functions by simply disabling/enabling the interrupts. With this modificati= on=20 the kernel works fine until we hit the user land then the system crashes. > The error occurs in the init user process (init_main.c:start_init:685). W= e=20 found out that the page fault is triggered while executing the subyte funct= ion=20 for the first time. See the error description below (unfortunately not show= n=20 in backtrace). > We compared the ULE scheduler with our RR implementation and it appears,= =20 that the parameters passed to subyte as well as the register values are=20 identical. We assume, that whatever caused the error is related to the thre= ad=20 locking replacement. >=20 > Every time the kernel want to modify thread data the corresponding thread= is=20 locked to prevent any interference by other threads. Since we are using a=20 single core machine why isn=92t it sufficient to simply disable interrupt w= hile=20 modifying thread data. Could you provide us with detailed information about= =20 the locking mechanism in FreeBSD and also answer the following questions,=20 please. >=20 > What is the purpose of thread_lock/thread_unlock besides protecting threa= d=20 data? > How does the TDQ LOCK works and how is it related to a thread LOCK? > - all thread LOCKs of the thread located in the run queue pointing to th= e=20 TDQ LOCK, and > - the TDQ LOCK points to the currently running thread > - on context switching the current thread passes the TDQ LOCK to the new= =20 chosen thread > - Could you explain the idea behind that locking concept, please?=20 > Any suggestions we shall care about in our own lock implementation? thread_lock is quite intertwined with other locks. E.g. when a thread is blocked on a turnstile, thread_lock() for that thread locks the 'ts_lock' spin mutex for that turnstile. If you want to replace thread lock, you need to change all the locks that td_lock can be to use your new primitive. You= 'd probably have an easier time just changing how mtx_lock_spin() works. (In= =20 fact, if you just disable 'options SMP', the stock kernel turns=20 mtx_lock_spin() into a function that just disables interrupts.) =46or your core dump, the first step would be to use gdb to map that addres= s to=20 a file line. For example, you can just do 'l *fork_exit+0x9d', or you can = do 'l *' where you use the value from the trap message. Looking at that can probably tell you why you panic'd. =2D-=20 John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 11 21:23:15 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B4CD9DC for ; Tue, 11 Feb 2014 21:23:15 +0000 (UTC) Received: from amailer.gwdg.de (amailer.gwdg.de [134.76.10.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7499413BE for ; Tue, 11 Feb 2014 21:23:15 +0000 (UTC) Received: from p5b159215.dip0.t-ipconnect.de ([91.21.146.21] helo=krabat.raven.hur) by mailer.gwdg.de with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.80) (envelope-from ) id 1WDKnP-0005V8-Gz; Tue, 11 Feb 2014 22:23:11 +0100 Message-ID: <52FA94BF.80304@gwdg.de> Date: Tue, 11 Feb 2014 22:23:11 +0100 From: Rainer Hurling User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Wojciech Puchar , freebsd-hackers@freebsd.org Subject: Re: FreeBSD UDF support References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Authenticated: Id:rhurlin X-Spam-Level: - X-Virus-Scanned: (clean) by exiscan+sophie X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 21:23:15 -0000 Am 11.02.2014 19:36, schrieb Wojciech Puchar: > what is the status? > > i see mount_udf and UDF filesystem in kernel options. man mount_udf > doesn't state it's read only - so i assume RW support is there. > > but no newfs_udf > > tried to compile udfclient from ports - doesn't compile, both clang and gcc > > FreeBSD 10 about month old from cvs. > > any ideas? Not a direct answer on your question with base mount_udf. Do you know https://github.com/williamdevries/UDF/ from William DeVries? His project comes from NetBSD and modernises the UDF standard towards 2.5x. With this driver, you would be able to read modern not encrypted Blu-rays and DVDs. The driver works in principle, some minor issues have to be solved. Unfortunately there is only very little interest to integrate this driver into base. Regards, Rainer From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 11 22:12:25 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C9623699 for ; Tue, 11 Feb 2014 22:12:25 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 82B0E188A for ; Tue, 11 Feb 2014 22:12:25 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1WDLZ0-0000AK-SA for freebsd-hackers@freebsd.org; Tue, 11 Feb 2014 23:12:22 +0100 Received: from 97.96.39.205 ([97.96.39.205]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 11 Feb 2014 23:12:22 +0100 Received: from dpejesh by 97.96.39.205 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 11 Feb 2014 23:12:22 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: David Shane Holden Subject: Re: Atom Board ACPI API MOPNV10J failing since 9.1 Date: Tue, 11 Feb 2014 17:12:11 -0500 Lines: 16 Message-ID: References: <52CF850A.9060906@erdgeist.org> <201401211215.22021.jhb@freebsd.org> <52DED060.4070800@yahoo.com> <201402111319.14546.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 97.96.39.205 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 In-Reply-To: <201402111319.14546.jhb@freebsd.org> X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 22:12:25 -0000 On 02/11/14 13:19, John Baldwin wrote: > > The patch I have in mind is a more general version of this that > would have the same effect. > > I haven't tested it in quite a while, but if you are brave you can > test it. > > http://people.freebsd.org/~jhb/patches/multipass.patch > When I tested your pci_bus_rman3.patch the other day I ran it on this Atom board. To my surprise the problem seemed to have been fixed somewhere along the line. I'm not sure if it was your patch, r261243, some other commit, or me just not having it configured properly to trigger the problem again. I'll try to spend some time tonight on it. From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 11 23:35:16 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D282112F for ; Tue, 11 Feb 2014 23:35:16 +0000 (UTC) Received: from mail-ve0-x231.google.com (mail-ve0-x231.google.com [IPv6:2607:f8b0:400c:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8C10E1024 for ; Tue, 11 Feb 2014 23:35:16 +0000 (UTC) Received: by mail-ve0-f177.google.com with SMTP id jz11so6646447veb.36 for ; Tue, 11 Feb 2014 15:35:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=0ISVvZc33YD3mAGBtafLJSHeaL/aDv+SpLdlUSOCWo0=; b=QBJFY+wO3lV3kDbno4bdmQ/OD7qk+IuAUuIPdNwotj9VUwyr8zrkHRi0gsFNefSnqt xfqW6AU2p/UHf8RItP33WGRayTvGG/8sEtLTrtgo/z6i5ni/pmoHwJftLIeqjJSXeAxq K5EAvQnYIboZ1uCu/y9U0l7w+rTcekcdP2BqeSiVkjMa2JfYfulrOY+gu0ggzn/FsPWB KJRR7rT4YuF2+Bc59UjAi7Jq2EAZEIRsu0fmG72h48x8/kFKcNdkEJ4axQw6L4DLtVXC QmvkkCCxgzVsaCQBjDF1NeGxQbSdtNFLERQjONA3jEhWMY2grIcS5gBQbBKYZOkeMfH9 vpBw== MIME-Version: 1.0 X-Received: by 10.52.236.132 with SMTP id uu4mr47231vdc.47.1392161715632; Tue, 11 Feb 2014 15:35:15 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.220.113.199 with HTTP; Tue, 11 Feb 2014 15:35:15 -0800 (PST) In-Reply-To: <1AECA876-08BC-48F8-B356-42CE35100805@ramattack.net> References: <1AECA876-08BC-48F8-B356-42CE35100805@ramattack.net> Date: Tue, 11 Feb 2014 15:35:15 -0800 X-Google-Sender-Auth: uPBrUYBZ0YyeOLrHHZzW-sU7lTw Message-ID: Subject: Re: mbuf_tag memory freeing issues with LRO enabled on the XENHVM driver From: Adrian Chadd To: Egoitz Aurrekoetxea Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 23:35:16 -0000 +net .. any ideas? -a On 11 February 2014 10:05, Egoitz Aurrekoetxea wrote: > Good afternoon, > > It seems the LRO code inside the FreeBSD's kernel is causing the first mbuf chain's anymbuf->m_hdr->mh_flags due to some failure not to contain the M_PKTHDR flag. > Have you ever seen this bad behavior with any of these drivers having LRO enabled : > > cxgb > cxgbe > e1000 > ixgbe > ixgbe > mxge > oce > qlxgb > qlxge > virtio > vxge > > The no presence of this flag, seems to be causing the kernel not to be able to enter : > > static void > mb_dtor_pack(void *mem, int size, void *arg) > { > struct mbuf *m; > > m = (struct mbuf *)mem; > if ((m->m_flags & M_PKTHDR) != 0) > m_tag_delete_chain(m, NULL); <============ to this function. > > causing the wired memory to increase a lot due mbuf_tags usage memory to be pretty high. > > I have noticed about this issue using the XENHVM network driver (xn), but taking a look at some other drivers using the same code as this one... have found the commented before ones.... > > Has anyone too suffered the issue?. > > Best regards, > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 12 02:02:08 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8639C29A for ; Wed, 12 Feb 2014 02:02:08 +0000 (UTC) Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6A1721354 for ; Wed, 12 Feb 2014 02:02:07 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.95,829,1384329600"; d="scan'208";a="101533348" Received: from vmwexceht06-prd.hq.netapp.com ([10.106.77.104]) by mx11-out.netapp.com with ESMTP; 11 Feb 2014 18:02:02 -0800 Received: from SACEXCMBX04-PRD.hq.netapp.com ([169.254.6.187]) by vmwexceht06-prd.hq.netapp.com ([10.106.77.104]) with mapi id 14.03.0123.003; Tue, 11 Feb 2014 18:02:02 -0800 From: "Gumpula, Suresh" To: "freebsd-hackers@freebsd.org" Subject: malloc(9) and its alignment Thread-Topic: malloc(9) and its alignment Thread-Index: Ac8nlkbcQYqIQNi5TDadKkROsoUT6Q== Date: Wed, 12 Feb 2014 02:02:01 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.106.53.53] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Feb 2014 02:02:08 -0000 Hi, It appears the malloc(9) returns 8 byte aligned ( UMA_ALIGN_PTR) pointers,= but in bus_dmamem_alloc we might end up checking for greater alignment if we take malloc(9) path instead contig_malloc. Can someone please confirm if malloc(9) returns different alignment pointer= s ? bus_dmamem_alloc(bus_dma_tag_t dmat, void** vaddr, int flags, bus_dmamap_t *mapp) { /* * XXX: * (dmat->alignment < dmat->maxsize) is just a quick hack; the exac= t * alignment guarantees of malloc need to be nailed down, and the * code below should be rewritten to take that into account. * * In the meantime, we'll warn the user if malloc gets it wrong. */ if ((dmat->maxsize <=3D PAGE_SIZE) && (dmat->alignment < dmat->maxsize) && dmat->lowaddr >=3D ptoa((vm_paddr_t)Maxmem)) { *vaddr =3D malloc(dmat->maxsize, M_DEVBUF, mflags); } else { *vaddr =3D contigmalloc(dmat->maxsize, M_DEVBUF, mflags, 0ul, dmat->lowaddr, dmat->alignment? dmat->alignment : = 1ul, dmat->boundary); }=20 if (vtophys(*vaddr) & (dmat->alignment - 1)) { NETAPP_MUTED_PRINTF("bus_dmamem_alloc failed to align memor= y properly.\n"); Regards, Suresh From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 12 05:09:35 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 70C2E3D1 for ; Wed, 12 Feb 2014 05:09:35 +0000 (UTC) Received: from mail-qa0-x22e.google.com (mail-qa0-x22e.google.com [IPv6:2607:f8b0:400d:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2A7D4154A for ; Wed, 12 Feb 2014 05:09:35 +0000 (UTC) Received: by mail-qa0-f46.google.com with SMTP id ii20so13204801qab.33 for ; Tue, 11 Feb 2014 21:09:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dataix.net; s=rsa; h=mime-version:date:message-id:subject:from:to:content-type; bh=zkYKm2Py3TiZA8+ZdtQmko19Zizw7ZYQHqzQxu/xH3U=; b=TU+0N7RL+nLY59sJiSKY5FGLZgm+oqjV5HfVg1xyyR4H7NZxmNHabZzJ7FxIGoisCO niV6tQHtre8EUtlYac5cVzy2eurJVKKxJLUwIFzgTgoI+4Vf1uUAdxrkfz3swdI3dPcn J00uUdXw+x7hwCKNRjn93uNyftlr6pzHHQ78I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=zkYKm2Py3TiZA8+ZdtQmko19Zizw7ZYQHqzQxu/xH3U=; b=bG9Cyek1I4sHG1nOd0KOCq5Cn5fe1mjjTlBFpp/K3iJNMTTwM20UjY5GDQYLFzkQPS a4iYBTmuud8Mn2UGVP+7x4ayBi3C1L1qvuRA5A15WBapRZCJVTrYcJU1F1lqNPfAURy+ MroyaYmOwIgBtrBWb21KHBGSUE75aAq9GWYA3L/RymnqGpyRavcy+Ssq+GHmTXR/PUCh SXQwMnJbQ4hN0dDA9EsTOufZXZ/VPQM9iLE7H3Ybm2juWTNxCGMlgWi5NbIPVfoY42Cf Jf3TACYvHyDV/Qwve1LRRXxdoIa/+jnUcwGIE2BQwR5Bz9s0Q7CSGIScv7/IYnKCuwVS sLHw== X-Gm-Message-State: ALoCoQkNZluohgW6wJfFfRgXKIdVd9MxNJ6Nb0Gu2Ygjus4uPoRP5B8q2HUVLJCo7LRZ7x4h19ZH MIME-Version: 1.0 X-Received: by 10.229.56.200 with SMTP id z8mr59353610qcg.1.1392181774284; Tue, 11 Feb 2014 21:09:34 -0800 (PST) Received: by 10.140.97.75 with HTTP; Tue, 11 Feb 2014 21:09:34 -0800 (PST) X-Originating-IP: [75.128.101.59] Date: Wed, 12 Feb 2014 00:09:34 -0500 Message-ID: Subject: Thoughts on Multi-Symlink Concept From: Jason Hellenthal To: freebsd-filesystems@freebsd.org, "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Feb 2014 05:09:35 -0000 Hi All, Now I already know the thoughts on symlink hell within filesystems all to well considering most Linux flavoring's. But I am curious as to what all your opinions would be to add symlink support to multiple target files much like what you could do with cat(1) or portalfs to include a bunch of files in one instance but similar to the following examples in place of such. Instead of: cat /path/to/files* ln -sm /path/to/files* ./my_concat_list cat ./my_concat_list Or ln -sm /path/to/file1 /path/to/file2 ./my_concat_filters pfctl -v -f ./my_concat_filters Personally while I know it's a hack, but I feel it would bring some glue to programs and other such situations that do not have file include support and add support per-say way to create a repeatable playlist to shorten user operations at any given time. Obviously this isn't anywhere else implemented and would need to be a BSD extension of ln(1) but I find that it could be a beneficial feature for those that could use it to its full potential. I've thought about the same instance also being done with hardlinks but I keep coming across the idea that there are too many race conditions that would be found with that. Anyway . . . opinions, thoughts, ideas, criticism . . . welcome. Thanks for your time. From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 12 11:08:34 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DF8E7467 for ; Wed, 12 Feb 2014 11:08:34 +0000 (UTC) Received: from mailhost.m5p.com (ip-2-1-0-2.r03.asbnva02.us.ce.gin.ntt.net [IPv6:2001:418:0:5000::16]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 61E47150D for ; Wed, 12 Feb 2014 11:08:34 +0000 (UTC) Received: from wonderland.m5p.com (localhost [IPv6:::1]) by mailhost.m5p.com (8.14.5/8.14.5) with ESMTP id s1CB8QSx029780 for ; Wed, 12 Feb 2014 06:08:31 -0500 (EST) (envelope-from george+freebsd@m5p.com) Message-ID: <52FB562A.20509@m5p.com> Date: Wed, 12 Feb 2014 06:08:26 -0500 From: George Mitchell User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: Thoughts on Multi-Symlink Concept References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.73 on 10.100.0.3 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (mailhost.m5p.com [IPv6:::1]); Wed, 12 Feb 2014 06:08:32 -0500 (EST) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Feb 2014 11:08:34 -0000 On 02/12/14 00:09, Jason Hellenthal wrote: > Hi All, > > Now I already know the thoughts on symlink hell within filesystems all to > well considering most Linux flavoring's. But I am curious as to what all > your opinions would be to add symlink support to multiple target files much > like what you could do with cat(1) or portalfs to include a bunch of files > in one instance but similar to the following examples in place of such. > > Instead of: cat /path/to/files* > ln -sm /path/to/files* ./my_concat_list > cat ./my_concat_list > > Or > > ln -sm /path/to/file1 /path/to/file2 ./my_concat_filters > pfctl -v -f ./my_concat_filters > > Personally while I know it's a hack, but I feel it would bring some glue to > programs and other such situations that do not have file include support > and add support per-say way to create a repeatable playlist to shorten user > operations at any given time. > > Obviously this isn't anywhere else implemented and would need to be a BSD > extension of ln(1) but I find that it could be a beneficial feature for > those that could use it to its full potential. > > I've thought about the same instance also being done with hardlinks but I > keep coming across the idea that there are too many race conditions that > would be found with that. > > Anyway . . . opinions, thoughts, ideas, criticism . . . welcome. > > > Thanks for your time. [...] My gut feeling is that it pushes functionality that belongs in userland down into the kernel, plus probably introducing new security loopholes. -- George From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 12 14:19:54 2014 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A302EE4 for ; Wed, 12 Feb 2014 14:19:54 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7585E178F for ; Wed, 12 Feb 2014 14:19:54 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WDafI-000MOg-Va; Wed, 12 Feb 2014 14:19:53 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s1CEJmSB074186; Wed, 12 Feb 2014 07:19:48 -0700 (MST) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX19lyvIjqE1NKsgY05EYw7sD Subject: Re: malloc(9) and its alignment From: Ian Lepore To: "Gumpula, Suresh" In-Reply-To: References: Content-Type: text/plain; charset="us-ascii" Date: Wed, 12 Feb 2014 07:19:48 -0700 Message-ID: <1392214788.1145.52.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Feb 2014 14:19:54 -0000 On Wed, 2014-02-12 at 02:02 +0000, Gumpula, Suresh wrote: > Hi, > It appears the malloc(9) returns 8 byte aligned ( UMA_ALIGN_PTR) pointers, but in bus_dmamem_alloc we might end up checking for greater alignment > if we take malloc(9) path instead contig_malloc. > Can someone please confirm if malloc(9) returns different alignment pointers ? > > bus_dmamem_alloc(bus_dma_tag_t dmat, void** vaddr, int flags, > bus_dmamap_t *mapp) > { > /* > * XXX: > * (dmat->alignment < dmat->maxsize) is just a quick hack; the exact > * alignment guarantees of malloc need to be nailed down, and the > * code below should be rewritten to take that into account. > * > * In the meantime, we'll warn the user if malloc gets it wrong. > */ > if ((dmat->maxsize <= PAGE_SIZE) && > (dmat->alignment < dmat->maxsize) && > dmat->lowaddr >= ptoa((vm_paddr_t)Maxmem)) { > *vaddr = malloc(dmat->maxsize, M_DEVBUF, mflags); > } else { > > *vaddr = contigmalloc(dmat->maxsize, M_DEVBUF, mflags, > 0ul, dmat->lowaddr, dmat->alignment? dmat->alignment : 1ul, > dmat->boundary); > } > if (vtophys(*vaddr) & (dmat->alignment - 1)) { > NETAPP_MUTED_PRINTF("bus_dmamem_alloc failed to align memory properly.\n"); > > Regards, > Suresh In my experience, the effective malloc(9) alignment ends up being the same as MINALLOCSIZE, which is UMA_SMALLEST_UNIT, which is 16 bytes on a system with 4K pages. At $work we overrode MINALLOCSIZE to 32 to work around cache line alignment problems in busdma for ARM systems a few years ago. There is a newer set of busdma allocator routines available in kern/subr_busdma_bufalloc.c which are designed to give a platform more control over alignment of busdma buffers smaller than a page. An example of using them can be found in arm/busdma_machdep[-v6].c. As far as I know, they're only being used on ARM platforms right now. -- Ian From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 12 19:40:18 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE4DD581; Wed, 12 Feb 2014 19:40:18 +0000 (UTC) Received: from mx12.netapp.com (mx12.netapp.com [216.240.18.77]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A4F631601; Wed, 12 Feb 2014 19:40:18 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.95,834,1384329600"; d="scan'208";a="142359855" Received: from vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) by mx12-out.netapp.com with ESMTP; 12 Feb 2014 11:40:16 -0800 Received: from SACEXCMBX04-PRD.hq.netapp.com ([169.254.6.187]) by vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) with mapi id 14.03.0123.003; Wed, 12 Feb 2014 11:40:16 -0800 From: "Gumpula, Suresh" To: Ian Lepore Subject: RE: malloc(9) and its alignment Thread-Topic: malloc(9) and its alignment Thread-Index: Ac8nlkbcQYqIQNi5TDadKkROsoUT6QAqkNoAAAYt8KA= Date: Wed, 12 Feb 2014 19:40:15 +0000 Message-ID: References: <1392214788.1145.52.camel@revolution.hippie.lan> In-Reply-To: <1392214788.1145.52.camel@revolution.hippie.lan> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.106.53.53] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Feb 2014 19:40:18 -0000 Thanks Ian for the reply. I will look at the ARM code, but I was thinkin= g why malloc(9) does not return bucket size aligned pointers.=20 Regards, Suresh -----Original Message----- From: Ian Lepore [mailto:ian@FreeBSD.org]=20 Sent: Wednesday, February 12, 2014 9:20 AM To: Gumpula, Suresh Cc: freebsd-hackers@freebsd.org Subject: Re: malloc(9) and its alignment On Wed, 2014-02-12 at 02:02 +0000, Gumpula, Suresh wrote: > Hi, > It appears the malloc(9) returns 8 byte aligned ( UMA_ALIGN_PTR)=20 > pointers, but in bus_dmamem_alloc we might end up checking for greater a= lignment if we take malloc(9) path instead contig_malloc. > Can someone please confirm if malloc(9) returns different alignment point= ers ? >=20 > bus_dmamem_alloc(bus_dma_tag_t dmat, void** vaddr, int flags, > bus_dmamap_t *mapp) > { > /* > * XXX: > * (dmat->alignment < dmat->maxsize) is just a quick hack; the ex= act > * alignment guarantees of malloc need to be nailed down, and the > * code below should be rewritten to take that into account. > * > * In the meantime, we'll warn the user if malloc gets it wrong. > */ > if ((dmat->maxsize <=3D PAGE_SIZE) && > (dmat->alignment < dmat->maxsize) && > dmat->lowaddr >=3D ptoa((vm_paddr_t)Maxmem)) { > *vaddr =3D malloc(dmat->maxsize, M_DEVBUF, mflags); > } else { >=20 > *vaddr =3D contigmalloc(dmat->maxsize, M_DEVBUF, mflags, > 0ul, dmat->lowaddr, dmat->alignment? dmat->alignment = : 1ul, > dmat->boundary); > }=20 > if (vtophys(*vaddr) & (dmat->alignment - 1)) { > NETAPP_MUTED_PRINTF("bus_dmamem_alloc failed to align=20 > memory properly.\n"); >=20 > Regards, > Suresh In my experience, the effective malloc(9) alignment ends up being the same = as MINALLOCSIZE, which is UMA_SMALLEST_UNIT, which is 16 bytes on a system = with 4K pages. At $work we overrode MINALLOCSIZE to 32 to work around cach= e line alignment problems in busdma for ARM systems a few years ago. There is a newer set of busdma allocator routines available in kern/subr_bu= sdma_bufalloc.c which are designed to give a platform more control over ali= gnment of busdma buffers smaller than a page. An example of using them can= be found in arm/busdma_machdep[-v6].c. As far as I know, they're only bei= ng used on ARM platforms right now. -- Ian From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 12 21:52:15 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 929112F4; Wed, 12 Feb 2014 21:52:15 +0000 (UTC) Received: from freebsd.czest.pl (freebsd.czest.pl [212.87.224.105]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 21A8612C8; Wed, 12 Feb 2014 21:52:14 +0000 (UTC) Received: from freebsd.czest.pl (freebsd.czest.pl [212.87.224.105]) by freebsd.czest.pl (8.14.5/8.14.5) with ESMTP id s1CLpO3i032464; Wed, 12 Feb 2014 21:51:24 GMT (envelope-from wkoszek@freebsd.czest.pl) Received: (from wkoszek@localhost) by freebsd.czest.pl (8.14.5/8.14.5/Submit) id s1CLpOSg032463; Wed, 12 Feb 2014 21:51:24 GMT (envelope-from wkoszek) Date: Wed, 12 Feb 2014 21:51:23 +0000 From: "Wojciech A. Koszek" To: freebsd-hackers@freebsd.org, freebsd-current@freebsd.org, soc-status@freebsd.org Subject: FreeBSD GSOC 2014: call for ideas! Message-ID: <20140212215123.GB31185@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-0.4 required=5.0 tests=RP_MATCHES_RCVD, SPF_HELO_PASS, SPF_PASS autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on freebsd.czest.pl X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (freebsd.czest.pl [212.87.224.105]); Wed, 12 Feb 2014 21:51:28 +0000 (UTC) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Feb 2014 21:52:15 -0000 (cross-posted message; keep discussion on hackers@ only) Hello, We want to participate in GSOC 2014. We need more ideas for students, who can be new to FreeBSD and Open Source. Submit your ideas here: http://tinyurl.com/FreeBSD-GSOC2014 If you make them attractive and clearly explained, there should be a bigger chance of getting them picked by somebody. All submitted ideas upon review will be put here: https://wiki.freebsd.org/SummerOfCode2014 If you have a Wiki access, please just copy the skeleton from the top of this page and use it for your task. Thanks, -- Wojciech A. Koszek wkoszek@FreeBSD.org http://www.koszek.com/ From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 12 22:07:13 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 150CA958; Wed, 12 Feb 2014 22:07:13 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DC8F313C9; Wed, 12 Feb 2014 22:07:12 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s1CM75kp077533 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 12 Feb 2014 14:07:06 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s1CM75dD077532; Wed, 12 Feb 2014 14:07:05 -0800 (PST) (envelope-from jmg) Date: Wed, 12 Feb 2014 14:07:05 -0800 From: John-Mark Gurney To: "Gumpula, Suresh" Subject: Re: malloc(9) and its alignment Message-ID: <20140212220705.GV34851@funkthat.com> Mail-Followup-To: "Gumpula, Suresh" , Ian Lepore , "freebsd-hackers@freebsd.org" References: <1392214788.1145.52.camel@revolution.hippie.lan> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 12 Feb 2014 14:07:06 -0800 (PST) Cc: "freebsd-hackers@freebsd.org" , Ian Lepore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Feb 2014 22:07:13 -0000 Gumpula, Suresh wrote this message on Wed, Feb 12, 2014 at 19:40 +0000: > Thanks Ian for the reply. I will look at the ARM code, but I was thinking why malloc(9) does not return bucket size aligned pointers. Always returning bucket sizes aligned pointers may not be ideal for a cache.. say you have a buffer of 512 bytes, where often only the first 128 bytes are used (but all 512 bytes may be)... If you always align at 512, some cache lines will be more heavily used than others, reducing the effective size of the cache... This is only one reason not aligning to size may be better... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 01:53:52 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE534125 for ; Thu, 13 Feb 2014 01:53:52 +0000 (UTC) Received: from natasha.panasas.com (natasha.panasas.com [209.166.131.148]) by mx1.freebsd.org (Postfix) with ESMTP id A4451153C for ; Thu, 13 Feb 2014 01:53:52 +0000 (UTC) Received: from seabiscuit.panasas.com (seabiscuit.panasas.com [172.17.132.204]) by natasha.panasas.com (8.13.1/8.13.1) with ESMTP id s1D1poEA017122 for ; Wed, 12 Feb 2014 20:51:50 -0500 Received: from SEABISCUIT.int.panasas.com ([172.17.132.204]) by seabiscuit ([172.17.132.204]) with mapi id 14.01.0438.000; Wed, 12 Feb 2014 17:51:49 -0800 From: "Pokala, Ravi" To: "freebsd-hackers@freebsd.org" Subject: AF-4Kn and DEV_BSIZE Thread-Topic: AF-4Kn and DEV_BSIZE Thread-Index: AQHPKF4nznad8D6aXki/11oesZDAuw== Date: Thu, 13 Feb 2014 01:51:48 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.3.9.131030 x-originating-ip: [172.17.133.77] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 01:53:52 -0000 Hi hackers, At Panasas, we're working on supporting AF-4Kn drives (and also optimizing for AF-512e drives while we're at it) in our legacy codebase, which is based on: 7.2-RELEASE - a bunch of drivers we don't need - a bunch of utilities we don't use + a bunch of bug fixes merged from 7-STABLE + a bunch of drivers back-ported from {8,9,10}-STABLE + a bunch of internal changes Since 7-STABLE doesn't support AF-4Kn, I'm looking at 10-STABLE for inspiration. We're only interested in SATA, so this basically breaks down into: ata(4) / ad(4) [1] GEOM [2] General kernel stuff I have a pretty good handle on the drivers and GEOM, but I'd like to discuss the "General kernel stuff" bucket. Of particular concern is DEV_BSIZE - and it's relatives: DEV_BSHIFT, btodb() and dbtob() - which are used pretty much everywhere. A bunch of that is in places that don't come anywhere near the drives, so I'm not worried about them. But when I see those macros used in places that *do* come near the drives, I get nervous. For example: sys/kern/vfs_aio.c sys/kern/vfs_bio.c sys/kern/kern_physio.c sys/vm/swap_pager.c sys/vm/vnode_pager.c Swapping, paging, and block IO all come near the drives. On the one hand, the fact that we're doing things in terms of 512-byte blocks all those places suggests that lots of stuff has to change to work with 4KB drive blocks. On the other hand, when I compare the use of those macros in our code base against what's in 10-STABLE, they are substantively the same; since 10-STABLE supports AF-4Kn, that implies it's okay to use those macros those ways. But I'm fuzzy on *how*, if we're passing around 512-byte buffers, everything still works okay with AF-4Kn drives. Can someone shed some light? Thanks, Ravi [1] The elephant in the room was the move from the old ATA driver to ATACAM. Fortunately, the code in sys/cam/ata/ata_da.c::adaregister() that maps from the logical and physical sector sizes detected from the drive, to the more abstract 'struct disk' used by GEOM, is pretty easy to understand and shoehorn into the 7-ish drivers. Once that's done and the disk softc is carrying around the logical sector size, then a bunch of places get changed to use that value instead of DEV_BSIZE. There are a few other nits, but that's the bulk of the changes needed for ata(4) / ad(4). [2] geom_disk.c::g_disk_create() copies 'struct disk's 'd_sectorsize' and 'd_stripesize' into the provider's 'sectorsize' and 'stripesize'. The rest of the GEOM stack uses those correctly, so there aren't any real changes required in GEOM. Hurray! --rp From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 05:08:34 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 910D3C59 for ; Thu, 13 Feb 2014 05:08:34 +0000 (UTC) Received: from mail-pa0-x230.google.com (mail-pa0-x230.google.com [IPv6:2607:f8b0:400e:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 604C017FE for ; Thu, 13 Feb 2014 05:08:34 +0000 (UTC) Received: by mail-pa0-f48.google.com with SMTP id kx10so10223306pab.7 for ; Wed, 12 Feb 2014 21:08:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=/yTULM99MKn6cPVba8StvlEWlZf5g4xfAEnVFOAE9vQ=; b=ASvhIkndNsVwYzo6rquWaDj/z5y7tiBFGBw3ifbQ5FRk4x0dBNAc3vxZrA/ztFPcap V6DT1AXGlt3+GVvDPMTZxqeZ2HOsvY9e1y9Aehnrcw+AhqunUl0zusKdAxqZ5lcHx1K+ jEWjmpDpuk2m2PZ04XLyCo9ow4ViQxF6WN3/krTb9rjK97SheK3t7/WgTY5XCxonoktt q/8SvS8vY8QcsEnJiNTZTkRHH3Cb26yDMWHTwGtbbReXzMj/QeIGgfEvyIrjZOLrFCtp C/jRAUo0YGcmKcuKJxtkrqMJ7ltZv4G0C7/yLxOHTKb0YTIg+RLMgk7KywlN7KL23536 CvUQ== X-Received: by 10.66.163.2 with SMTP id ye2mr43396492pab.110.1392268113860; Wed, 12 Feb 2014 21:08:33 -0800 (PST) Received: from [10.20.30.117] (75-101-82-48.static.sonic.net. [75.101.82.48]) by mx.google.com with ESMTPSA id qh2sm5182846pab.13.2014.02.12.21.08.32 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 12 Feb 2014 21:08:32 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Thoughts on Multi-Symlink Concept From: Jordan Hubbard In-Reply-To: Date: Wed, 12 Feb 2014 21:08:30 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Jason Hellenthal X-Mailer: Apple Mail (2.1827) Cc: freebsd-filesystems@freebsd.org, "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 05:08:34 -0000 On Feb 11, 2014, at 9:09 PM, Jason Hellenthal = wrote: > Instead of: cat /path/to/files* > ln -sm /path/to/files* ./my_concat_list > cat ./my_concat_list >=20 > Or >=20 > ln -sm /path/to/file1 /path/to/file2 ./my_concat_filters > pfctl -v -f ./my_concat_filters Globbing is done in user land (by the shell) - you wouldn=92t want to = push that down into the kernel, which is either what you=92d have to do = or you=92d need a user land daemon which did round-trips with the kernel = to do the translation, which would also need to make sure to get all of = the process permission stuff right since the user id / gid / $CWD would = all potentially affect the expansion of the =93symlink=94. There are other problems. Since =93my_concat_list=94 now expands to = multiple files, you have to pre-expand its contents before actually = execing cat(1), and since filenames can have spaces and other magic = characters (to the shell) in them, you would need to be able to deal = with escaping all of that. Just like=85 the shell does! Even worse still, you=92d be essentially creating a new file type with = very non-file like semantics. Even variant symlinks (/bin -> = /${ARCH}/bin), which can expand differently depending on the user = context, have clearly understandable semantics - you know that the = symlink is going to expand to exactly one file no matter what ARCH is = set to. How, conversely, would you expect =93cat < my_concat_filters=94 = or =93dd if=3Dmy_concat_filters of=3D=85=94 to work? You=92ve = essentially created a construct that can cause truly weird behaviors = unless you use it in exactly one way and one way only (as a file = argument list), the user having to explicitly know the difference when = they consume the symlink. That creates a whole raft of layering = violations, and is very =93not Unix=94, to be honest. If you want to get creative along these lines without breaking Unix, I = think creating some new type of artificial directory that doesn=92t = actually physically contain files but can flexibly reference a = collection of files based on some sort of query - that might be = interesting. - Jordan From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 05:17:23 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9C149F21 for ; Thu, 13 Feb 2014 05:17:23 +0000 (UTC) Received: from mail-pd0-x233.google.com (mail-pd0-x233.google.com [IPv6:2607:f8b0:400e:c02::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6B9EF1897 for ; Thu, 13 Feb 2014 05:17:23 +0000 (UTC) Received: by mail-pd0-f179.google.com with SMTP id fp1so9580288pdb.24 for ; Wed, 12 Feb 2014 21:17:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=FTlk5tdh4pBqNakEiADhHP0yI9UDcFToJKREmbk5rVw=; b=zkJaBZnmXHX1l8OKuQ7RXRjhL1zzw6XQcRxQMJUk6LfdzkUQzC8BaS2TOPOvIZttES pSMgzfl75gAVNYrqTyl+QEI//k9yPXVy+6tdd7AIaq317tdz6T+seMz57ZYSK5mqhDkL +tyySh+rIPRPOkf3kFD2DczTBT2SlXw4hS9nqpIweA6F4usnNq4aBxbljFxiCmg8q2HW rqQl+iDUME0B+ssVoANBvVaNp6DEzLe0ncg8xbIpuDPCzzma5a2WrxILxvx48b/UO/iI day/CmD78alTQIuo0e3eO1URiaCzTPzZvTEOlqWH+WQC/O7B4EX+bWELE3VcX0ZyZRLq lP6A== X-Received: by 10.68.203.102 with SMTP id kp6mr34365950pbc.14.1392268642410; Wed, 12 Feb 2014 21:17:22 -0800 (PST) Received: from [10.20.30.117] (75-101-82-48.static.sonic.net. [75.101.82.48]) by mx.google.com with ESMTPSA id js7sm2013165pbc.35.2014.02.12.21.17.21 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 12 Feb 2014 21:17:21 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Thoughts on Multi-Symlink Concept From: Jordan Hubbard In-Reply-To: Date: Wed, 12 Feb 2014 21:17:19 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Jason Hellenthal X-Mailer: Apple Mail (2.1827) Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 05:17:23 -0000 On Feb 12, 2014, at 9:08 PM, Jordan Hubbard = wrote: > Globbing is done in user land (by the shell) - you wouldn=92t want to = push that down into the kernel, which is either what you=92d have to do = or you=92d need a user land daemon which did round-trips with the kernel = to do the translation, which would also need to make sure to get all of = the process permission stuff right since the user id / gid / $CWD would = all potentially affect the expansion of the =93symlink=94. Actually, just to correct myself, there is a third way, which is that = you could make the shell also do the expansion of the symlink (or = interpose it into libc), but now you=92d just be stacking one weird hack = on top of another weird hack. It=92s still not a good idea for all the = reasons I mentioned, at least not as a =93symlink=94. Maybe some new = type of shell builtin, though I=92m not sure how/where you=92d use it. - Jordan From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 06:41:12 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3F6D3117 for ; Thu, 13 Feb 2014 06:41:12 +0000 (UTC) Received: from mail.bitblocks.com (ns1.bitblocks.com [173.228.5.8]) by mx1.freebsd.org (Postfix) with ESMTP id 1B59A105F for ; Thu, 13 Feb 2014 06:41:11 +0000 (UTC) Received: from altos.bitblocks.com (altos.bitblocks.com [192.168.125.3]) by mail.bitblocks.com (Postfix) with ESMTP id 2EB7CB827; Wed, 12 Feb 2014 22:35:03 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Thoughts on Multi-Symlink Concept From: Bakul Shah In-Reply-To: Date: Wed, 12 Feb 2014 22:35:01 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Jordan Hubbard X-Mailer: Apple Mail (2.1827) Cc: freebsd-filesystems@freebsd.org, "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 06:41:12 -0000 On Feb 12, 2014, at 9:08 PM, Jordan Hubbard = wrote: >=20 > On Feb 11, 2014, at 9:09 PM, Jason Hellenthal = wrote: >=20 >> Instead of: cat /path/to/files* >> ln -sm /path/to/files* ./my_concat_list >> cat ./my_concat_list >>=20 >> Or >>=20 >> ln -sm /path/to/file1 /path/to/file2 ./my_concat_filters >> pfctl -v -f ./my_concat_filters >=20 > Globbing is done in user land (by the shell) - you wouldn=92t want to = push that down into the kernel, which is either what you=92d have to do = or you=92d need a user land daemon which did round-trips with the kernel = to do the translation, which would also need to make sure to get all of = the process permission stuff right since the user id / gid / $CWD would = all potentially affect the expansion of the =93symlink=94. >=20 > There are other problems. Since =93my_concat_list=94 now expands to = multiple files, you have to pre-expand its contents before actually = execing cat(1), and since filenames can have spaces and other magic = characters (to the shell) in them, you would need to be able to deal = with escaping all of that. Just like=85 the shell does! >=20 > Even worse still, you=92d be essentially creating a new file type with = very non-file like semantics. Even variant symlinks (/bin -> = /${ARCH}/bin), which can expand differently depending on the user = context, have clearly understandable semantics - you know that the = symlink is going to expand to exactly one file no matter what ARCH is = set to. How, conversely, would you expect =93cat < my_concat_filters=94 = or =93dd if=3Dmy_concat_filters of=3D=85=94 to work? You=92ve = essentially created a construct that can cause truly weird behaviors = unless you use it in exactly one way and one way only (as a file = argument list), the user having to explicitly know the difference when = they consume the symlink. That creates a whole raft of layering = violations, and is very =93not Unix=94, to be honest. May be a similar idea can be used to implement file striping & mirroring = on top of existing filesystems -- modern distributed FS seem to reinvent = a lot stuff. Instead one can just build a simple thin layer and use = local and remote simple filesystems to create a distributed FS. I = wouldn't use symlinks but indicate additional semantics via some = metadata. > If you want to get creative along these lines without breaking Unix, I = think creating some new type of artificial directory that doesn=92t = actually physically contain files but can flexibly reference a = collection of files based on some sort of query - that might be = interesting. Lot of such variations are possible using fusefs From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 09:15:04 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D83BCFC for ; Thu, 13 Feb 2014 09:15:04 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0B3461BD4 for ; Thu, 13 Feb 2014 09:15:03 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.7/8.14.7) with ESMTP id s1D9EkbV082462; Thu, 13 Feb 2014 10:14:46 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.7/8.14.7/Submit) with ESMTP id s1D9EjWO082438; Thu, 13 Feb 2014 10:14:46 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Thu, 13 Feb 2014 10:14:45 +0100 (CET) From: Wojciech Puchar To: Rainer Hurling Subject: Re: FreeBSD UDF support In-Reply-To: <52FA94BF.80304@gwdg.de> Message-ID: References: <52FA94BF.80304@gwdg.de> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.4.3 (wojtek.tensor.gdynia.pl [127.0.0.1]); Thu, 13 Feb 2014 10:14:46 +0100 (CET) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 09:15:04 -0000 > His project comes from NetBSD and modernises the UDF standard towards > 2.5x. With this driver, you would be able to read modern not encrypted > Blu-rays and DVDs. The driver works in principle, some minor issues have > to be solved. > > Unfortunately there is only very little interest to integrate this > driver into base. > i need RW support so it is not much use for me. From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 09:20:40 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 93BB3B7 for ; Thu, 13 Feb 2014 09:20:40 +0000 (UTC) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.187]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 07DD01C92 for ; Thu, 13 Feb 2014 09:20:39 +0000 (UTC) Received: from dynamic.1.17.e8edf383af80.685b35818d2c.afb.bredband2.com (dynamic.1.17.e8edf383af80.685b35818d2c.afb.bredband2.com [31.208.68.40]) by mrelayeu.kundenserver.de (node=mreue103) with ESMTP (Nemesis) id 0LkRpr-1Vd9sU2anj-00cMq2; Thu, 13 Feb 2014 10:20:37 +0100 Message-ID: <52FC8E61.5050200@FreeBSD.org> Date: Thu, 13 Feb 2014 10:20:33 +0100 From: Christian Brueffer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD UDF support References: <52FA94BF.80304@gwdg.de> In-Reply-To: X-Enigmail-Version: 1.6 OpenPGP: id=3A67DC36; url=http://people.freebsd.org/~brueffer/brueffer.key.asc Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="nD7HwSXHsIn4XETPqXVqGOas2W7x5AKcN" X-Provags-ID: V02:K0:keKcAleIvIjTuepRULZH2u2vTQRF3dxGfpnZXcPPUEO OGBO8NS5bJN+mCXtx3yl2WaDe67DhZoPHqo2KL9tqSjz1ollpR AUP344sQF/BguXFezZQj1Falo1JdTkdWlz1LAJeiaguV7GyVUm OnHLXHe1W6VRiV9zyDnSVRhvPWzT9EvEWOcCP4vzpeLPuSqQHm lStNVQ3dYN/7HwVikb9/gEWIk4UEuMsNsdR7SXNGZZR1noZ6Bq q7ve237fDwKBMM/lxcuOZFu9QEN+21yUcoGtzOdQ/fvaTe0BVZ Aei6lz/9ht+O+3owe2/yr1Px+acU7RcQFxJdQxsumEC/d3FoZL FHoq5mKM3XkDM7olXFcZYOLE+mYLzf23gafoC0X5PF5vrm0IoT KFZtsXk4k4g6vzF4C8FCOwD8za5qloqgjt5d22reBOrPzn28K6 dxozz X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 09:20:40 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --nD7HwSXHsIn4XETPqXVqGOas2W7x5AKcN Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 2/13/14 10:14 AM, Wojciech Puchar wrote: >> His project comes from NetBSD and modernises the UDF standard towards >> 2.5x. With this driver, you would be able to read modern not encrypted= >> Blu-rays and DVDs. The driver works in principle, some minor issues ha= ve >> to be solved. >> >> Unfortunately there is only very little interest to integrate this >> driver into base. >> > i need RW support so it is not much use for me. NetBSD has newfs_udf(8), but it doesn't look like anyone has ported it to FreeBSD yet, Chris --nD7HwSXHsIn4XETPqXVqGOas2W7x5AKcN Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQJ8BAEBCgBmBQJS/I5kXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ5QzhCQjQ5MDgzNDUwNjkyOUM5Mjg2NDE3 OEM4MzY5ODQ3RTE2NDg3AAoJEHjINphH4WSHR2QP/2DbXEm6U3I230kMKhM8M4bx kePSlfFmk0jlUYepSgXouwO0LkaOr/RImoU+ZlXvuqN716CoeXAjxwh82JZqzUTX dwvZkDICWcM16lGz2sTGRrs3TQk9WdtSca6GLZEFB7ayQD2j/wJeOcYAule6ky67 1wMUCs3zh1kLv/o+AsFDWYmajGtvkHEyiIUaZ+8JL8zraRx5iOg4xg8ZNidIo8Rh ca1w6PyqNSq+owKab1wDU9n5KwtFuv1nIBBQugUrH7x+uz3bmofhwDrkzo2AknkS l58iUMr+2HvIoQ5AoA1aptCuZgPIv5368mFTyleS3OutZJ+EJ+g/2LJgEaHkjYob 6jv615oZtR0DwMjku1BGWiyqlwBbawoeiydt3/iNBel7DYGgKE2QE1sPS8beAmYi wY1d9hVhPfLiSDB1bRuU5b9YTmsG3K9l2I2DjMAbVZSEaobJWnrOD+7lwtCuBQmj 4Rk/Tb1OZTG8Gml1fSQZM1kaYoqw0w39KhkRNI6JesUXV4o6fdNkL7UdozuAFTKk W7dRsN8eaihHl8fwmFyKbcx+4B1k0XqLmIkiDKMl6dk6ca19xT+xtmIQeQyWtvoU MRJUwlbEl7ck1r5GVQQjBpCJVZLbX+Ue6lr0Wtk0dbkWshtMusA0EuUeNNzR+Saz OcehOGRgTM7hC+b0H/Yi =ZX4u -----END PGP SIGNATURE----- --nD7HwSXHsIn4XETPqXVqGOas2W7x5AKcN-- From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 09:05:32 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BA1EC989 for ; Thu, 13 Feb 2014 09:05:32 +0000 (UTC) Received: from mail-yh0-x22e.google.com (mail-yh0-x22e.google.com [IPv6:2607:f8b0:4002:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7D6A81AF2 for ; Thu, 13 Feb 2014 09:05:32 +0000 (UTC) Received: by mail-yh0-f46.google.com with SMTP id v1so9736388yhn.19 for ; Thu, 13 Feb 2014 01:05:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=SnFAYCrURSVl8JCJVNQduXL0RfmLEjq3GtLltm6/OUU=; b=exIWAp+tYAauuKmfHkwoLeGUgcoPaOIcwpczW37JCZPNNCIlF4ltYZBY2XL9kxYf/8 9CO4gimI3UFn05j8gVqJfFfvR+eMuH8D5FsjqryHEn6Sv4zKTLCmCEs1V1fSawv7sGwT CERWwXn/I8Fb+yWlGhrShZWDD1POxWHE3EKFT4+37uh1lNzIEYT+uMaFDx9N6IYEpXsM C+IkAsjJOkT5+8t7KKwhU6STcc3VGwf7BxI5dODmHrNSbmT3J6CR6K/sSPJykhnmutaB FGWwONqMvC2CP3LYPPDhpPEuTU6mQuuiNUQXpaoILA9p/VjUEmSnWq+ChFcFeZGFAXDu c3DA== MIME-Version: 1.0 X-Received: by 10.236.101.227 with SMTP id b63mr225473yhg.37.1392282331706; Thu, 13 Feb 2014 01:05:31 -0800 (PST) Received: by 10.170.43.129 with HTTP; Thu, 13 Feb 2014 01:05:31 -0800 (PST) Date: Thu, 13 Feb 2014 11:05:31 +0200 Message-ID: Subject: Receiving jumbo frames From: Viktor Penkoff To: freebsd-hackers@freebsd.org X-Mailman-Approved-At: Thu, 13 Feb 2014 13:20:53 +0000 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 09:05:32 -0000 Hi, folks! I'm writing an extension functionality to not-yet published network driver. I'm receiving the typical ethernet frames without problems. Considering the datasheet of the device, I'm capable of receiving jumbo frames. When I try to do that, e.g. to send jumbo frame of 8000 bytes, I'm receiving only a limited count of them - 105, then the kernel crashes with the following message: "panic: vm_fault: fault on nofault entry, addr: cfcec000". I have inspected a kernel dump with kgdb and the problem occurs at the function bus_dmamap_sync. The line that the panic happens is in a standard invoking receive interrupt function. Packet data is stored in memory buffers, with any single packet spanning multiple buffers if necessary. The buffers are allocated by the CPU and are managed through chained descriptor lists. When I'm sending jumbo packets, e.g. 8008 bytes in size each, due to live debugging with jtag and kgdb, I've got the following information for the mbuf: % ping -I eth0 -c 1 -s 8000 -M dont A.B.C.D -v -p ff {m_hdr =3D {mh_next =3D 0x0, mh_nextpkt =3D 0x0, mh_data =3D 0xc3d41800 "", mh_len =3D 2048, mh_flags =3D 3, mh_type =3D 1, pad =3D "\000"}, M_dat = =3D { MH =3D {MH_pkthdr =3D {rcvif =3D 0x0, header =3D 0x0, len =3D 2048, flo= wid =3D 0, csum_flags =3D 0, csum_data =3D 0, tso_segsz =3D 0, PH_vt =3D { vt_vtag =3D 0, vt_nrecs =3D 0}, tags =3D {slh_first =3D 0x0}, dsa_tag =3D {0, 0}}, MH_dat =3D {MH_ext =3D {ext_buf =3D 0xc3d41800= "", ext_free =3D 0, ext_arg1 =3D 0x0, ext_arg2 =3D 0x0, ext_size =3D = 2048, ref_cnt =3D 0xc3d3f244, ext_type =3D 6}, In this case, the given mbuf must occupy ~4 descriptors, IMO. Following the next pointer in the descriptor list shows that the CPU has an ownership. Nevertheless, =F2he byte count of the received frame shows th= at 8056 bytes are received. More interesting is the fact that an echo reply is given back. But after a constant count of received packets, the kernel hangs. Some background information: To enable the jumbo frame, one must set the appropriate register. At the software level, a ring buffer with the descriptors is implemented. The device, under which the driver runs, is arm-based. The freebsd version is 8.0. The device uses SerialDMA queues for transmitting and receiving. To receive packets, the CPU must perform the following: 1. Prepare a linked list of descriptors 2. Configure a given queue with the address of the first descriptor in the list, 3. enable SerialDMA; With the transmission - I don't have any problems. The logic is the same as by the reception of packets - ring buffer with descriptors. Any ideas what can cause this type of crashes? From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 14:50:25 2014 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 67E409C8; Thu, 13 Feb 2014 14:50:25 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 259A31B9C; Thu, 13 Feb 2014 14:50:24 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WDxcG-000KrL-Ri; Thu, 13 Feb 2014 14:50:17 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s1DEoCtp075195; Thu, 13 Feb 2014 07:50:12 -0700 (MST) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX18JYactPpInJZASldI6+v6A Subject: Re: Receiving jumbo frames From: Ian Lepore To: Viktor Penkoff In-Reply-To: References: Content-Type: text/plain; charset="koi8-r" Date: Thu, 13 Feb 2014 07:50:12 -0700 Message-ID: <1392303012.1145.81.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by damnhippie.dyndns.org id s1DEoCtp075195 Cc: freebsd-hackers@FreeBSD.org, freebsd-arm X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 14:50:25 -0000 On Thu, 2014-02-13 at 11:05 +0200, Viktor Penkoff wrote: > Hi, folks! I'm writing an extension functionality to not-yet published > network driver. > I'm receiving the typical ethernet frames without problems. Considering= the > datasheet of the device, > I'm capable of receiving jumbo frames. When I try to do that, e.g. to s= end > jumbo frame of 8000 bytes, I'm receiving only a limited count of them = - > 105, then the kernel crashes with the following message: > "panic: vm_fault: fault on nofault entry, addr: cfcec000". > I have inspected a kernel dump with kgdb and the problem occurs at the > function bus_dmamap_sync. >=20 > The line that the panic happens is in a standard invoking receive > interrupt function. Packet data is stored in memory buffers, with any > single packet spanning multiple buffers if necessary. The buffers are > allocated by the CPU and are managed through chained descriptor lists. > When I'm sending jumbo packets, e.g. 8008 bytes in size each, due to li= ve > debugging with jtag and kgdb, I've got the following information for th= e > mbuf: >=20 > % ping -I eth0 -c 1 -s 8000 -M dont A.B.C.D -v -p ff >=20 > {m_hdr =3D {mh_next =3D 0x0, mh_nextpkt =3D 0x0, mh_data =3D 0xc3d41800= "", > mh_len =3D 2048, mh_flags =3D 3, mh_type =3D 1, pad =3D "\000"}, M_= dat =3D { > MH =3D {MH_pkthdr =3D {rcvif =3D 0x0, header =3D 0x0, len =3D 2048,= flowid =3D 0, > csum_flags =3D 0, csum_data =3D 0, tso_segsz =3D 0, PH_vt =3D { > vt_vtag =3D 0, vt_nrecs =3D 0}, tags =3D {slh_first =3D 0x0}, > dsa_tag =3D {0, 0}}, MH_dat =3D {MH_ext =3D {ext_buf =3D 0xc3d4= 1800 "", > ext_free =3D 0, ext_arg1 =3D 0x0, ext_arg2 =3D 0x0, ext_size = =3D 2048, > ref_cnt =3D 0xc3d3f244, ext_type =3D 6}, >=20 > In this case, the given mbuf must occupy ~4 descriptors, IMO. Following= the > next pointer in the descriptor list shows that the CPU has > an ownership. Nevertheless, =D4he byte count of the received frame show= s that > 8056 bytes are received. >=20 > More interesting is the fact that an echo reply is given back. But aft= er a > constant count of received packets, the kernel hangs. >=20 > Some background information: > To enable the jumbo frame, one must set the appropriate register. > At the software level, a ring buffer with the descriptors is implemente= d. > The device, under which the driver runs, is arm-based. > The freebsd version is 8.0. The device uses SerialDMA queues for > transmitting and receiving. > To receive packets, the CPU must perform the following: > 1. Prepare a linked list of descriptors > 2. Configure a given queue with the address of the first descriptor= in > the list, > 3. enable SerialDMA; >=20 > With the transmission - I don't have any problems. The logic is the sam= e as > by the reception of packets - ring buffer with descriptors. >=20 > Any ideas what can cause this type of crashes? You mention FreeBSD 8.0, that's pretty old. There have been many fixes to the arm busdma code since then. We still use 8.2 at work, and while we know there are problems with the busdma code that have been fixed in later revisions, they aren't affecting us so we leave well enough alone. We don't use jumbo frames though. At the very least you should apply the changeset r236086 to your code, although it's not all that likely to fix your problem because the mbufs you're working with should be aligned on 2k boundaries and never require a partial cacheline flush. Are you allocating the buffers with bus_dmamem_alloc() then attaching them to mbuf chains as external buffers? Or are you using one of the mbuf allocation routines? When interfacing with the busdma code, are you using bus_dmamap_load_mbuf()? Posting the full backtrace from the panic might help. More information about the arm chip/system you're using might be helpful too. (I've added the freebsd-arm mailing list to the CC, this is pretty likely to be an arm-specific problem rather than a general freebsd thing.) -- Ian From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 15:30:35 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9E1A36C8; Thu, 13 Feb 2014 15:30:35 +0000 (UTC) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4AB931FF7; Thu, 13 Feb 2014 15:30:34 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.8/8.14.8) with ESMTP id s1DFUXLQ078554 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 13 Feb 2014 08:30:33 -0700 (MST) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.8/8.14.8/Submit) with ESMTP id s1DFUX4G078551; Thu, 13 Feb 2014 08:30:33 -0700 (MST) (envelope-from wblock@wonkity.com) Date: Thu, 13 Feb 2014 08:30:33 -0700 (MST) From: Warren Block To: Christian Brueffer Subject: Re: FreeBSD UDF support In-Reply-To: <52FC8E61.5050200@FreeBSD.org> Message-ID: References: <52FA94BF.80304@gwdg.de> <52FC8E61.5050200@FreeBSD.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Thu, 13 Feb 2014 08:30:34 -0700 (MST) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 15:30:35 -0000 On Thu, 13 Feb 2014, Christian Brueffer wrote: > On 2/13/14 10:14 AM, Wojciech Puchar wrote: >>> His project comes from NetBSD and modernises the UDF standard towards >>> 2.5x. With this driver, you would be able to read modern not encrypted >>> Blu-rays and DVDs. The driver works in principle, some minor issues have >>> to be solved. >>> >>> Unfortunately there is only very little interest to integrate this >>> driver into base. >>> >> i need RW support so it is not much use for me. > > NetBSD has newfs_udf(8), but it doesn't look like anyone has ported it > to FreeBSD yet, GSOC candidates have been requested. That might be perfect. From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 20:51:51 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CFD0DBF4 for ; Thu, 13 Feb 2014 20:51:51 +0000 (UTC) Received: from mail-ob0-x233.google.com (mail-ob0-x233.google.com [IPv6:2607:f8b0:4003:c01::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9A684129D for ; Thu, 13 Feb 2014 20:51:51 +0000 (UTC) Received: by mail-ob0-f179.google.com with SMTP id wo20so12923781obc.10 for ; Thu, 13 Feb 2014 12:51:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=G5mpQ6cIqY3viCbPxNwU9dAQAQwJbf+tZ4J0nJ/rGyE=; b=LOySa7juAwT7FXj2wXn1N9yvXVF51rd89Q71RSaNwESri+OofLbDbpofwK2WojTQg4 kqxtk9tSCYtpgxUpE6sC5R39c7tX/6OJvnZtGGaDGU68S/K7Z5bQ4tWDcqb+MitUpZIG yKz7Y4V4bFLIhJ7UNDAso/OO21J2URehEbyczKNYfp/tZ8rHoXgeuhodqCOAoZejQD50 k5YGXjLevkJAHK1rBYqH9flrVKxM1MtCTkKfQYJm2WXQFJh4XGMhpWM7DVDy7w8y9U+f MzqEpx/cfa5SnmL8AYVEsz/u4W/RaRi2zTDNmu8eWep+5lOrahS7/ZyUVXtasw6vNqA6 ORFw== MIME-Version: 1.0 X-Received: by 10.60.54.202 with SMTP id l10mr2868117oep.29.1392324710747; Thu, 13 Feb 2014 12:51:50 -0800 (PST) Received: by 10.182.74.4 with HTTP; Thu, 13 Feb 2014 12:51:50 -0800 (PST) Date: Thu, 13 Feb 2014 12:51:50 -0800 Message-ID: Subject: Debugging rw lock From: Vijay Singh To: hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 20:51:51 -0000 I am running into an issue where an rw lock is read locked and never unlocked, and causes a system to livelock. I was wondering if its possible to figure out which thread owns the read lock? It's the tcp pcbinfo lock. (kgdb-amd64-7.4-08) show_rwlock rw name : tcp class: rw flags: {SLEEP, INITED, WITNESS, RECURSE, UPGRADABLE} state: RLOCK: 1 locks waiters: writers Any help is appreciated. -vijay From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 20:53:35 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5BA7D0D for ; Thu, 13 Feb 2014 20:53:35 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id C247212C0 for ; Thu, 13 Feb 2014 20:53:35 +0000 (UTC) Received: from Alfreds-MacBook-Pro.local (unknown [50.204.88.5]) by elvis.mu.org (Postfix) with ESMTPSA id 5B2E21A3C3F; Thu, 13 Feb 2014 12:53:27 -0800 (PST) Message-ID: <52FD30D9.6050604@mu.org> Date: Thu, 13 Feb 2014 12:53:45 -0800 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Vijay Singh , hackers@freebsd.org Subject: Re: Debugging rw lock References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 20:53:35 -0000 Keep a stack of rwlocks owned in the struct thread. -Alfred On 2/13/14, 12:51 PM, Vijay Singh wrote: > I am running into an issue where an rw lock is read locked and never > unlocked, and causes a system to livelock. I was wondering if its possible > to figure out which thread owns the read lock? > > It's the tcp pcbinfo lock. > > (kgdb-amd64-7.4-08) show_rwlock rw > name : tcp > class: rw > flags: {SLEEP, INITED, WITNESS, RECURSE, UPGRADABLE} > state: RLOCK: 1 locks > waiters: writers > > Any help is appreciated. > > -vijay > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 20:59:11 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BBB18132 for ; Thu, 13 Feb 2014 20:59:11 +0000 (UTC) Received: from mail-oa0-x235.google.com (mail-oa0-x235.google.com [IPv6:2607:f8b0:4003:c02::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 815BE1312 for ; Thu, 13 Feb 2014 20:59:11 +0000 (UTC) Received: by mail-oa0-f53.google.com with SMTP id m1so13121369oag.26 for ; Thu, 13 Feb 2014 12:59:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=3zVQFsIxlYn6QYaAj7efkFGmy3ozRk0tKmfF2w90yiQ=; b=upBN/pLjyqc179lqqkTiyS1bnlFwuJ9poKXCgpmBT4p285BoeFY2BcMFR87649+qv5 LOgtpDmsphHG2cIxDAxPL3/QuG/7XPbmPk95G3MeGOKPZrNxNuE+gTC9XHHfvWnk6OcB ojvBslXHR06iN4bfkYbc7ss3hymRHIKWeDza0ySblegLp0FBTe6+BemtFuoZWe7zRX4+ +ERD7gzf0nAoqsCAo1d97zWHVChFrK02XwikECmkA/UlWoCaLw1bxX+sypb3NeLl507z 5k8O+TKZPjsJI7EFvR0O8HF4GnrjrK/FJxGVLWgRq0VF2Ra0SjWiT+vJcQeVHlbPGX1H N4xw== MIME-Version: 1.0 X-Received: by 10.182.53.72 with SMTP id z8mr3036252obo.36.1392325150681; Thu, 13 Feb 2014 12:59:10 -0800 (PST) Received: by 10.182.74.4 with HTTP; Thu, 13 Feb 2014 12:59:10 -0800 (PST) In-Reply-To: <52FD30D9.6050604@mu.org> References: <52FD30D9.6050604@mu.org> Date: Thu, 13 Feb 2014 12:59:10 -0800 Message-ID: Subject: Re: Debugging rw lock From: Vijay Singh To: Alfred Perlstein Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 20:59:11 -0000 You're talking about instrumenting the code, right? But which thread? I was thinking of augmenting the rw lock to record the readers, but wanted to check if something is possible without instrumentation. On Thu, Feb 13, 2014 at 12:53 PM, Alfred Perlstein wrote: > Keep a stack of rwlocks owned in the struct thread. > > -Alfred > > On 2/13/14, 12:51 PM, Vijay Singh wrote: > >> I am running into an issue where an rw lock is read locked and never >> unlocked, and causes a system to livelock. I was wondering if its possible >> to figure out which thread owns the read lock? >> >> It's the tcp pcbinfo lock. >> >> (kgdb-amd64-7.4-08) show_rwlock rw >> name : tcp >> class: rw >> flags: {SLEEP, INITED, WITNESS, RECURSE, UPGRADABLE} >> state: RLOCK: 1 locks >> waiters: writers >> >> Any help is appreciated. >> >> -vijay >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org >> " >> >> > From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 21:02:42 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AB4DB4F5 for ; Thu, 13 Feb 2014 21:02:42 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 9555013B9 for ; Thu, 13 Feb 2014 21:02:42 +0000 (UTC) Received: from Alfreds-MacBook-Pro.local (unknown [50.204.88.5]) by elvis.mu.org (Postfix) with ESMTPSA id 995431A3CCC; Thu, 13 Feb 2014 13:02:36 -0800 (PST) Message-ID: <52FD32FF.70106@mu.org> Date: Thu, 13 Feb 2014 13:02:55 -0800 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Vijay Singh Subject: Re: Debugging rw lock References: <52FD30D9.6050604@mu.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 21:02:42 -0000 On 2/13/14, 12:59 PM, Vijay Singh wrote: > You're talking about instrumenting the code, right? But which thread? I was > thinking of augmenting the rw lock to record the readers, but wanted to > check if something is possible without instrumentation. If rw locks are implemented using low level atomics then you're going to make the very slow and have a LOT of work to do as opposed to just using a thread specific storage to implement it. You're better off just making use of the fact that only "curthread" can access the per-thread stack and just use that. Or at least that's how it works in my brain. -Alfred > > > On Thu, Feb 13, 2014 at 12:53 PM, Alfred Perlstein wrote: > >> Keep a stack of rwlocks owned in the struct thread. >> >> -Alfred >> >> On 2/13/14, 12:51 PM, Vijay Singh wrote: >> >>> I am running into an issue where an rw lock is read locked and never >>> unlocked, and causes a system to livelock. I was wondering if its possible >>> to figure out which thread owns the read lock? >>> >>> It's the tcp pcbinfo lock. >>> >>> (kgdb-amd64-7.4-08) show_rwlock rw >>> name : tcp >>> class: rw >>> flags: {SLEEP, INITED, WITNESS, RECURSE, UPGRADABLE} >>> state: RLOCK: 1 locks >>> waiters: writers >>> >>> Any help is appreciated. >>> >>> -vijay >>> _______________________________________________ >>> freebsd-hackers@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org >>> " >>> >>> > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 21:06:33 2014 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AD714837 for ; Thu, 13 Feb 2014 21:06:33 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8061F140F for ; Thu, 13 Feb 2014 21:06:33 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WE3UO-0005S0-Aj; Thu, 13 Feb 2014 21:06:32 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s1DL6RLu075580; Thu, 13 Feb 2014 14:06:27 -0700 (MST) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1+pLJtB9yY3Ryq43c/ix/tH Subject: Re: Debugging rw lock From: Ian Lepore To: Vijay Singh In-Reply-To: References: <52FD30D9.6050604@mu.org> Content-Type: text/plain; charset="us-ascii" Date: Thu, 13 Feb 2014 14:06:27 -0700 Message-ID: <1392325587.1145.96.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 21:06:33 -0000 Does option DEADLKRES not work with rwlocks? (I've never used it, just seen it in the NOTES). -- Ian On Thu, 2014-02-13 at 12:59 -0800, Vijay Singh wrote: > You're talking about instrumenting the code, right? But which thread? I was > thinking of augmenting the rw lock to record the readers, but wanted to > check if something is possible without instrumentation. > > > On Thu, Feb 13, 2014 at 12:53 PM, Alfred Perlstein wrote: > > > Keep a stack of rwlocks owned in the struct thread. > > > > -Alfred > > > > On 2/13/14, 12:51 PM, Vijay Singh wrote: > > > >> I am running into an issue where an rw lock is read locked and never > >> unlocked, and causes a system to livelock. I was wondering if its possible > >> to figure out which thread owns the read lock? > >> > >> It's the tcp pcbinfo lock. > >> > >> (kgdb-amd64-7.4-08) show_rwlock rw > >> name : tcp > >> class: rw > >> flags: {SLEEP, INITED, WITNESS, RECURSE, UPGRADABLE} > >> state: RLOCK: 1 locks > >> waiters: writers > >> > >> Any help is appreciated. > >> > >> -vijay > >> _______________________________________________ > >> freebsd-hackers@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org > >> " > >> > >> > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 22:03:07 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D3D6EBD1; Thu, 13 Feb 2014 22:03:07 +0000 (UTC) Received: from mail-ob0-x229.google.com (mail-ob0-x229.google.com [IPv6:2607:f8b0:4003:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8CBD8191E; Thu, 13 Feb 2014 22:03:07 +0000 (UTC) Received: by mail-ob0-f169.google.com with SMTP id wo20so12938202obc.14 for ; Thu, 13 Feb 2014 14:03:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=gJfIorT/xkjI5tRi+l3HulWAnKbYYgusrWZ/48Kbc8o=; b=EbD/wXWJgm3/MwOqLNDXrrZDR2D3L6kixDRyGjFl2I9I0y31Luz61MFmv7t4B4GTNs i0M+3AZh50M/Dsu81wNe716+y0tqoJ1dhwx4oo34Yo42Xv74Z3lYzmLe3OXxwi9uAocV BQXYeNRmTOp5rrhw5CG9KE7YoPnja7S1B3W121mdcs5zvIOPw5EfPNQm1reOcEHMM8q3 vbof86l4n976wBGJ0t9f8iqpy9s4KWN4w8ZQqE1YdWRZui4YLw1Eql5AjxXYC7m6EXLL Iro7OcACSEPynqjZJ8Y7d3CoTTuJatqWKbsz6gz645Kr2Ku7O4NaBj6b6+JYBIHHQD2S Emtg== MIME-Version: 1.0 X-Received: by 10.60.62.243 with SMTP id b19mr3280180oes.42.1392328986737; Thu, 13 Feb 2014 14:03:06 -0800 (PST) Received: by 10.76.130.196 with HTTP; Thu, 13 Feb 2014 14:03:06 -0800 (PST) In-Reply-To: <1392325587.1145.96.camel@revolution.hippie.lan> References: <52FD30D9.6050604@mu.org> <1392325587.1145.96.camel@revolution.hippie.lan> Date: Thu, 13 Feb 2014 17:03:06 -0500 Message-ID: Subject: Re: Debugging rw lock From: Ryan Stone To: Ian Lepore Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" , Vijay Singh X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 22:03:07 -0000 On Thu, Feb 13, 2014 at 4:06 PM, Ian Lepore wrote: > Does option DEADLKRES not work with rwlocks? (I've never used it, just > seen it in the NOTES). > > -- Ian DEADLKRES will panic the system after a thread has been blocked for a timeout period (I think that it's something ridiculous like 30 minutes). However it's of no use when trying to debugging a leaked read lock, because the thread that lost the lock will have left behind no clues as to it's identity. If you have some kind of reproduction scenario, the quickest hack that might get you an answer would be to change INP_INFO_RLOCK to actually take a wlock instead. Rather than instrumenting the code, you could use the dtrace lockstat provider to log every lock/unlock and post-process the output to find the culprit. Make sure that you have r258541 if you want to use lockstat. http://svnweb.freebsd.org/changeset/base/258541 From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 23:57:34 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 64141376; Thu, 13 Feb 2014 23:57:34 +0000 (UTC) Received: from mail-qc0-x22c.google.com (mail-qc0-x22c.google.com [IPv6:2607:f8b0:400d:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0FCC31359; Thu, 13 Feb 2014 23:57:33 +0000 (UTC) Received: by mail-qc0-f172.google.com with SMTP id c9so19191327qcz.3 for ; Thu, 13 Feb 2014 15:57:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=cz3U0btyXW8MwXJuj5oXkC8m9w5u4r8gLJ6IfDdUCeU=; b=d1OZ6lB8+z9iX8cldj+Pfm4lXHdqQKo+SNqdR4Z+9lqcsq6r7V8kDYMep0maQ6YpTE oTXQtBXusFnoVHpqMEWP/vvkQvlJOBmcJlyoGL2oB4SVkN3hsPqO3Izf59okQuRHlHL2 NgktTfJI9gobd+He/1EpzbLtp9KqPofCA72MrqD4tfAXjjXX8/vnk+KdIJkqxsUXb7V7 BYsdK0VdHHQ63R1Bm/wSnwC48wS/A202Tic9ngqDKAFPY9uovDYt8gvVtTWTJFV6xidQ imZv+Wl1NtvMChdbtfhr0Y9WfXpSl/sUU+tk0ZwJbSxz1r7Vf7jl2By5xiEENzaZDcPx 2OSA== MIME-Version: 1.0 X-Received: by 10.140.50.131 with SMTP id s3mr7347333qga.12.1392335853133; Thu, 13 Feb 2014 15:57:33 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.16.10 with HTTP; Thu, 13 Feb 2014 15:57:33 -0800 (PST) Date: Thu, 13 Feb 2014 15:57:33 -0800 X-Google-Sender-Auth: rVGTpt9nbFWANg8oBRjJkIXiVQk Message-ID: Subject: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? From: Adrian Chadd To: "freebsd-arch@freebsd.org" , "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 23:57:34 -0000 Hi, Whilst digging into collisions in the flowtable code I discovered that a bunch of them are due to some of the flowtable_lookup() code running on a different CPU - the contention on the l2/l3 lookup lock(s) was enough to block things so they'd get an obvious chance to be migrated. So this led me to wonder whether in a fully preemptive kernel, a running kernel thread would stay on the current CPU until it hit a very specific subset of things (exited to userland, hit a lock, etc.) Apparently (according to kan and rwatson) this is how we define fully preemptive - it's not just interruptable at almost any point, but the running task may be interrupted and rescheduled on a different CPU outside of specific critical sections. This means that for the flowtable case, the current setup (without atomics to maintain the lists) can only work if all threads doing work with the flowtable structures (ie, lookup, insert, purge) have to be CPU pinned. Otherwise we may have the situation where: sequentually: * lookup occurs on CPU A; * lookup succeeds on CPU A for some almost-expired entry; * preemption occurs, and it gets scheduled to CPU B; then simultaneously: * CPU A's flowtable purge code runs, and decides to purge entries including the current one; * the code now running on CPU B has an item from the CPU A flowtable, and dereferences it as it's being freed, leading to potential badness. Now, it's a ridiculously small window of opportunity, but I'd rather the code be written to be correct and mostly-fast versus very fast and potentially exploding. I'm sure those in operations would agree. :-) So, my questions: * is this actually how fully pre-emptive kernels _may_ behave? * I believe there's a difference between what 4BSD and ULE will do here - is this the case? What are the scheduler behaviours? * are there any other areas in the kernel that rely on pcpu uma zones / curcpu indexes for things outside of sched_pin() ? Thanks, -a From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 02:07:55 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 71C563EB; Fri, 14 Feb 2014 02:07:55 +0000 (UTC) Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4D3641CEF; Fri, 14 Feb 2014 02:07:54 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.95,842,1384329600"; d="scan'208";a="102046970" Received: from vmwexceht02-prd.hq.netapp.com ([10.106.76.240]) by mx11-out.netapp.com with ESMTP; 13 Feb 2014 18:07:52 -0800 Received: from SACEXCMBX04-PRD.hq.netapp.com ([169.254.6.187]) by vmwexceht02-prd.hq.netapp.com ([10.106.76.240]) with mapi id 14.03.0123.003; Thu, 13 Feb 2014 18:07:51 -0800 From: "Gumpula, Suresh" To: John-Mark Gurney Subject: RE: malloc(9) and its alignment Thread-Topic: malloc(9) and its alignment Thread-Index: Ac8nlkbcQYqIQNi5TDadKkROsoUT6QAqkNoAAAYt8KAACiPngAApxumg Date: Fri, 14 Feb 2014 02:07:51 +0000 Message-ID: References: <1392214788.1145.52.camel@revolution.hippie.lan> <20140212220705.GV34851@funkthat.com> In-Reply-To: <20140212220705.GV34851@funkthat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.106.53.53] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-hackers@freebsd.org" , Ian Lepore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 02:07:55 -0000 Thanks for the explanation John. How about porting ARM way of handling req= uired alignment with creation of separate zones And allocating with uma_zalloc(zone) for AMD64 too for bus_dmamem_alloc? Thanks Suresh -----Original Message----- From: John-Mark Gurney [mailto:jmg@funkthat.com]=20 Sent: Wednesday, February 12, 2014 5:07 PM To: Gumpula, Suresh Cc: Ian Lepore; freebsd-hackers@freebsd.org Subject: Re: malloc(9) and its alignment Gumpula, Suresh wrote this message on Wed, Feb 12, 2014 at 19:40 +0000: > Thanks Ian for the reply. I will look at the ARM code, but I was think= ing why malloc(9) does not return bucket size aligned pointers.=20 Always returning bucket sizes aligned pointers may not be ideal for a cache= .. say you have a buffer of 512 bytes, where often only the first 128 bytes are used (but all 512 bytes may be)... If you always align at 51= 2, some cache lines will be more heavily used than others, reducing the eff= ective size of the cache... This is only one reason not aligning to size may be better... --=20 John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 02:14:14 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 34E81688 for ; Fri, 14 Feb 2014 02:14:14 +0000 (UTC) Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C05D61D78 for ; Fri, 14 Feb 2014 02:14:13 +0000 (UTC) Received: from mandree.no-ip.org ([78.48.204.13]) by mail.gmx.com (mrgmx003) with ESMTPSA (Nemesis) id 0LedVG-1VP4552qb8-00qR8r for ; Fri, 14 Feb 2014 03:14:05 +0100 Received: from [IPv6:::1] (localhost6.localdomain6 [IPv6:::1]) by apollo.emma.line.org (Postfix) with ESMTP id 6B9C923CEEB for ; Fri, 14 Feb 2014 03:14:04 +0100 (CET) Message-ID: <52FD7BEC.9050807@gmx.de> Date: Fri, 14 Feb 2014 03:14:04 +0100 From: Matthias Andree User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? References: In-Reply-To: X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:Rp7STEizRTaJkdGWZKyve6QWAE2hao+Wcc72N/ukzd9wqKunBl7 jDM6LYODLg48HOuI8gLivmxd/CR8QJqhhqU95pr0nsGoxgTKvLZMY7S3BEXLbmpVor9G7PZ Ui0AinI5hhl57Ej1QZnqGbRxsM0ZYwQocj6ZocSgSszTgczfx2zJDImxqxMpBt0pu/BaEv5 R3p8CDF3zvtUdtYpS7FkA== X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 02:14:14 -0000 Am 14.02.2014 00:57, schrieb Adrian Chadd: > Hi, > > Whilst digging into collisions in the flowtable code I discovered that > a bunch of them are due to some of the flowtable_lookup() code running > on a different CPU - the contention on the l2/l3 lookup lock(s) was > enough to block things so they'd get an obvious chance to be migrated. > > So this led me to wonder whether in a fully preemptive kernel, a > running kernel thread would stay on the current CPU until it hit a > very specific subset of things (exited to userland, hit a lock, etc.) > > Apparently (according to kan and rwatson) this is how we define fully > preemptive - it's not just interruptable at almost any point, but the > running task may be interrupted and rescheduled on a different CPU > outside of specific critical sections. There's one more question to ask, and that is if you have a thread that warmed up its caches and you migrate it to a different core - what happens with the execution time? It might become longer because you don't usually share or migrate the L1/L2 cache contents, so I'd naively expect lower cache hit ratios. Certainly there is not a simple answer to that, but should someone start thinking about scheduler implementations, the "setup overhead" for the switch to a different core might be relevant. (Whether it's really more than the extra effort a scheduler would require to spend to make a better decision is then yet one more question.) From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 02:37:05 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CA09D9E1; Fri, 14 Feb 2014 02:37:05 +0000 (UTC) Received: from mail-ob0-x22b.google.com (mail-ob0-x22b.google.com [IPv6:2607:f8b0:4003:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 740EC1FF2; Fri, 14 Feb 2014 02:37:05 +0000 (UTC) Received: by mail-ob0-f171.google.com with SMTP id wp4so13291458obc.16 for ; Thu, 13 Feb 2014 18:37:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MERphlNKfDE1BAVIvYiPemycFmItkkla/PRxwB3dpxM=; b=KiEsJ8C7bnC51BcfqhM8FStxMpvZIZjUYzzLbNXrcoGoQx1L2tuxg63IBCfl9fkbH2 +G6wVB/38tnwx7NKlaFi0q2OMamqPvKLms2SkIg2vtxtGjNZqC0GnLsqniznupmRMI/l AQuQx667IRktJxrSBn+/HWfGTZs5L0HoLPPc/yy1GRAFHV2tYh4gWZMO8rftLZY7MylQ iFTKLcDIf4CwyaJ5fhEq/b5KS6kWZykuudpK1ckJt9tapXiymvxxBbiSB8ItVbqE2X/l m9BY6nGlkOO29PoG7GSgqy+cDk9vA5Nak4Z0jRpnVRJEavjVHj0Fd0x03CMRTotXi13P zc7Q== MIME-Version: 1.0 X-Received: by 10.182.113.195 with SMTP id ja3mr4262074obb.46.1392345424631; Thu, 13 Feb 2014 18:37:04 -0800 (PST) Received: by 10.76.130.196 with HTTP; Thu, 13 Feb 2014 18:37:04 -0800 (PST) In-Reply-To: References: Date: Thu, 13 Feb 2014 21:37:04 -0500 Message-ID: Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? From: Ryan Stone To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 02:37:05 -0000 On Thu, Feb 13, 2014 at 6:57 PM, Adrian Chadd wrote: > sequentually: > > * lookup occurs on CPU A; > * lookup succeeds on CPU A for some almost-expired entry; > * preemption occurs, and it gets scheduled to CPU B; > > then simultaneously: > > * CPU A's flowtable purge code runs, and decides to purge entries > including the current one; > * the code now running on CPU B has an item from the CPU A flowtable, > and dereferences it as it's being freed, leading to potential badness. This kind of scenario is definitely possible. All of the FreeBSD kernel code that deals with lockless per-cpu data structures that I have seen (e.g. uma) has used critical_enter()/critical_exit() to prevent preemption, and have been careful to invalidate their references to the per-cpu data if they have to drop the critical section. I don't believe that sched_pin() is good enough because I don't believe that it handles the scenario when thread A gets a reference and then is preempted, thread B frees the entry, and then A is scheduled and uses the now-freed entry. However I'm really not familiar at all with flowtable so maybe there's something preventing that. From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 03:06:09 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BED17444; Fri, 14 Feb 2014 03:06:09 +0000 (UTC) Received: from mail-qc0-x231.google.com (mail-qc0-x231.google.com [IPv6:2607:f8b0:400d:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6AD0B15A7; Fri, 14 Feb 2014 03:06:09 +0000 (UTC) Received: by mail-qc0-f177.google.com with SMTP id i8so19073631qcq.36 for ; Thu, 13 Feb 2014 19:06:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=+jioV4xyVCq1T00uG72ww/G+bXecll9niyHinUnbWKk=; b=Vj8r3VYM1y7MtT/BiWRwTs6KQJ1zda90xdUJb0isC8K/6gKnYXyY2XWekbn74LIXOv huLHFOt4ntEpHHwFqL6UE4YaEaji8wPlWnaj6bR0sb0obnHmWQEBMKYAUxHZmgiUpzH8 3cr/7oZGkIcEkI4aBdYIKR+YiuBh4Y841GKfslmjBNIcZONFnv9mmfxGsMLh8ry1Ho97 X7OGZg4F3uRIzmgv783PtCGn0QrVmHxA2iQt7ECG4vcXJ2PkcmWJ+iF8RqqcOM9O3Re+ VGqypWDzsDnGcB9p99v5+7gJ82qgnJhbTnQS7JTvd/kPlRSBBB92DtafDn2PS+CMJJ84 37Fw== MIME-Version: 1.0 X-Received: by 10.224.11.136 with SMTP id t8mr8929293qat.26.1392347167800; Thu, 13 Feb 2014 19:06:07 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.16.10 with HTTP; Thu, 13 Feb 2014 19:06:07 -0800 (PST) In-Reply-To: References: Date: Thu, 13 Feb 2014 19:06:07 -0800 X-Google-Sender-Auth: Mev_E25aX-jq1TG6UrCYnr52XZM Message-ID: Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? From: Adrian Chadd To: Ryan Stone Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 03:06:09 -0000 On 13 February 2014 18:37, Ryan Stone wrote: > On Thu, Feb 13, 2014 at 6:57 PM, Adrian Chadd wrote: >> sequentually: >> >> * lookup occurs on CPU A; >> * lookup succeeds on CPU A for some almost-expired entry; >> * preemption occurs, and it gets scheduled to CPU B; >> >> then simultaneously: >> >> * CPU A's flowtable purge code runs, and decides to purge entries >> including the current one; >> * the code now running on CPU B has an item from the CPU A flowtable, >> and dereferences it as it's being freed, leading to potential badness. > > This kind of scenario is definitely possible. All of the FreeBSD > kernel code that deals with lockless per-cpu data structures that I > have seen (e.g. uma) has used critical_enter()/critical_exit() to > prevent preemption, and have been careful to invalidate their > references to the per-cpu data if they have to drop the critical > section. > > I don't believe that sched_pin() is good enough because I don't > believe that it handles the scenario when thread A gets a reference > and then is preempted, thread B frees the entry, and then A is > scheduled and uses the now-freed entry. However I'm really not > familiar at all with flowtable so maybe there's something preventing > that. Oh man, I hate you for bringing that up. So, both flowtable_lookup_common() and flowtable_insert() does do a critical_enter() / critical_exit() during the list operations, so that's ok. It won't be pre-empted. If I wrap flowtable_lookup() with sched_pin() so it covers both the lookup path and the insert path, then it won't get scheduled on another CPU. Then the critical sections around the list operations guarantee it won't end up with a freed entry. The fl entry uptime is updated in the critical section so it shouldn't end up overlapping with the flow purger in a way that has it removed. So, I think that's okay. I think once i add sched_pin() in the right spots, this crap won't use a stale entry. Well, as long as the entry get used before the flow cleaner picks it up. Phew! -adrian From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 01:40:14 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 195C5C2A; Fri, 14 Feb 2014 01:40:14 +0000 (UTC) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) by mx1.freebsd.org (Postfix) with ESMTP id 972BD1AEA; Fri, 14 Feb 2014 01:40:13 +0000 (UTC) Received: from amavis-proxy-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3fQHMv5WdfzGN5h; Fri, 14 Feb 2014 02:40:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= user-agent:message-id:references:in-reply-to:organization :subject:subject:from:from:date:date:content-transfer-encoding :content-type:content-type:mime-version:received:received :received:received; s=jakla2; t=1392342007; x=1394934008; bh=kDo KUn3U+RvZMlXJpHmtG0KaTR73yonxHwl56mQFTwU=; b=AwVoyxr0viFQApHnxz1 TNopms1gjcPSIAxe/O89FY4z5yNkBVSv+Vb88ldOaPakgbvCA0bGh9SJ7FvC8wgD L9z8zhdIBT3nDsfRZaAJJofu/zV/lJimrtn1eImkXATBD55sznoDaAQXQpnfHo2f MjYSL9On5dIjnvyhxUvGu+IM= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-proxy-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10012) with ESMTP id tPS2Fa_CqwvM; Fri, 14 Feb 2014 02:40:07 +0100 (CET) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP; Fri, 14 Feb 2014 02:40:07 +0100 (CET) Received: from neli.ijs.si (neli.ijs.si [193.2.4.95]) by mildred.ijs.si (Postfix) with ESMTP id 96711153; Fri, 14 Feb 2014 02:40:07 +0100 (CET) Received: from sleepy.ijs.si ([2001:1470:ff80:e001::1:1]) by neli.ijs.si with HTTP (HTTP/1.1 POST); Fri, 14 Feb 2014 02:40:07 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 14 Feb 2014 02:40:07 +0100 From: Mark Martinec To: freebsd-net@freebsd.org Subject: Re: Merge ping+ping6 and traceroue+traceroute6 to single =?UTF-8?Q?utilities=3F?= Organization: J. Stefan Institute In-Reply-To: References: <1063008459.20140111160525@serebryakov.spb.ru> <52D14140.3090003@gibfest.dk> <20140111.143644.41639035.sthaug@nethelp.no> <52D15185.50802@gibfest.dk> <0c45208d87526a80f559ac09e28163c2.authenticated@ultimatedns.net> Message-ID: <3feb1ef62b087b8bf00f34c42c5c2954@mailbox.ijs.si> X-Sender: Mark.Martinec+freebsd@ijs.si User-Agent: Roundcube Webmail/0.9.5 X-Mailman-Approved-At: Fri, 14 Feb 2014 03:16:15 +0000 Cc: freebsd-hackers@freebsd.org, "Wojciech A. Koszek" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 01:40:14 -0000 2014-02-11, Mark Martinec wrote: > Remember the original PHK's story ( http://bikeshed.com/ ) ? > It ended favourably for the sleep(1) command, it got its new feature. > What can be learned there is: just needs someone to do it and be > persistent enough to be accepted. > > Looks like a perfect task for Google Summer of Code 2014, > time to apply is very near: > http://www.google-melange.com/gsoc/homepage/google/gsoc2014 2014-02-12, Kevin Oberman wrote: > For those who are new at IPv6, the ping6 and traceroute6 commands come > from > the WIDE KAME project. KAME developed one of the earliest IPv6 stacks > and > WIDE used FreeBSD. It became the FreeBSD IPv6 stack and the ping6 and > traceroute6 utilities were brought in with the rest of the KAME code. > > When these tools were written, the IPv6 stack and the supporting > libraries > and APIs were very primitive. I suspect that it was quicker to write > new > tools than to try to integrate IPv6 into the existing standard tools > and, > when things were so rough, there was a clear effort to avoid changes to > working IPv4 code. Separate IPv4 and IPv6 tools made sense then, but > the > need has long vanished... probably even before the KAME project ended. > But > the old, separate tools lived on through simple inertia. > > And so it is today. Inertia is NO reason that it should be this way > forever. I have submitted two entries for FreeBSD Google Summer of Code 2014: https://wiki.freebsd.org/SummerOfCode2014 (should show up there eventually after a review, I hope), one for a unified ping and ping6, the other for a unified traceroute and traceroute6. My first impression was that it may be possible to do both in a single 12 week GSoC job, although after checking existing source code and writing the proposal it now looks to me more like two full-time summer jobs, if they are to be done properly and with attention to details. Looking for one, or preferably two, mentors for students for these tasks. I wonder if Bjoern A. Zeeb wouldn't be the best man for the job ;) Mark From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 03:24:39 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ADD5C84E for ; Fri, 14 Feb 2014 03:24:39 +0000 (UTC) Received: from mail-ie0-x22d.google.com (mail-ie0-x22d.google.com [IPv6:2607:f8b0:4001:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 64BBC174B for ; Fri, 14 Feb 2014 03:24:39 +0000 (UTC) Received: by mail-ie0-f173.google.com with SMTP id y20so4335102ier.18 for ; Thu, 13 Feb 2014 19:24:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dataix.net; s=rsa; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=o5dK6cIEU7C/Nt+rTWCYeR9vAKfwaTFuBy1toEAIako=; b=LssX3ZwPn8AARx2vDeWShlvreID+Qw51vHZux0Aqe8padHmplpCTcomqxZ0vlglTwa XhEBEugkNvD4HQ3/CHj5P1m72cnqpaV5qa3r14+998kNc3pkjw6RjJAwlh+OxeWsO2H3 mLUovAHzJTDc42a2DARSS0YH6lUnfOzCzOoXs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=o5dK6cIEU7C/Nt+rTWCYeR9vAKfwaTFuBy1toEAIako=; b=TN/DFGj5KF1Lan1rSU+pgr/1xnWcbRPBckxMVpJiMlGFjWNARuekj6zusmtmewo3WJ tP0UMXsa72JWykTh1fsXyazjWzsu1WCgvFkXc1dwZsNgWYH+TZTelJVNh8XsW2v9Q9rK ADeWufxSzDIREW16dYZmyg9QocTzPzDfWbvDax5hJUeuUTD4bmev9JYLQFSHbbIsOpHF LvUyxJYywkjeSY91H44iQMZGhw7XLrvtH3Eoo5fXVwuqNVZraCURXP/ByT8bBHIa+npv MgszhG94lxMSzqkuRTtEnS2W1ZkK2ssOHqpVKseHIS5Xl/HVbY5h/QSUtMmvb0NbtpHg CAcA== X-Gm-Message-State: ALoCoQlaNuv5nW1gZw/k4+KkmilpcE+ee7/GlwB2dl+tzWMvfGU2uDxl4YVZ7/TXXJu7Rg6xPW3Z X-Received: by 10.50.137.71 with SMTP id qg7mr431215igb.38.1392348278652; Thu, 13 Feb 2014 19:24:38 -0800 (PST) Received: from [172.31.35.2] (75-128-101-59.dhcp.sgnw.mi.charter.com. [75.128.101.59]) by mx.google.com with ESMTPSA id c17sm590678igo.4.2014.02.13.19.24.36 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 13 Feb 2014 19:24:36 -0800 (PST) References: <1063008459.20140111160525@serebryakov.spb.ru> <52D14140.3090003@gibfest.dk> <20140111.143644.41639035.sthaug@nethelp.no> <52D15185.50802@gibfest.dk> <0c45208d87526a80f559ac09e28163c2.authenticated@ultimatedns.net> <3feb1ef62b087b8bf00f34c42c5c2954@mailbox.ijs.si> Mime-Version: 1.0 (1.0) In-Reply-To: <3feb1ef62b087b8bf00f34c42c5c2954@mailbox.ijs.si> Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-DE296154-25EF-470F-8900-A78FBC004AF7; protocol="application/pkcs7-signature" Content-Transfer-Encoding: 7bit Message-Id: <05380FA6-4E24-4BFA-AB96-81665C867ABA@dataix.net> X-Mailer: iPhone Mail (11B554a) From: Jason Hellenthal Subject: Re: Merge ping+ping6 and traceroue+traceroute6 to single utilities? Date: Thu, 13 Feb 2014 22:24:33 -0500 To: Mark Martinec Cc: "freebsd-net@freebsd.org" , "Wojciech A. Koszek" , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 03:24:39 -0000 --Apple-Mail-DE296154-25EF-470F-8900-A78FBC004AF7 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable > On Feb 13, 2014, at 20:40, Mark Martinec wr= ote: >=20 > 2014-02-11, Mark Martinec wrote: >> Remember the original PHK's story ( http://bikeshed.com/ ) ? >> It ended favourably for the sleep(1) command, it got its new feature. >> What can be learned there is: just needs someone to do it and be >> persistent enough to be accepted. >> Looks like a perfect task for Google Summer of Code 2014, >> time to apply is very near: >> http://www.google-melange.com/gsoc/homepage/google/gsoc2014 >=20 >=20 > 2014-02-12, Kevin Oberman wrote: >> For those who are new at IPv6, the ping6 and traceroute6 commands come fr= om >> the WIDE KAME project. KAME developed one of the earliest IPv6 stacks and= >> WIDE used FreeBSD. It became the FreeBSD IPv6 stack and the ping6 and >> traceroute6 utilities were brought in with the rest of the KAME code. >> When these tools were written, the IPv6 stack and the supporting librarie= s >> and APIs were very primitive. I suspect that it was quicker to write new >> tools than to try to integrate IPv6 into the existing standard tools and,= >> when things were so rough, there was a clear effort to avoid changes to >> working IPv4 code. Separate IPv4 and IPv6 tools made sense then, but the >> need has long vanished... probably even before the KAME project ended. Bu= t >> the old, separate tools lived on through simple inertia. >> And so it is today. Inertia is NO reason that it should be this way forev= er. >=20 >=20 > I have submitted two entries for FreeBSD Google Summer of Code 2014: >=20 > https://wiki.freebsd.org/SummerOfCode2014 >=20 > (should show up there eventually after a review, I hope), >=20 > one for a unified ping and ping6, the other for a unified traceroute > and traceroute6. My first impression was that it may be possible to do > both in a single 12 week GSoC job, although after checking existing > source code and writing the proposal it now looks to me more like > two full-time summer jobs, if they are to be done properly and with > attention to details. >=20 > Looking for one, or preferably two, mentors for students for these tasks. > I wonder if Bjoern A. Zeeb wouldn't be the best man for the job ;) >=20 Awesome, personally that would seem like the best route not only to have the= focus on the tool itself but to put focus on achieving one or another eithe= r way it's progress. If we were voting I couldn't agree more with Bjoern if he would accept it.. C= ouldn't imagine someone more in depth to fit the task.= --Apple-Mail-DE296154-25EF-470F-8900-A78FBC004AF7 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIUOTCCBjAw ggUYoAMCAQICAwaijjANBgkqhkiG9w0BAQsFADCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0 YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcx ODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlhdGUgQ2xpZW50IENB MB4XDTEzMDUxODA4NTA0OFoXDTE0MDUxOTIyMDk0N1owSDEfMB0GA1UEAwwWamhlbGxlbnRoYWxA ZGF0YWl4Lm5ldDElMCMGCSqGSIb3DQEJARYWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCASIwDQYJ KoZIhvcNAQEBBQADggEPADCCAQoCggEBALgnYFS1bWZr3KhKBzWAdRwrY+En+RRV8nCaYubqrMG+ YJbuenaIKSbIuFiDWipW4RHYTpE28pKaSnaVTG9WtAZvsWj0gYN9g2fYCnCOUceES2Yvi3RavxpB hsuzKIfsHb8iNNSEuczLu6gn4mQyaHwE4x6xSUKmbK8njR+YoF522F60wjsnq5dlOJdTrhDfObE5 5P23279WbRp8azgZX1VRB66wdKRDuSI1vBts4Nsha2paXd6HUUduHrPACBQREJTGXN8XtEKVwo63 aKUhRgtUwHNEuSWck/xwVl7PBUWH2dORAWTCqHjNuCKNOQ1/0LMiyMj7FdsBjN4dgL4YZpsCAwEA AaOCAtwwggLYMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggr BgEFBQcDBDAdBgNVHQ4EFgQU29qUrmZtgQ7ZVoDKogfpJOSfk+YwHwYDVR0jBBgwFoAUU3Ltkpzg 2ssBXHx+ljVO8tS4UYIwIQYDVR0RBBowGIEWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCAUwGA1Ud IASCAUMwggE/MIIBOwYLKwYBBAGBtTcBAgMwggEqMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9wb2xpY3kucGRmMIH3BggrBgEFBQcCAjCB6jAnFiBTdGFydENvbSBDZXJ0aWZp Y2F0aW9uIEF1dGhvcml0eTADAgEBGoG+VGhpcyBjZXJ0aWZpY2F0ZSB3YXMgaXNzdWVkIGFjY29y ZGluZyB0byB0aGUgQ2xhc3MgMSBWYWxpZGF0aW9uIHJlcXVpcmVtZW50cyBvZiB0aGUgU3RhcnRD b20gQ0EgcG9saWN5LCByZWxpYW5jZSBvbmx5IGZvciB0aGUgaW50ZW5kZWQgcHVycG9zZSBpbiBj b21wbGlhbmNlIG9mIHRoZSByZWx5aW5nIHBhcnR5IG9ibGlnYXRpb25zLjA2BgNVHR8ELzAtMCug KaAnhiVodHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9jcnR1MS1jcmwuY3JsMIGOBggrBgEFBQcBAQSB gTB/MDkGCCsGAQUFBzABhi1odHRwOi8vb2NzcC5zdGFydHNzbC5jb20vc3ViL2NsYXNzMS9jbGll bnQvY2EwQgYIKwYBBQUHMAKGNmh0dHA6Ly9haWEuc3RhcnRzc2wuY29tL2NlcnRzL3N1Yi5jbGFz czEuY2xpZW50LmNhLmNydDAjBgNVHRIEHDAahhhodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS8wDQYJ KoZIhvcNAQELBQADggEBAHsw8/Hw07gsNTKYnld74NBFtHnQOPkXYuccWx3j0PGQe9nqNxeingBf 2yvx+xBQzBoi4J1u84Jbrbe8Ii3+LLD/QMW9cN0SBIgRStPQLVee4STdjeabGmpXQa7omC02wYYO 83qh6CgJEIbmrsBSZH8ZSVrjkC4UmZS8wAQMS3qTWAPF0ZQGWx2+Gks2fXuacyt2LpNR+p9ogjAZ 1/rmUKjNhQZLswytaLRUdwAwSfQ3+TNs68h6Kv1LC3bNGBT3NEtr2q/nzzb5MzuFcDE6f9exroAC 4BHmokAprhna/vZdb6BrPjpXgRAlWAh3wEMxw75M9S/Nbzj/jNp+I+lvUJYwggY0MIIEHKADAgEC AgEeMA0GCSqGSIb3DQEBBQUAMH0xCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMSkwJwYDVQQDEyBT dGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAeFw0wNzEwMjQyMTAxNTVaFw0xNzEwMjQy MTAxNTVaMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xh c3MgMSBQcmltYXJ5IEludGVybWVkaWF0ZSBDbGllbnQgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IB DwAwggEKAoIBAQDHCYPMzi3YGrEppC4Tq5a+ijKDjKaIQZZVR63UbxIP6uq/I0fhCu+cQhoUfE6E RKKnu8zPf1Jwuk0tsvVCk6U9b+0UjM0dLep3ZdE1gblK/1FwYT5Pipsu2yOMluLqwvsuz9/9f1+1 PKHG/FaR/wpbfuIqu54qzHDYeqiUfsYzoVflR80DAC7hmJ+SmZnNTWyUGHJbBpA8Q89lGxahNvur yGaC/o2/ceD2uYDX9U8Eg5DpIpGQdcbQeGarV04WgAUjjXX5r/2dabmtxWMZwhZna//jdiSyrrSM TGKkDiXm6/3/4ebfeZuCYKzN2P8O2F/Xe2AC/Y7zeEsnR7FOp+uXAgMBAAGjggGtMIIBqTAPBgNV HRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUU3Ltkpzg2ssBXHx+ljVO8tS4 UYIwHwYDVR0jBBgwFoAUTgvvGqRAW6UXaYcwyjRoQ9BBrvIwZgYIKwYBBQUHAQEEWjBYMCcGCCsG AQUFBzABhhtodHRwOi8vb2NzcC5zdGFydHNzbC5jb20vY2EwLQYIKwYBBQUHMAKGIWh0dHA6Ly93 d3cuc3RhcnRzc2wuY29tL3Nmc2NhLmNydDBbBgNVHR8EVDBSMCegJaAjhiFodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9zZnNjYS5jcmwwJ6AloCOGIWh0dHA6Ly9jcmwuc3RhcnRzc2wuY29tL3Nmc2Nh LmNybDCBgAYDVR0gBHkwdzB1BgsrBgEEAYG1NwECATBmMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3 LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3Ns LmNvbS9pbnRlcm1lZGlhdGUucGRmMA0GCSqGSIb3DQEBBQUAA4ICAQAKgwh9eKssBly4Y4xerhy5 I3dNoXHYfYa8PlVLL/qtXnkFgdtY1o95CfegFJTwqBBmf8pyTUnFsukDFUI22zF5bVHzuJ+GxhnS qN2sD1qetbYwBYK2iyYA5Pg7Er1A+hKMIzEzcduRkIMmCeUTyMyikfbUFvIBivtvkR8ZFAk22BZy +pJfAoedO61HTz4qSfQoCRcLN5A0t4DkuVhTMXIzuQ8CnykhExD6x4e6ebIbrjZLb7L+ocR0y4Yj Cl/Pd4MXU91y0vTipgr/O75CDUHDRHCCKBVmz/Rzkc/b970MEeHt5LC3NiWTgBSvrLEuVzBKM586 YoRD9Dy3OHQgWI270g+5MYA8GfgI/EPT5G7xPbCDz+zjdH89PeR3U4So4lSXur6H6vp+m9TQXPF3 a0LwZrp8MQ+Z77U1uL7TelWO5lApsbAonrqASfTpaprFVkL4nyGH+NHST2ZJPWIBk81i6Vw0ny0q ZW2Niy/QvVNKbb43A43ny076khXO7cNbBIRdJ/6qQNq9Bqb5C0Q5nEsFcj75oxQRqlKf6TcvGbjx kJh8BYtv9ePsXklAxtm8J7GCUBthHSQgepbkOexhJ0wP8imUkyiPHQ0GvEnd83129fZjoEhdGwXV 27ioRKbj/cIq7JRXun0NbeY+UdMYu9jGfIpDLtUUGSgsg2zMGs5R4jCCB8kwggWxoAMCAQICAQEw DQYJKoZIhvcNAQEFBQAwfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzAp BgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0 Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzNloXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3Vy ZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRpZmlj YXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4 n5V7tTOQ8nJi1sE2ICIkXs7pd/JDCqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZ Z7bEBn0KnnSF1nlMgDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N 3gs7SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5+kbWEyqo uaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2ROFzYh87xL6gU1YlbFEJ 96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWpW+pDDAL/mevobE1wWyllnN2qXcyvATHs DOvSjejqnHvmbvcnZgwaSNduQuM/3iE+e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B 7hIQDcYyfxj//PT6zW6R6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/ kLfjkdLd78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg8nov gdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQTUksCAwEAAaOCAlIw ggJOMAwGA1UdEwQFMAMBAf8wCwYDVR0PBAQDAgGuMB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD 0EGu8jBkBgNVHR8EXTBbMCygKqAohiZodHRwOi8vY2VydC5zdGFydGNvbS5vcmcvc2ZzY2EtY3Js LmNybDAroCmgJ4YlaHR0cDovL2NybC5zdGFydGNvbS5vcmcvc2ZzY2EtY3JsLmNybDCCAV0GA1Ud IASCAVQwggFQMIIBTAYLKwYBBAGBtTcBAQEwggE7MC8GCCsGAQUFBwIBFiNodHRwOi8vY2VydC5z dGFydGNvbS5vcmcvcG9saWN5LnBkZjA1BggrBgEFBQcCARYpaHR0cDovL2NlcnQuc3RhcnRjb20u b3JnL2ludGVybWVkaWF0ZS5wZGYwgdAGCCsGAQUFBwICMIHDMCcWIFN0YXJ0IENvbW1lcmNpYWwg KFN0YXJ0Q29tKSBMdGQuMAMCAQEagZdMaW1pdGVkIExpYWJpbGl0eSwgcmVhZCB0aGUgc2VjdGlv biAqTGVnYWwgTGltaXRhdGlvbnMqIG9mIHRoZSBTdGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhv cml0eSBQb2xpY3kgYXZhaWxhYmxlIGF0IGh0dHA6Ly9jZXJ0LnN0YXJ0Y29tLm9yZy9wb2xpY3ku cGRmMBEGCWCGSAGG+EIBAQQEAwIABzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wg Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkwDQYJKoZIhvcNAQEFBQADggIBABZsmfRmDDT10IVefQrs 2hBOOBxe36YlBUuRMsHoO/E93UQJWwdJiinLZgK3sZr3JZgJPI4b4d02hytLu2jTOWY9oCbH8jmR HVGrgnt+1c5a5OIDV3Bplwj5XlimCt+MBppFFhY4Cl5X9mLHegIF5rwetfKe9Kkpg/iyFONuKIdE w5Aa3jipPKxDTWRFzt0oqVzyc3sE+Bfoq7HzLlxkbnMxOhK4vLMR5H2PgVGaO42J9E2TZns8A+3T mh2a82VQ9aDQdZ8vr/DqgkOY+GmciXnEQ45GcuNkNhKv9yUeOImQd37Da2q5w8tES6x4kIvnxywe SxFEyDRSJ80KXZ+FwYnVGnjylRBTMt2AhGZ12bVoKPthLr6EqDjAmRKGpR5nZK0GLi+pcIXHlg98 iWX1jkNUDqvdpYA5lGDANMmWcCyjEvUfSHu9HH5rt52Q9CI7rvj8Ksr6glKg769LVZPrwbXwIous NE4mIgShhyx1SrflfRPXuAxkwDbSyS+GEowjCcEbgjtzSaNqV4eU5dZ4xZlDY+NN4Hct4WWZcmkE GkcJ5g8BViT7H78OealYLrnECQF+lbptAAY+supKEDnY0Cv1v+x1v5cCxQkbCNxVN+KB+zeEQ2Ig yudWS2Xq/mzBJJMkoTTrBf+aIq6bfT/xZVEKpjBqs/SIHIAN/HKK6INeMYIDbzCCA2sCAQEwgZQw gYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUg RGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFBy aW1hcnkgSW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMAkGBSsOAwIaBQCgggGvMBgGCSqGSIb3 DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE0MDIxNDAzMjQzNVowIwYJKoZIhvcN AQkEMRYEFDGGPv6hogiWrNccLz/tj9gXmebDMIGlBgkrBgEEAYI3EAQxgZcwgZQwgYwxCzAJBgNV BAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBD ZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50 ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMIGnBgsqhkiG9w0BCRACCzGBl6CBlDCBjDELMAkGA1UE BhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENl cnRpZmljYXRlIFNpZ25pbmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRl cm1lZGlhdGUgQ2xpZW50IENBAgMGoo4wDQYJKoZIhvcNAQEBBQAEggEARnUiPZ4zkiVIgPU0MKec rU+oIBY6zFTCoGIp9ymCKwbqj1B/O2cTXcqc8lyDTGxdRJvLQGRgaFk1G4L/fZSTcEYrduDhKubH tNZvN1zGdgCd7K/x8LpXrcOwUo/OsRCK2bRbYSsZaHbddjLcCEKyoCn/e5lQWcGsLt42jGYJENr2 wYkJPOv/X1LghOFbkR8x0OgS2xZdXEbAOcTugiyrw9yk/kN1cEblBy0qwGW9fcJQEdRSAG6ekpx3 Am6Z7hDVXeESjq5cRlLNHt4xilv6bOPHkp7pqSzKijXV4P79D1sbb0jwH/xzOLF+NbOq5WLXGMFr 03UmFkvyMS5UQm3UiAAAAAAAAA== --Apple-Mail-DE296154-25EF-470F-8900-A78FBC004AF7-- From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 03:48:03 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 55874FFA for ; Fri, 14 Feb 2014 03:48:03 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0DF7518C9 for ; Fri, 14 Feb 2014 03:48:02 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.7/8.14.7) with ESMTP id s1E3lqMd008621 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 13 Feb 2014 19:47:55 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <52FD91E2.4080903@freebsd.org> Date: Fri, 14 Feb 2014 11:47:46 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Alfred Perlstein , Vijay Singh Subject: Re: Debugging rw lock References: <52FD30D9.6050604@mu.org> <52FD32FF.70106@mu.org> In-Reply-To: <52FD32FF.70106@mu.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 03:48:03 -0000 On 2/14/14, 5:02 AM, Alfred Perlstein wrote: > > On 2/13/14, 12:59 PM, Vijay Singh wrote: >> You're talking about instrumenting the code, right? But which >> thread? I was >> thinking of augmenting the rw lock to record the readers, but >> wanted to >> check if something is possible without instrumentation. > > If rw locks are implemented using low level atomics then you're > going to make the very slow and have a LOT of work to do as opposed > to just using a thread specific storage to implement it. You're > better off just making use of the fact that only "curthread" can > access the per-thread stack and just use that. Or at least that's > how it works in my brain. maybe do it the other way around and count the read locks a thread has.. then when there is a problem go see what they hold.. (or add the ability to detail 10 locks each or something) > > -Alfred > > >> >> >> On Thu, Feb 13, 2014 at 12:53 PM, Alfred Perlstein >> wrote: >> >>> Keep a stack of rwlocks owned in the struct thread. >>> >>> -Alfred >>> >>> On 2/13/14, 12:51 PM, Vijay Singh wrote: >>> >>>> I am running into an issue where an rw lock is read locked and never >>>> unlocked, and causes a system to livelock. I was wondering if its >>>> possible >>>> to figure out which thread owns the read lock? >>>> >>>> It's the tcp pcbinfo lock. >>>> >>>> (kgdb-amd64-7.4-08) show_rwlock rw >>>> name : tcp >>>> class: rw >>>> flags: {SLEEP, INITED, WITNESS, RECURSE, UPGRADABLE} >>>> state: RLOCK: 1 locks >>>> waiters: writers >>>> >>>> Any help is appreciated. >>>> >>>> -vijay >>>> _______________________________________________ >>>> freebsd-hackers@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>>> To unsubscribe, send any mail to >>>> "freebsd-hackers-unsubscribe@freebsd.org >>>> " >>>> >>>> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to >> "freebsd-hackers-unsubscribe@freebsd.org" >> > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 05:38:03 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 63C05101 for ; Fri, 14 Feb 2014 05:38:03 +0000 (UTC) Received: from mail-oa0-f48.google.com (mail-oa0-f48.google.com [209.85.219.48]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2406010CA for ; Fri, 14 Feb 2014 05:38:02 +0000 (UTC) Received: by mail-oa0-f48.google.com with SMTP id l6so13988267oag.35 for ; Thu, 13 Feb 2014 21:37:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=4QWLzZORTjSAzfx9gnB5cJwJfYlDR8w1xYs0PgB7HoI=; b=dW43OXdrjraSsrd1cKA+BE8Kz1+C+AgFkOkfbsUGIyqlTwbWXJOrcpr+isTAp+eNqV TNGrDoPXPi6P6mfMy1WC3cPawk8fazzx2Ny97fFG8bZ00SR4Nnl6YwfhFhHiohtyekBj 07jMgU5nSnLy40348OT3WWFghYYic7KviMLf1LfQns6U2VL+f6de+MlmgN6uylWaeQRP X58zGQDbcbWkn3GU7uK7QbmlXKsmN/My/oEA2rKTaeKWx4I+qMyXY/kpUNP0it+A+0Qx cbjO8DVKJ7RtuCF/3NHAkXJ5GIpFBFHSrgoFWf8Oo6EAv/dYBte1peslJDBt82MTNLtA AKhQ== X-Gm-Message-State: ALoCoQmXcvxGXrawZmRrymW796TEgpJTAQTJnlcK76GbWWDJSEWjfBhTfQou/AzgxE19jvF2XPmB X-Received: by 10.60.231.194 with SMTP id ti2mr4776993oec.41.1392356276289; Thu, 13 Feb 2014 21:37:56 -0800 (PST) Received: from [172.21.0.93] (67-198-60-238.static.grandenetworks.net. [67.198.60.238]) by mx.google.com with ESMTPSA id tr7sm29054794oec.0.2014.02.13.21.37.49 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 13 Feb 2014 21:37:50 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Thoughts on Multi-Symlink Concept From: Jim Thompson In-Reply-To: Date: Thu, 13 Feb 2014 23:37:49 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <40637D5C-EBBB-49C0-BE82-BA644C32D778@netgate.com> References: To: Jordan Hubbard X-Mailer: Apple Mail (2.1827) Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 05:38:03 -0000 On Feb 12, 2014, at 11:17 PM, Jordan Hubbard = wrote: >=20 > On Feb 12, 2014, at 9:08 PM, Jordan Hubbard = wrote: >=20 >> Globbing is done in user land (by the shell) - you wouldn=92t want to = push that down into the kernel, which is either what you=92d have to do = or you=92d need a user land daemon which did round-trips with the kernel = to do the translation, which would also need to make sure to get all of = the process permission stuff right since the user id / gid / $CWD would = all potentially affect the expansion of the =93symlink=94. >=20 > Actually, just to correct myself, there is a third way, which is that = you could make the shell also do the expansion of the symlink (or = interpose it into libc), but now you=92d just be stacking one weird hack = on top of another weird hack. It=92s still not a good idea for all the = reasons I mentioned, at least not as a =93symlink=94. Maybe some new = type of shell builtin, though I=92m not sure how/where you=92d use it. There is a fourth way. Just embed a (call to a) shell script in the = symlink. If Pyramid=92s conditional symbolic links were the computed = goto of the filesystem, and variant symlinks are a modern .. er=85 = variant of same, =20 then embedding a call to an external program has to be the = Turing-complete =85 er=85 variant. amiright? What could go wrong? :-) OK, seriously.. once upon a time there was an OS named Mach. It had = the concept of a name server. Pathname resolution worked like this: consider /mnt/readme.txt where /mnt is a mounted filesystem. =95 libc asks the root filesystem server about /mnt/readme.txt. =95 The root filesystem returns a port to the mnt filesystem = server (matching /mnt) and the retry name /readme.txt. =95 libc asks the mnt filesystem server about /readme.txt. =95 The mnt filesystem server returns a port to itself and = records that this port refers to the regular file /readme.txt. While a regular filesystem server will just serve the data as stored in = a filesystem on disk, on Mach there can be servers providing purely = virtual information, or a mixture of both. In Mach-land, it is up to the server to behave and provide consistent = and useful data on each remote procedure call. If it does not, the = results may not match the expectations of the user and confuse him. You are lost in a maze of twisty little filesystems, all alike=85. OK, more seriously. This seems straight-forward to implement via FUSE. You might look at = fsfipl & filterFS http://sourceforge.net/projects/fsfipi/ http://filterfs.sourceforge.net.= From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 09:22:35 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B7D30568; Fri, 14 Feb 2014 09:22:35 +0000 (UTC) Received: from mail-vc0-x230.google.com (mail-vc0-x230.google.com [IPv6:2607:f8b0:400c:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5BE6E1222; Fri, 14 Feb 2014 09:22:35 +0000 (UTC) Received: by mail-vc0-f176.google.com with SMTP id la4so9138450vcb.35 for ; Fri, 14 Feb 2014 01:22:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ivC0a4n2HHy8ApjJC3TkB6Ds8nhM2LrQ8JT0ClCKB40=; b=JzoyMgcYDC5ghKjOx3kfR+8LvG9d8d3fYW+gknqJVQZVCJSgNVqIo38fRgL/t5Dlj+ OZT2ZkQzDeLUEyTsCKB6s/JEM2C+5h1R5Xnatcljkp85Eb4BOcJESE8UPTLUdNi0q9Ds eOQ7y34SPUEnm0h879JmRPrIaKHfjlBLtCjLelph6G8WafHKLoJz0lRbjpabQhzd6Kq/ VdNwiPJVX/jhI8KpUSNDL+bDkXwxI4xoVdcgEFFlOw8V+cIX7gVNNB31iUPuvRBZ7Yhm +QgPnClFP954GalGd3PTBAHHAnEGneq4Kfa+8LcoOI2ivWdK59Z2ONi8N9axSO2o+lyo 0xXw== MIME-Version: 1.0 X-Received: by 10.52.247.231 with SMTP id yh7mr325212vdc.34.1392369754283; Fri, 14 Feb 2014 01:22:34 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.220.113.199 with HTTP; Fri, 14 Feb 2014 01:22:34 -0800 (PST) In-Reply-To: References: Date: Fri, 14 Feb 2014 01:22:34 -0800 X-Google-Sender-Auth: R5K41CJg3nunNfEWN3a5V0YcGOk Message-ID: Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? From: Adrian Chadd To: Ryan Stone Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 09:22:35 -0000 Ok, so now I remember the other odd thing. I was seeing the sending context(s) jumping from one CPU to another during flowtable_insert_common(), around the locking bits. But I thread pinned all the sender user threads! So, why would the senders still be scheduled on other CPUs if I've pinned the userland threads? (and yes, I verified that the userland threads weren't moving around.) -a From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 17:51:34 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D04A11B; Fri, 14 Feb 2014 17:51:34 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5D8021453; Fri, 14 Feb 2014 17:51:34 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id C96A7B917; Fri, 14 Feb 2014 12:51:30 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? Date: Fri, 14 Feb 2014 11:39:48 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201402141139.49158.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 14 Feb 2014 12:51:30 -0500 (EST) Cc: "freebsd-hackers@freebsd.org" , Adrian Chadd , Ryan Stone X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 17:51:34 -0000 On Friday, February 14, 2014 4:22:34 am Adrian Chadd wrote: > Ok, so now I remember the other odd thing. > > I was seeing the sending context(s) jumping from one CPU to another > during flowtable_insert_common(), around the locking bits. > > But I thread pinned all the sender user threads! > > So, why would the senders still be scheduled on other CPUs if I've > pinned the userland threads? > > (and yes, I verified that the userland threads weren't moving around.) Can you clarify a bit? It's not clear how sender thraeds differ from userland threads differ from sender user threads. (I.e. one reading is that these are all the same thing and should thus all be pinned (I assume you mean using cpuset to bind them to specific cores rather than sched_pin)) -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 17:55:05 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3B8135BC; Fri, 14 Feb 2014 17:55:05 +0000 (UTC) Received: from mail-qc0-x229.google.com (mail-qc0-x229.google.com [IPv6:2607:f8b0:400d:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C63C714B0; Fri, 14 Feb 2014 17:55:04 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id w7so20684898qcr.28 for ; Fri, 14 Feb 2014 09:55:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=bjmYs8IWxXTAY/Adfo+BHnmFW552Egd/N3gmaf2V0MI=; b=NykmRvtWx1snhjJvcyRxii/54BtRZKYfkDCyrvo9i6vqO/alk1+yy3NutonXZWiRnl GgCFuBJhRFMmQnoC0D3rcgH/Xp7Nt3/G7CSNxyxcmf1zyjw1ftQBlnRtgNmg2fb3mRHB /WA/4j+vmT3A49WSrV3UeLzD+WY71cQKZxgUnqRrEoWRzq7aUGGomuNvdA2EWlSTS3c1 9V9+GgpuYdQNQq9EuY0b00PY/S3A7PrWHonmrzQD2dk78S++9Sq6dFkhpJI5jP5AKyhc HrVe66T3Ezj4iC9nUK4zDIBQaIb1GFsxwBtmuZpiolMpPX5F0mlGeWMsLE6VJyokWrO+ 3qFg== MIME-Version: 1.0 X-Received: by 10.224.11.136 with SMTP id t8mr15755403qat.26.1392400503873; Fri, 14 Feb 2014 09:55:03 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.16.10 with HTTP; Fri, 14 Feb 2014 09:55:03 -0800 (PST) In-Reply-To: <201402141139.49158.jhb@freebsd.org> References: <201402141139.49158.jhb@freebsd.org> Date: Fri, 14 Feb 2014 09:55:03 -0800 X-Google-Sender-Auth: CZZ-aGEmfEN_lLThkm6dptv_s7U Message-ID: Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? From: Adrian Chadd To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" , Ryan Stone , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 17:55:05 -0000 On 14 February 2014 08:39, John Baldwin wrote: > On Friday, February 14, 2014 4:22:34 am Adrian Chadd wrote: >> Ok, so now I remember the other odd thing. >> >> I was seeing the sending context(s) jumping from one CPU to another >> during flowtable_insert_common(), around the locking bits. >> >> But I thread pinned all the sender user threads! >> >> So, why would the senders still be scheduled on other CPUs if I've >> pinned the userland threads? >> >> (and yes, I verified that the userland threads weren't moving around.) > > Can you clarify a bit? It's not clear how sender thraeds differ from > userland threads differ from sender user threads. (I.e. one reading > is that these are all the same thing and should thus all be pinned > (I assume you mean using cpuset to bind them to specific cores rather > than sched_pin)) Yup, I'm doing a manual, poor-mans RSS in lieu of merging in roberts stuff: * the userland threads are using the cpuset call to map a thread into a cpuset, yes * the NIC TX/RX ring routines in cxgbe are pinned to the same CPU as the userland threads -a From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 18:18:51 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9E827E0D; Fri, 14 Feb 2014 18:18:51 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 71C2116EC; Fri, 14 Feb 2014 18:18:51 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 7E401B9A3; Fri, 14 Feb 2014 13:18:50 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Subject: Re: Debugging rw lock Date: Fri, 14 Feb 2014 12:59:15 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <52FD32FF.70106@mu.org> <52FD91E2.4080903@freebsd.org> In-Reply-To: <52FD91E2.4080903@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201402141259.15712.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 14 Feb 2014 13:18:50 -0500 (EST) Cc: hackers@freebsd.org, Alfred Perlstein , Vijay Singh X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 18:18:51 -0000 On Thursday, February 13, 2014 10:47:46 pm Julian Elischer wrote: > On 2/14/14, 5:02 AM, Alfred Perlstein wrote: > > > > On 2/13/14, 12:59 PM, Vijay Singh wrote: > >> You're talking about instrumenting the code, right? But which > >> thread? I was > >> thinking of augmenting the rw lock to record the readers, but > >> wanted to > >> check if something is possible without instrumentation. > > > > If rw locks are implemented using low level atomics then you're > > going to make the very slow and have a LOT of work to do as opposed > > to just using a thread specific storage to implement it. You're > > better off just making use of the fact that only "curthread" can > > access the per-thread stack and just use that. Or at least that's > > how it works in my brain. > > maybe do it the other way around and count the read locks a thread > has.. then when there is a problem go see what they hold.. > (or add the ability to detail 10 locks each or something) We already do that. Vijay, you can look at td_rw_rlocks in each struct thread. You can at least ignore the threads with a count set to zero. The other option is to run with WITNESS enabled. You can then do 'show all locks' in ddb and you can see which thread owns the lock (and the file and line number where it acquired it) -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 18:18:54 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C9364E11; Fri, 14 Feb 2014 18:18:54 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 96C4416EE; Fri, 14 Feb 2014 18:18:54 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 92A71B9DF; Fri, 14 Feb 2014 13:18:53 -0500 (EST) From: John Baldwin To: Adrian Chadd Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? Date: Fri, 14 Feb 2014 13:18:44 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <201402141139.49158.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201402141318.44743.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 14 Feb 2014 13:18:53 -0500 (EST) Cc: "freebsd-hackers@freebsd.org" , Ryan Stone , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 18:18:54 -0000 On Friday, February 14, 2014 12:55:03 pm Adrian Chadd wrote: > On 14 February 2014 08:39, John Baldwin wrote: > > On Friday, February 14, 2014 4:22:34 am Adrian Chadd wrote: > >> Ok, so now I remember the other odd thing. > >> > >> I was seeing the sending context(s) jumping from one CPU to another > >> during flowtable_insert_common(), around the locking bits. > >> > >> But I thread pinned all the sender user threads! > >> > >> So, why would the senders still be scheduled on other CPUs if I've > >> pinned the userland threads? > >> > >> (and yes, I verified that the userland threads weren't moving around.) > > > > Can you clarify a bit? It's not clear how sender thraeds differ from > > userland threads differ from sender user threads. (I.e. one reading > > is that these are all the same thing and should thus all be pinned > > (I assume you mean using cpuset to bind them to specific cores rather > > than sched_pin)) > > Yup, I'm doing a manual, poor-mans RSS in lieu of merging in roberts stuff: > > * the userland threads are using the cpuset call to map a thread into > a cpuset, yes > * the NIC TX/RX ring routines in cxgbe are pinned to the same CPU as > the userland threads If they are all cpuset to a single CPU, they should not migrate, though I think sched_bind() can override that. However, that requires code to explicitly call sched_bind() which should be rare. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 18:26:16 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A20F03E0 for ; Fri, 14 Feb 2014 18:26:16 +0000 (UTC) Received: from mail-la0-f52.google.com (mail-la0-f52.google.com [209.85.215.52]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1A05317AA for ; Fri, 14 Feb 2014 18:26:15 +0000 (UTC) Received: by mail-la0-f52.google.com with SMTP id c6so9682444lan.11 for ; Fri, 14 Feb 2014 10:26:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=lxweHO6XgFZ880npI09YRyplbMA5yctEUxQ5ZIF0V+M=; b=adHrlMglcpjjxYrpcRIOFGhjVSIjFuz0H0B+QeLq0kd5drV0HNvmFZohf4ZZiZSiWX oz3L89g2/wfip6xExhBmAqJwTZcbD0JNuGs85PgawXEZl3tY3PVazhqw74QEEm8uXlzU gi8O9gWUCzedQDgHm788H8O6QoJhNEEF3dAC34h+EcIueBfe84iVk15Q3Q5Qd2wr+Ugv pnAO8v2W1VGzQbXXFRD7kxEBltsuLoA9Gm61zPu43DK53ejxJ5ckA93qKp7/SUcJJ4OE XUFB81T4KoaBqbQVD3PfL3J7JgBir1W/yra/7Zv8Grr0cdefmdLTDRm1EJnToI3zg8zo lYQw== X-Gm-Message-State: ALoCoQl5OJKtqwVsYRaw1id8KU5E+WWl2IfAEFPXlLLCuTqFkmx0AlGh+Vr7LwAI8WLbPk13lMNu X-Received: by 10.112.160.161 with SMTP id xl1mr10184lbb.71.1392402368412; Fri, 14 Feb 2014 10:26:08 -0800 (PST) Received: from [192.168.1.2] ([89.169.173.68]) by mx.google.com with ESMTPSA id cu8sm6711323lbb.12.2014.02.14.10.26.07 for (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 14 Feb 2014 10:26:07 -0800 (PST) Message-ID: <52FE5FBF.3090104@freebsd.org> Date: Fri, 14 Feb 2014 22:26:07 +0400 From: Andrey Chernov User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: John Baldwin , Adrian Chadd Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? References: <201402141139.49158.jhb@freebsd.org> <201402141318.44743.jhb@freebsd.org> In-Reply-To: <201402141318.44743.jhb@freebsd.org> X-Enigmail-Version: 1.7a1pre Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" , Ryan Stone , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 18:26:16 -0000 On 14.02.2014 22:18, John Baldwin wrote: > On Friday, February 14, 2014 12:55:03 pm Adrian Chadd wrote: >> On 14 February 2014 08:39, John Baldwin wrote: >>> On Friday, February 14, 2014 4:22:34 am Adrian Chadd wrote: >>>> Ok, so now I remember the other odd thing. >>>> >>>> I was seeing the sending context(s) jumping from one CPU to another >>>> during flowtable_insert_common(), around the locking bits. >>>> >>>> But I thread pinned all the sender user threads! >>>> >>>> So, why would the senders still be scheduled on other CPUs if I've >>>> pinned the userland threads? >>>> >>>> (and yes, I verified that the userland threads weren't moving around.) >>> >>> Can you clarify a bit? It's not clear how sender thraeds differ from >>> userland threads differ from sender user threads. (I.e. one reading >>> is that these are all the same thing and should thus all be pinned >>> (I assume you mean using cpuset to bind them to specific cores rather >>> than sched_pin)) >> >> Yup, I'm doing a manual, poor-mans RSS in lieu of merging in roberts stuff: >> >> * the userland threads are using the cpuset call to map a thread into >> a cpuset, yes >> * the NIC TX/RX ring routines in cxgbe are pinned to the same CPU as >> the userland threads > > If they are all cpuset to a single CPU, they should not migrate, though > I think sched_bind() can override that. However, that requires code to > explicitly call sched_bind() which should be rare. > Due to this bug, not fixed yet, the real picture is more complex: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/163585 -- http://ache.vniz.net/ From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 19:10:44 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 700EE2D6; Fri, 14 Feb 2014 19:10:44 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 434551B9F; Fri, 14 Feb 2014 19:10:44 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 990C7B9A0; Fri, 14 Feb 2014 14:10:40 -0500 (EST) From: John Baldwin To: Andrey Chernov Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? Date: Fri, 14 Feb 2014 14:10:29 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <201402141318.44743.jhb@freebsd.org> <52FE5FBF.3090104@freebsd.org> In-Reply-To: <52FE5FBF.3090104@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="koi8-r" Content-Transfer-Encoding: 7bit Message-Id: <201402141410.29325.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 14 Feb 2014 14:10:40 -0500 (EST) Cc: "freebsd-hackers@freebsd.org" , Adrian Chadd , Ryan Stone , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 19:10:44 -0000 On Friday, February 14, 2014 1:26:07 pm Andrey Chernov wrote: > On 14.02.2014 22:18, John Baldwin wrote: > > On Friday, February 14, 2014 12:55:03 pm Adrian Chadd wrote: > >> On 14 February 2014 08:39, John Baldwin wrote: > >>> On Friday, February 14, 2014 4:22:34 am Adrian Chadd wrote: > >>>> Ok, so now I remember the other odd thing. > >>>> > >>>> I was seeing the sending context(s) jumping from one CPU to another > >>>> during flowtable_insert_common(), around the locking bits. > >>>> > >>>> But I thread pinned all the sender user threads! > >>>> > >>>> So, why would the senders still be scheduled on other CPUs if I've > >>>> pinned the userland threads? > >>>> > >>>> (and yes, I verified that the userland threads weren't moving around.) > >>> > >>> Can you clarify a bit? It's not clear how sender thraeds differ from > >>> userland threads differ from sender user threads. (I.e. one reading > >>> is that these are all the same thing and should thus all be pinned > >>> (I assume you mean using cpuset to bind them to specific cores rather > >>> than sched_pin)) > >> > >> Yup, I'm doing a manual, poor-mans RSS in lieu of merging in roberts stuff: > >> > >> * the userland threads are using the cpuset call to map a thread into > >> a cpuset, yes > >> * the NIC TX/RX ring routines in cxgbe are pinned to the same CPU as > >> the userland threads > > > > If they are all cpuset to a single CPU, they should not migrate, though > > I think sched_bind() can override that. However, that requires code to > > explicitly call sched_bind() which should be rare. > > > > Due to this bug, not fixed yet, the real picture is more complex: > http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/163585 Eh, that bug report has no useful details, as in, it doesn't list the actual commands run. If you do 'cpuset -l 6 -s 1' to force all processes to only use CPU6, then yes, of course the other CPUs are idle because that's what you _asked_ for. AFAICT, that is all the original reporter did. At work we regularly add and remove CPUs from the default set (set 1) on hundreds of machines every day with ULE without any issues. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 19:18:31 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5809B5DE; Fri, 14 Feb 2014 19:18:31 +0000 (UTC) Received: from mail-qc0-x235.google.com (mail-qc0-x235.google.com [IPv6:2607:f8b0:400d:c01::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E3E271BF4; Fri, 14 Feb 2014 19:18:30 +0000 (UTC) Received: by mail-qc0-f181.google.com with SMTP id e9so20437027qcy.26 for ; Fri, 14 Feb 2014 11:18:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=jsS5cQgxKWF0vUwePrX33qb3N38GlytJ045P/Pc/MyM=; b=ZrVoWZAu+Zzsd450YJPyL8vJShIYYS3yNeIHfPHDBcFk+GNQ89Es9lG5k2t1ek0+I3 un/88l3ydkG2Z+Z6gMq4mvzed9ZFgCyRGD/OoWg0/9Rwtz4JlWv6/3DH7id9CwEqtqwp M/wGlljgaBCERAfGWLGIKcHqBx4i/4rYBDpwrd9xigN6gHJDX5Fp5GHJGGd1xKz3opdP iCLuTwXFlQ/DBDrVswPcUWoc2qZmkmBHYazuRydLkgM94V43d0b5wYyeki/y8qTiKhfW sD3XefnghiAgVa+kvAAu4OcH34m0k12NIyKqhdQ627x/1AxWJIX77S1iySretoCiYX2c vWhw== MIME-Version: 1.0 X-Received: by 10.229.179.5 with SMTP id bo5mr16190937qcb.21.1392405510104; Fri, 14 Feb 2014 11:18:30 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.16.10 with HTTP; Fri, 14 Feb 2014 11:18:30 -0800 (PST) In-Reply-To: <201402141318.44743.jhb@freebsd.org> References: <201402141139.49158.jhb@freebsd.org> <201402141318.44743.jhb@freebsd.org> Date: Fri, 14 Feb 2014 11:18:30 -0800 X-Google-Sender-Auth: QEybOPM7Tbhp6qOcVV4ns4tncyg Message-ID: Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? From: Adrian Chadd To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" , Ryan Stone , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 19:18:31 -0000 On 14 February 2014 10:18, John Baldwin wrote: > If they are all cpuset to a single CPU, they should not migrate, though > I think sched_bind() can override that. However, that requires code to > explicitly call sched_bind() which should be rare. Yup. That's why I'm confused. I'm rebuilding -HEAD now with the latest flowtable changes. I'll add in my debugging afterward and trigger the particular scenario where it's behaving badly and do some more diagnostics. Thanks, -a From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 22:08:47 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5DD46DFB for ; Fri, 14 Feb 2014 22:08:47 +0000 (UTC) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DEDD41BB6 for ; Fri, 14 Feb 2014 22:08:46 +0000 (UTC) Received: by mail-ee0-f54.google.com with SMTP id e53so5970774eek.13 for ; Fri, 14 Feb 2014 14:08:39 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=D6/BAheko3zijfo+zJbs/3UvzngaykW0h1HWOqOH22g=; b=Cv3TO5WsMo4LRsTEaohahFQ5bkkLJgfzxemiRJ26JWOme05eeumWIrWPHQMysN6sSW YSykjEeXAlfxIJ99JuZhStF7OROmznOizURe4c5oK0AdS08kUI8Fu0p2oTB2V1wZ5nJF 5Ni9FGdT09QXqVc6ntUPD6gsF5MnDu+Bf3dMDI62ozoxUwqwYhS9jXrq9PECeaavJl11 kIQwSv7CHEDNXqsXNDKg3Pb1diU9o35YD+NIOQi8xd3JTtGCAu75ztLc/7RVtzQ+y7sf apYqEUBczrdiM3tZ1WkWGerkfR+yUZZA9VIikT7Zsf0j5tJ7PoKZ0uGOK08twJ+/JlZP uuiw== X-Gm-Message-State: ALoCoQmHxGqJBt3EfCFFKQRxAiXdCVcaXpuk/SahAOEyqiki+CkrEg4ORhk/iEzZZlPRbe1pF0WT X-Received: by 10.14.203.197 with SMTP id f45mr2188243eeo.90.1392415719153; Fri, 14 Feb 2014 14:08:39 -0800 (PST) Received: from [192.168.1.2] ([89.169.173.68]) by mx.google.com with ESMTPSA id y47sm24635115eel.14.2014.02.14.14.08.38 for (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 14 Feb 2014 14:08:38 -0800 (PST) Message-ID: <52FE93E6.6030705@freebsd.org> Date: Sat, 15 Feb 2014 02:08:38 +0400 From: Andrey Chernov User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: John Baldwin Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? References: <201402141318.44743.jhb@freebsd.org> <52FE5FBF.3090104@freebsd.org> <201402141410.29325.jhb@freebsd.org> In-Reply-To: <201402141410.29325.jhb@freebsd.org> X-Enigmail-Version: 1.7a1pre Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" , Adrian Chadd , Ryan Stone , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 22:08:47 -0000 On 14.02.2014 23:10, John Baldwin wrote: >> Due to this bug, not fixed yet, the real picture is more complex: >> http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/163585 > > Eh, that bug report has no useful details, as in, it doesn't list the > actual commands run. If you do 'cpuset -l 6 -s 1' to force all > processes to only use CPU6, then yes, of course the other CPUs are idle > because that's what you _asked_ for. AFAICT, that is all the original > reporter did. At work we regularly add and remove CPUs from the > default set (set 1) on hundreds of machines every day with ULE without > any issues. Probably original report lack certain commands, but I provide the link to the port which reproduces this bug too. All threads there are assigned to the _different_ CPUs and appears as result on single one with SCHED_ULE (not with SCHED_4BSD). And it is what original reporter mean too. It surely happens, maybe not the first time, but on 2nd-3rd. It means that cpuset_setaffinity() is completely broken form SCHED_ULE at least for 3 years. -- http://ache.vniz.net/ From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 22:36:24 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DB65E7CF for ; Fri, 14 Feb 2014 22:36:23 +0000 (UTC) Received: from mail-ee0-f52.google.com (mail-ee0-f52.google.com [74.125.83.52]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 67B391DBB for ; Fri, 14 Feb 2014 22:36:23 +0000 (UTC) Received: by mail-ee0-f52.google.com with SMTP id e53so5966302eek.39 for ; Fri, 14 Feb 2014 14:36:15 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=x8xxOQBDIs7K3LPirTh/iZf1+TrjzpM+a/QZiY5ZdHk=; b=U0FPXCgayTaxqvbEKO2047YlEIS+J8K/Q+Mo8q+cxLFOacnX+CJ3jN3LbcFIgcCOdr D3pY8zMiDTUDO7DAwMBJDG93BE4nJ2RAv+p6voXB6Wg1xYHeHUzA/r0e/uj1nKmpTef4 e5kpFFoQlEx9ZDEkJlCHtaN04iEwSmv3ihZnKYX/HNJDutg1PD+6iTQ0PqQtXM5NPgcQ 3nUGGVYlSTyp3B1K1to+t1FgGpWe/MjDAfjk7J4BnSWHNHrfgAxcEMqBgok5nwADsZVV /ZgLrxo2JIA90ADrVujyf6A5yNXWeFa+5rA0VL1s3osR3uUXWIL8VxdmVxgSijlaXjCL fxfA== X-Gm-Message-State: ALoCoQm3ZZNOC0VHFq6lyheDNEDVAEHgdLwAftvL7or3RF8FiMKJ/UUSltIElZoBmpeEj3XOOtjN X-Received: by 10.14.87.129 with SMTP id y1mr12051788eee.38.1392417375195; Fri, 14 Feb 2014 14:36:15 -0800 (PST) Received: from [192.168.1.2] ([89.169.173.68]) by mx.google.com with ESMTPSA id 46sm24812320ees.4.2014.02.14.14.36.14 for (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 14 Feb 2014 14:36:14 -0800 (PST) Message-ID: <52FE9A5E.5050300@freebsd.org> Date: Sat, 15 Feb 2014 02:36:14 +0400 From: Andrey Chernov User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: John Baldwin Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? References: <201402141318.44743.jhb@freebsd.org> <52FE5FBF.3090104@freebsd.org> <201402141410.29325.jhb@freebsd.org> <52FE93E6.6030705@freebsd.org> In-Reply-To: <52FE93E6.6030705@freebsd.org> X-Enigmail-Version: 1.7a1pre Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" , Adrian Chadd , Ryan Stone , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 22:36:24 -0000 On 15.02.2014 2:08, Andrey Chernov wrote: > On 14.02.2014 23:10, John Baldwin wrote: > >>> Due to this bug, not fixed yet, the real picture is more complex: >>> http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/163585 >> >> Eh, that bug report has no useful details, as in, it doesn't list the >> actual commands run. If you do 'cpuset -l 6 -s 1' to force all >> processes to only use CPU6, then yes, of course the other CPUs are idle >> because that's what you _asked_ for. AFAICT, that is all the original >> reporter did. At work we regularly add and remove CPUs from the >> default set (set 1) on hundreds of machines every day with ULE without >> any issues. > > Probably original report lack certain commands, but I provide the link > to the port which reproduces this bug too. All threads there are > assigned to the _different_ CPUs and appears as result on single one > with SCHED_ULE (not with SCHED_4BSD). And it is what original reporter > mean too. It surely happens, maybe not the first time, but on 2nd-3rd. > It means that cpuset_setaffinity() is completely broken form SCHED_ULE > at least for 3 years. > This is code example from cpuminer port, in case you are interested, it is very simple: static inline void affine_to_cpu(int id, int cpu) { cpuset_t set; CPU_ZERO(&set); CPU_SET(cpu, &set); cpuset_setaffinity(CPU_LEVEL_WHICH, CPU_WHICH_CPUSET, -1, sizeof(cpuset_t), &set); } ... num_processors = sysconf(_SC_NPROCESSORS_CONF); if (!opt_n_threads) opt_n_threads = num_processors; ... In the thread itself: struct thr_info *mythr = userdata; int thr_id = mythr->id; if (num_processors > 1 && opt_n_threads % num_processors == 0) { if (!opt_quiet) applog(LOG_INFO, "Binding thread %d to cpu %d", thr_id, thr_id % num_processors); affine_to_cpu(thr_id, thr_id % num_processors); } -- http://ache.vniz.net/ From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 14 22:57:21 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 472FFBC9; Fri, 14 Feb 2014 22:57:21 +0000 (UTC) Received: from mail-qc0-x229.google.com (mail-qc0-x229.google.com [IPv6:2607:f8b0:400d:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CF900105A; Fri, 14 Feb 2014 22:57:20 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id w7so21352854qcr.0 for ; Fri, 14 Feb 2014 14:57:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=dyaQZ1O5yEnbLCb25tmt99o5o9IQoLAU+LgPmywsIvQ=; b=BUmkYv3u0Q9Xt+0CwwVZ1zjlj7/Z5XHysaSZabgZ4ieowgLRp2MlZWasAQ+0t8qGmJ +gKztJK/q3xEUaZieEQubZ43El//eQCcOgIR+ho+24CMWQgaxSpIgnN8v4fGBjWexrKR mZDsutgmTcp3vpyAJ9+/oSben9byJgSVJr7v/VLZG+xkVdunxtfSqWfoQ77mUV8IzNfz 67g4stqyIrqyv6PXnl6qFcfcr5VVVecby9FfbAXUEsL0I2wEd8GJiEUMQAwCc1FZhRk9 ZFjobHyNK7jE2o6z79juyAGQSIt2NqHJ+V/Ki0XIJUdD+2BKva2Gmz3Qp4HzOUGUv3/U 3IoQ== MIME-Version: 1.0 X-Received: by 10.224.61.2 with SMTP id r2mr18094159qah.49.1392418640030; Fri, 14 Feb 2014 14:57:20 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.16.10 with HTTP; Fri, 14 Feb 2014 14:57:19 -0800 (PST) In-Reply-To: References: <201402141139.49158.jhb@freebsd.org> <201402141318.44743.jhb@freebsd.org> Date: Fri, 14 Feb 2014 14:57:19 -0800 X-Google-Sender-Auth: vlg3IRlPpAsLpOvGMj4V9O8uvdc Message-ID: Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? From: Adrian Chadd To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" , Ryan Stone , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 22:57:21 -0000 [snip] So it turns out that the threads somehow migrating between CPUs during flowtable_lookup_common() is the clock swi(s), which I'm guessing are driving the per-CPU TCP callwheel timeouts. It turns out the per-CPU clock swis aren't CPU-pinned, so they can be preempted and migrated. I'm not sure if this is correct behaviour. I'll experiment with pinning these to their base CPU and see if this causes issues. Thanks, -a From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 15 00:00:06 2014 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EC25564A; Sat, 15 Feb 2014 00:00:06 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 83E51149D; Sat, 15 Feb 2014 00:00:04 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id CAA20878; Sat, 15 Feb 2014 02:00:02 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1WESfq-000OIM-70; Sat, 15 Feb 2014 02:00:02 +0200 Message-ID: <52FEADC9.2040608@FreeBSD.org> Date: Sat, 15 Feb 2014 01:59:05 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Andrey Chernov , John Baldwin Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? References: <201402141318.44743.jhb@freebsd.org> <52FE5FBF.3090104@freebsd.org> <201402141410.29325.jhb@freebsd.org> <52FE93E6.6030705@freebsd.org> <52FE9A5E.5050300@freebsd.org> In-Reply-To: <52FE9A5E.5050300@freebsd.org> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Feb 2014 00:00:07 -0000 on 15/02/2014 00:36 Andrey Chernov said the following: > On 15.02.2014 2:08, Andrey Chernov wrote: >> On 14.02.2014 23:10, John Baldwin wrote: >> >>>> Due to this bug, not fixed yet, the real picture is more complex: >>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/163585 >>> >>> Eh, that bug report has no useful details, as in, it doesn't list the >>> actual commands run. If you do 'cpuset -l 6 -s 1' to force all >>> processes to only use CPU6, then yes, of course the other CPUs are idle >>> because that's what you _asked_ for. AFAICT, that is all the original >>> reporter did. At work we regularly add and remove CPUs from the >>> default set (set 1) on hundreds of machines every day with ULE without >>> any issues. >> >> Probably original report lack certain commands, but I provide the link >> to the port which reproduces this bug too. All threads there are >> assigned to the _different_ CPUs and appears as result on single one >> with SCHED_ULE (not with SCHED_4BSD). And it is what original reporter >> mean too. It surely happens, maybe not the first time, but on 2nd-3rd. >> It means that cpuset_setaffinity() is completely broken form SCHED_ULE >> at least for 3 years. >> > > This is code example from cpuminer port, in case you are interested, it is very simple: > > static inline void affine_to_cpu(int id, int cpu) > { > cpuset_t set; > CPU_ZERO(&set); > CPU_SET(cpu, &set); > cpuset_setaffinity(CPU_LEVEL_WHICH, CPU_WHICH_CPUSET, -1, sizeof(cpuset_t), &set); I think that CPU_WHICH_TID should have been used here. > } > ... > num_processors = sysconf(_SC_NPROCESSORS_CONF); > if (!opt_n_threads) > opt_n_threads = num_processors; > ... > In the thread itself: > struct thr_info *mythr = userdata; > int thr_id = mythr->id; > > if (num_processors > 1 && opt_n_threads % num_processors == 0) { > if (!opt_quiet) > applog(LOG_INFO, "Binding thread %d to cpu %d", > thr_id, thr_id % num_processors); > affine_to_cpu(thr_id, thr_id % num_processors); > } > -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 15 00:11:02 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E1488884; Sat, 15 Feb 2014 00:11:01 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B33B91566; Sat, 15 Feb 2014 00:11:01 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s1F0B0hl017264 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 14 Feb 2014 16:11:00 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s1F0B0ZW017263; Fri, 14 Feb 2014 16:11:00 -0800 (PST) (envelope-from jmg) Date: Fri, 14 Feb 2014 16:11:00 -0800 From: John-Mark Gurney To: Andriy Gapon Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? Message-ID: <20140215001100.GS34851@funkthat.com> Mail-Followup-To: Andriy Gapon , Andrey Chernov , John Baldwin , "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" References: <201402141318.44743.jhb@freebsd.org> <52FE5FBF.3090104@freebsd.org> <201402141410.29325.jhb@freebsd.org> <52FE93E6.6030705@freebsd.org> <52FE9A5E.5050300@freebsd.org> <52FEADC9.2040608@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <52FEADC9.2040608@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Fri, 14 Feb 2014 16:11:00 -0800 (PST) Cc: "freebsd-hackers@freebsd.org" , Andrey Chernov , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Feb 2014 00:11:02 -0000 Andriy Gapon wrote this message on Sat, Feb 15, 2014 at 01:59 +0200: > on 15/02/2014 00:36 Andrey Chernov said the following: > > On 15.02.2014 2:08, Andrey Chernov wrote: > >> On 14.02.2014 23:10, John Baldwin wrote: > >> > >>>> Due to this bug, not fixed yet, the real picture is more complex: > >>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/163585 > >>> > >>> Eh, that bug report has no useful details, as in, it doesn't list the > >>> actual commands run. If you do 'cpuset -l 6 -s 1' to force all > >>> processes to only use CPU6, then yes, of course the other CPUs are idle > >>> because that's what you _asked_ for. AFAICT, that is all the original > >>> reporter did. At work we regularly add and remove CPUs from the > >>> default set (set 1) on hundreds of machines every day with ULE without > >>> any issues. > >> > >> Probably original report lack certain commands, but I provide the link > >> to the port which reproduces this bug too. All threads there are > >> assigned to the _different_ CPUs and appears as result on single one > >> with SCHED_ULE (not with SCHED_4BSD). And it is what original reporter > >> mean too. It surely happens, maybe not the first time, but on 2nd-3rd. > >> It means that cpuset_setaffinity() is completely broken form SCHED_ULE > >> at least for 3 years. > >> > > > > This is code example from cpuminer port, in case you are interested, it is very simple: > > > > static inline void affine_to_cpu(int id, int cpu) > > { > > cpuset_t set; > > CPU_ZERO(&set); > > CPU_SET(cpu, &set); > > cpuset_setaffinity(CPU_LEVEL_WHICH, CPU_WHICH_CPUSET, -1, sizeof(cpuset_t), &set); > > I think that CPU_WHICH_TID should have been used here. I agree... cpuset(2): The which argument determines how the value of id is interpreted and is of type cpuwhich_t. The which argument may have the following values: CPU_WHICH_TID id is lwpid_t (thread id) CPU_WHICH_PID id is pid_t (process id) CPU_WHICH_CPUSET id is a cpusetid_t (cpuset id) CPU_WHICH_IRQ id is an irq number An id of '-1' may be used with a which of CPU_WHICH_TID, CPU_WHICH_PID, or CPU_WHICH_CPUSET to mean the current thread, process, or current thread's cpuset. All cpuset syscalls allow this usage. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 15 00:33:52 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 43DE3AD2; Sat, 15 Feb 2014 00:33:52 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1866216FD; Sat, 15 Feb 2014 00:33:51 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s1F0XouW017563 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 14 Feb 2014 16:33:51 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s1F0XnNj017562; Fri, 14 Feb 2014 16:33:49 -0800 (PST) (envelope-from jmg) Date: Fri, 14 Feb 2014 16:33:49 -0800 From: John-Mark Gurney To: "Gumpula, Suresh" Subject: Re: malloc(9) and its alignment Message-ID: <20140215003349.GT34851@funkthat.com> Mail-Followup-To: "Gumpula, Suresh" , "freebsd-hackers@freebsd.org" , Ian Lepore References: <1392214788.1145.52.camel@revolution.hippie.lan> <20140212220705.GV34851@funkthat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Fri, 14 Feb 2014 16:33:51 -0800 (PST) Cc: "freebsd-hackers@freebsd.org" , Ian Lepore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Feb 2014 00:33:52 -0000 Gumpula, Suresh wrote this message on Fri, Feb 14, 2014 at 02:07 +0000: > Thanks for the explanation John. How about porting ARM way of handling required alignment with creation of separate zones > And allocating with uma_zalloc(zone) for AMD64 too for bus_dmamem_alloc? It looks like the code in HEAD is different than the code you originally quoted, this is because HEAD added support for DMAR from Intel VT-d, the code is now in bounce_bus_dmamem_alloc in x86/x86/busdma_bounce.c, but it still has the same comment: 398 /* 399 * XXX: 400 * (dmat->alignment < dmat->maxsize) is just a quick hack; the exact 401 * alignment guarantees of malloc need to be nailed down, and the 402 * code below should be rewritten to take that into account. 403 * 404 * In the meantime, we'll warn the user if malloc gets it wrong. 405 */ Per porting arm's way of handling alignment, that makes sense... Though arm's way forces all allocations to be aligned to the size of a buffer, but I don't see a better way to handle it.. > -----Original Message----- > From: John-Mark Gurney [mailto:jmg@funkthat.com] > Sent: Wednesday, February 12, 2014 5:07 PM > To: Gumpula, Suresh > Cc: Ian Lepore; freebsd-hackers@freebsd.org > Subject: Re: malloc(9) and its alignment > > Gumpula, Suresh wrote this message on Wed, Feb 12, 2014 at 19:40 +0000: > > Thanks Ian for the reply. I will look at the ARM code, but I was thinking why malloc(9) does not return bucket size aligned pointers. > > Always returning bucket sizes aligned pointers may not be ideal for a cache.. say you have a buffer of 512 bytes, where often only the first > 128 bytes are used (but all 512 bytes may be)... If you always align at 512, some cache lines will be more heavily used than others, reducing the effective size of the cache... > > This is only one reason not aligning to size may be better... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 15 01:59:34 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B348D3D0 for ; Sat, 15 Feb 2014 01:59:34 +0000 (UTC) Received: from mail-la0-f44.google.com (mail-la0-f44.google.com [209.85.215.44]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2E9831B9D for ; Sat, 15 Feb 2014 01:59:33 +0000 (UTC) Received: by mail-la0-f44.google.com with SMTP id hr13so9943202lab.31 for ; Fri, 14 Feb 2014 17:59:25 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=crcNec/eWEFO9B2ZEEMjiB9e7w9iwi4nq9pGBSlNO8A=; b=MXI4mdafdrPA46+wKC/SvnsRJP1sX3WF5aK6BX/eccXBQ3J4I7xgppxsczfwDFcq9l d+fvFG4+728C7kgp9/rMafYUqs4YwuIwG0OL3phDMKK+VVWsrnjEHgTnPEQ06XL/JtnJ 4FTZ4Kig8aNvZ17czSSqgSVB30o8O4XzsEar18bmc6CZELizaya7OfjotbUVdL9idmPk hQ3w+GXWdvNbwkD50oy/ucbD8Mw6uOXJDNZwWqZf5hZXL0rs4Kys4XN5/YYM/Fzc9sIV oKZVIXSoXbwLt4VKELaxQF3GgZmTRxjAD+FZzbzoLN4DERUhGNRFz75Xc7BLlxb5E85G S7lg== X-Gm-Message-State: ALoCoQnTE0qIPVD1GLUqCDpMV7rFYr2tBOw4utwhIB4cV/DkWZ/E5xAtvCwJqGcZmBjYCOf47dc7 X-Received: by 10.112.236.3 with SMTP id uq3mr7325815lbc.14.1392429565787; Fri, 14 Feb 2014 17:59:25 -0800 (PST) Received: from [192.168.1.2] ([89.169.173.68]) by mx.google.com with ESMTPSA id gi5sm7913379lbc.4.2014.02.14.17.59.24 for (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 14 Feb 2014 17:59:25 -0800 (PST) Message-ID: <52FEC9FC.20707@freebsd.org> Date: Sat, 15 Feb 2014 05:59:24 +0400 From: Andrey Chernov User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Andriy Gapon , John Baldwin Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? References: <201402141318.44743.jhb@freebsd.org> <52FE5FBF.3090104@freebsd.org> <201402141410.29325.jhb@freebsd.org> <52FE93E6.6030705@freebsd.org> <52FE9A5E.5050300@freebsd.org> <52FEADC9.2040608@FreeBSD.org> In-Reply-To: <52FEADC9.2040608@FreeBSD.org> X-Enigmail-Version: 1.7a1pre Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Feb 2014 01:59:34 -0000 On 15.02.2014 3:59, Andriy Gapon wrote: >> This is code example from cpuminer port, in case you are interested, it is very simple: >> >> static inline void affine_to_cpu(int id, int cpu) >> { >> cpuset_t set; >> CPU_ZERO(&set); >> CPU_SET(cpu, &set); >> cpuset_setaffinity(CPU_LEVEL_WHICH, CPU_WHICH_CPUSET, -1, sizeof(cpuset_t), &set); > > I think that CPU_WHICH_TID should have been used here. > You are right, thanx! -- http://ache.vniz.net/ From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 15 02:30:42 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE20C825 for ; Sat, 15 Feb 2014 02:30:41 +0000 (UTC) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 579E81DAA for ; Sat, 15 Feb 2014 02:30:40 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id y1so9732056lam.41 for ; Fri, 14 Feb 2014 18:30:39 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=mxuMC739luJjPFi9wwM5jy8RiUDPXezylsHk+FHZ3rI=; b=KMF6I1hKa44UUVCzrUiJfpHmb41SmGZ23vOTQggeMwhz98PK3oFvs0WU57hMQgSvmN 1KsELRQ5jZbT2E79+aLmmUG3/r7GNV1+J+m99CCTV42G4Y8q1z7J2KwOJ5e8A663vwiL WJVn87Ja5NbivbBbSItw0CTYkGVev3f5hgQtk6SrZVLwmHkmyJUJwPh0hnrLjNNpyqvV MScqXMzvtcLiO+wi+Fyg1PnpNY5TyDVJTgOG+tgDSUc+FDJOUOo5OQ/HqZYaTkchNLEz 0V1zzy8SpYEb/OftseqRMO2BERCcKhJuxNZPaRvqdsnlm3/vTqNycLf0q47HQmlM+Z1s TBbA== X-Gm-Message-State: ALoCoQki7SXEatY20LEtyGVjdC+G9jXsa+WsuM77UYsCpo1Bg3EVjfM/7zTU8IAfU4V7r3+Rfgbh X-Received: by 10.152.201.197 with SMTP id kc5mr14074lac.77.1392431439118; Fri, 14 Feb 2014 18:30:39 -0800 (PST) Received: from [192.168.1.2] ([89.169.173.68]) by mx.google.com with ESMTPSA id w2sm11141662lad.4.2014.02.14.18.30.38 for (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 14 Feb 2014 18:30:38 -0800 (PST) Message-ID: <52FED14E.50304@freebsd.org> Date: Sat, 15 Feb 2014 06:30:38 +0400 From: Andrey Chernov User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Andriy Gapon , John Baldwin , "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? References: <201402141318.44743.jhb@freebsd.org> <52FE5FBF.3090104@freebsd.org> <201402141410.29325.jhb@freebsd.org> <52FE93E6.6030705@freebsd.org> <52FE9A5E.5050300@freebsd.org> <52FEADC9.2040608@FreeBSD.org> <20140215001100.GS34851@funkthat.com> In-Reply-To: <20140215001100.GS34851@funkthat.com> X-Enigmail-Version: 1.7a1pre Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Feb 2014 02:30:42 -0000 On 15.02.2014 4:11, John-Mark Gurney wrote: >>> This is code example from cpuminer port, in case you are interested, it is very simple: >>> >>> static inline void affine_to_cpu(int id, int cpu) >>> { >>> cpuset_t set; >>> CPU_ZERO(&set); >>> CPU_SET(cpu, &set); >>> cpuset_setaffinity(CPU_LEVEL_WHICH, CPU_WHICH_CPUSET, -1, sizeof(cpuset_t), &set); >> >> I think that CPU_WHICH_TID should have been used here. > > I agree... cpuset(2): > The which argument determines how the value of id is interpreted and is > of type cpuwhich_t. The which argument may have the following values: > > CPU_WHICH_TID id is lwpid_t (thread id) > CPU_WHICH_PID id is pid_t (process id) > CPU_WHICH_CPUSET id is a cpusetid_t (cpuset id) > CPU_WHICH_IRQ id is an irq number > > An id of '-1' may be used with a which of CPU_WHICH_TID, CPU_WHICH_PID, > or CPU_WHICH_CPUSET to mean the current thread, process, or current > thread's cpuset. All cpuset syscalls allow this usage. The question still remains: why SCHED_ULE and SCHED_4BSD do different things here on CPU_WHICH_CPUSET == -1 (current thread's cpuset)? It looks like SCHED_ULE changes per/process mask while SCHED_4BSD change per/thread mask in that case. -- http://ache.vniz.net/ From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 15 11:29:32 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BA2ED454; Sat, 15 Feb 2014 11:29:32 +0000 (UTC) Received: from amailer.gwdg.de (amailer.gwdg.de [134.76.10.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 78BEE1D6A; Sat, 15 Feb 2014 11:29:31 +0000 (UTC) Received: from p5b159215.dip0.t-ipconnect.de ([91.21.146.21] helo=krabat.raven.hur) by mailer.gwdg.de with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.80) (envelope-from ) id 1WEdQw-0004Dz-1Y; Sat, 15 Feb 2014 12:29:22 +0100 Message-ID: <52FF4F91.8060602@gwdg.de> Date: Sat, 15 Feb 2014 12:29:21 +0100 From: Rainer Hurling User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Warren Block , Christian Brueffer Subject: Re: FreeBSD UDF support References: <52FA94BF.80304@gwdg.de> <52FC8E61.5050200@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Authenticated: Id:rhurlin X-Spam-Level: - X-Virus-Scanned: (clean) by exiscan+sophie Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Feb 2014 11:29:32 -0000 Am 13.02.2014 16:30, schrieb Warren Block: > On Thu, 13 Feb 2014, Christian Brueffer wrote: > >> On 2/13/14 10:14 AM, Wojciech Puchar wrote: >>>> His project comes from NetBSD and modernises the UDF standard towards >>>> 2.5x. With this driver, you would be able to read modern not encrypted >>>> Blu-rays and DVDs. The driver works in principle, some minor issues >>>> have >>>> to be solved. >>>> >>>> Unfortunately there is only very little interest to integrate this >>>> driver into base. >>>> >>> i need RW support so it is not much use for me. >> >> NetBSD has newfs_udf(8), but it doesn't look like anyone has ported it >> to FreeBSD yet, > > GSOC candidates have been requested. That might be perfect. I don't know when it was added, but it is always listed as No #6: https://wiki.freebsd.org/SummerOfCode2014#Port_NetBSD.27s_UDF_implementation