From owner-freebsd-arch@FreeBSD.ORG Sun Aug 24 09:59:07 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 89C98A01; Sun, 24 Aug 2014 09:59:07 +0000 (UTC) Received: from mail-wi0-x236.google.com (mail-wi0-x236.google.com [IPv6:2a00:1450:400c:c05::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CD86F3BFB; Sun, 24 Aug 2014 09:59:06 +0000 (UTC) Received: by mail-wi0-f182.google.com with SMTP id d1so1324063wiv.9 for ; Sun, 24 Aug 2014 02:59:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=M1QLHpq3PsJ4b4A9C7ZWKPtbYPHBUw901WQSANmGi9U=; b=LTCRTUHEL6oJZrY12wdh5ziSDbiqVp6w07h/WapMhkIHN0jmzg76kaYAapUXC0u4LO qX3lQo6zWXA4/GOIGjPIhoBPrNrCRF4JCtK9XlQGo4xhcbjHFqCj40uMgOgViUmBuwOE xJOwgYMM2p0qPoxB93sdQpbwRTdwqHm9hoNiRrGCji+O8VkhSDmuUwKZ8GILqBsJcWoi pSgIA1eh22nAZFQ7TAXattm6pMQo6RF4uyJKoHU7DlZPaTm2T1OHCHkd75RfFMH9P2LE 70XBjZKeujHUQQUnFaOfhiS1zyOySMWfmU2YBrtxm+54ZIXGyk+HkJzcSn/UfEEOCXof 9zbg== X-Received: by 10.194.222.230 with SMTP id qp6mr15485174wjc.23.1408874345071; Sun, 24 Aug 2014 02:59:05 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id eb4sm18793882wic.16.2014.08.24.02.59.03 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Sun, 24 Aug 2014 02:59:04 -0700 (PDT) Date: Sun, 24 Aug 2014 11:59:01 +0200 From: Mateusz Guzik To: John Baldwin Subject: Re: current fd allocation idiom Message-ID: <20140824095901.GA2045@dft-labs.eu> References: <20140717235538.GA15714@dft-labs.eu> <20140813015627.GC17869@dft-labs.eu> <201408201110.10431.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <201408201110.10431.jhb@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Konstantin Belousov , Benjamin Kaduk , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Aug 2014 09:59:07 -0000 On Wed, Aug 20, 2014 at 11:10:10AM -0400, John Baldwin wrote: > On Friday, August 15, 2014 7:20:03 pm Benjamin Kaduk wrote: > > On Tue, 12 Aug 2014, Mateusz Guzik wrote: > > > > > On Tue, Aug 12, 2014 at 09:31:15PM -0400, Benjamin Kaduk wrote: > > > > On Tue, Aug 12, 2014 at 7:36 PM, Mateusz Guzik > wrote: > > > > > > > > > I would expect soabort to result in a timeout/reset as opposed to > regular > > > > > connection close. > > > > > > > > > > Comments around soabort suggest it should not be used as a replacement > > > > > for close, but maybe this is largely because of what the other end > will > > > > > see. That will need to be investigated. > > > > > > > > > > > > > > I added some text regarding soabort to socket.9 in r266962 -- does that > > > > help clarify the situation? > > > > > > > > > > Nope. :-) > > > > > > It is unclear if the only motivation here is making sure nobody else > > > sees the socket when given thread calls soabort. This would be easily > > > guaranteed in here: fd allocation failed, fp with given socket was never > > > exposed to anyone. > > > > > > So, if you say soabort would work here just fine, I'm happy to use it > > > and blame you for problems. :-) > > > > Hmm, I was hoping that jhb would chime in and save me from being on the > > hook, but it does look like soabort() would be acceptable in this case. > > I think having the EMFILE/ENFILE case use the same exact logic as a listen > queue overflow (i.e. soabort()) is correct. > So I was playing with converting stuff and I think I hit the wall with procdesc. procdesc is instantianed after fork cannot fail anymore. I'm not so sure we can finish up procdesc for a process which didn't fork yet, and I seriously doubt we can just kill the new process to revert stuff (e.g. what about SIGCHLD or kqueue events which fire). That said, like previously proposed plan would be: where applicable: 1. allocate fp 2. do stuff 3. install fp under free fd in cases like procdesc: 1. reserve fd 2. allocate fp 3. do stuff 4. install fp This introduces a special case to dup2 when newfd is reserved. Linux fails in this case with, so I believe we can get away with it as well. When it comes to avoiding taking filedesc lock multiple times for succeedeing case, we can get away with the following: 1. finit(..); 2. ensure all writes are completed 3. fdp->fd_ofiles[fd].fde_file = fp; -- Mateusz Guzik From owner-freebsd-arch@FreeBSD.ORG Sun Aug 24 10:16:06 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E0A3BCDA; Sun, 24 Aug 2014 10:16:05 +0000 (UTC) Received: from mail-wg0-x22b.google.com (mail-wg0-x22b.google.com [IPv6:2a00:1450:400c:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2CDC53D5E; Sun, 24 Aug 2014 10:16:05 +0000 (UTC) Received: by mail-wg0-f43.google.com with SMTP id l18so11861110wgh.2 for ; Sun, 24 Aug 2014 03:16:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=8xRJAplaZCpmxDTVdye/Dp1Gs1pVhoR0ZzzLkMj28PI=; b=CtuTRvlo+haWF+71JG8/6iFglhiplv4d4WHw00MP/uDOOVTrbhr253umB+ALyloTaC nZF7JSlesEHl+JQPsDEN1FR1O00JhpUakoe1fiVf2YxKjL2+CL2dIDgnGUqwAkyI1QxM mHhPikEmCkzoF9kp41z73rdkj3v4n+gu9TFaHzY/DJ2cDz03mNpr0JV7LjwO/6I8r+bA QU/lE7pe0Qy1duGquc+oYKF1yYq35XZP+8ygCpMwqTw5kwCXvTGIMANH1S5Pt5le0YYL ZO1vbwps+DWM1aPYeO2t+0FXpkD7T9C1c6u4GuZTT3dXWZkyANoPMtjryz9ZK8zVNQC7 ZGsw== X-Received: by 10.180.86.65 with SMTP id n1mr8436375wiz.41.1408875363147; Sun, 24 Aug 2014 03:16:03 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id oz9sm39764125wjb.20.2014.08.24.03.16.01 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Sun, 24 Aug 2014 03:16:02 -0700 (PDT) Date: Sun, 24 Aug 2014 12:15:59 +0200 From: Mateusz Guzik To: John Baldwin Subject: Re: current fd allocation idiom Message-ID: <20140824101559.GB2045@dft-labs.eu> References: <20140717235538.GA15714@dft-labs.eu> <20140813015627.GC17869@dft-labs.eu> <201408201110.10431.jhb@freebsd.org> <20140824095901.GA2045@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20140824095901.GA2045@dft-labs.eu> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Konstantin Belousov , Benjamin Kaduk , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Aug 2014 10:16:06 -0000 On Sun, Aug 24, 2014 at 11:59:01AM +0200, Mateusz Guzik wrote: > On Wed, Aug 20, 2014 at 11:10:10AM -0400, John Baldwin wrote: > > On Friday, August 15, 2014 7:20:03 pm Benjamin Kaduk wrote: > > > On Tue, 12 Aug 2014, Mateusz Guzik wrote: > > > > > > > On Tue, Aug 12, 2014 at 09:31:15PM -0400, Benjamin Kaduk wrote: > > > > > On Tue, Aug 12, 2014 at 7:36 PM, Mateusz Guzik > > wrote: > > > > > > > > > > > I would expect soabort to result in a timeout/reset as opposed to > > regular > > > > > > connection close. > > > > > > > > > > > > Comments around soabort suggest it should not be used as a replacement > > > > > > for close, but maybe this is largely because of what the other end > > will > > > > > > see. That will need to be investigated. > > > > > > > > > > > > > > > > > I added some text regarding soabort to socket.9 in r266962 -- does that > > > > > help clarify the situation? > > > > > > > > > > > > > Nope. :-) > > > > > > > > It is unclear if the only motivation here is making sure nobody else > > > > sees the socket when given thread calls soabort. This would be easily > > > > guaranteed in here: fd allocation failed, fp with given socket was never > > > > exposed to anyone. > > > > > > > > So, if you say soabort would work here just fine, I'm happy to use it > > > > and blame you for problems. :-) > > > > > > Hmm, I was hoping that jhb would chime in and save me from being on the > > > hook, but it does look like soabort() would be acceptable in this case. > > > > I think having the EMFILE/ENFILE case use the same exact logic as a listen > > queue overflow (i.e. soabort()) is correct. > > > > So I was playing with converting stuff and I think I hit the wall with > procdesc. > > procdesc is instantianed after fork cannot fail anymore. I'm not so sure > we can finish up procdesc for a process which didn't fork yet, and I > seriously doubt we can just kill the new process to revert stuff (e.g. what > about SIGCHLD or kqueue events which fire). > > That said, like previously proposed plan would be: > > where applicable: > 1. allocate fp > 2. do stuff > 3. install fp under free fd > > in cases like procdesc: > 1. reserve fd > 2. allocate fp > 3. do stuff > 4. install fp > > This introduces a special case to dup2 when newfd is reserved. Linux > fails in this case with, so I believe we can get away with it as well. > > When it comes to avoiding taking filedesc lock multiple times for > succeedeing case, we can get away with the following: > > 1. finit(..); > 2. ensure all writes are completed > 3. fdp->fd_ofiles[fd].fde_file = fp; > Erm, of course we need to make sure fdtable is not going to be allocated from under us and we would need seqeuence counters or something else to ensure we updated proper copy and however was messing with it sees the update. -- Mateusz Guzik From owner-freebsd-arch@FreeBSD.ORG Sun Aug 24 19:42:46 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7341E158; Sun, 24 Aug 2014 19:42:46 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EC0E03E5F; Sun, 24 Aug 2014 19:42:45 +0000 (UTC) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s7OJgeO7097581 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 24 Aug 2014 22:42:40 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua s7OJgeO7097581 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id s7OJgea5097580; Sun, 24 Aug 2014 22:42:40 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 24 Aug 2014 22:42:40 +0300 From: Konstantin Belousov To: Mateusz Guzik Subject: Re: current fd allocation idiom Message-ID: <20140824194240.GY2737@kib.kiev.ua> References: <20140717235538.GA15714@dft-labs.eu> <20140813015627.GC17869@dft-labs.eu> <201408201110.10431.jhb@freebsd.org> <20140824095901.GA2045@dft-labs.eu> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="D7UQ7r/72P+7HXeb" Content-Disposition: inline In-Reply-To: <20140824095901.GA2045@dft-labs.eu> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: Benjamin Kaduk , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Aug 2014 19:42:46 -0000 --D7UQ7r/72P+7HXeb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Aug 24, 2014 at 11:59:01AM +0200, Mateusz Guzik wrote: > On Wed, Aug 20, 2014 at 11:10:10AM -0400, John Baldwin wrote: > > On Friday, August 15, 2014 7:20:03 pm Benjamin Kaduk wrote: > > > On Tue, 12 Aug 2014, Mateusz Guzik wrote: > > >=20 > > > > On Tue, Aug 12, 2014 at 09:31:15PM -0400, Benjamin Kaduk wrote: > > > > > On Tue, Aug 12, 2014 at 7:36 PM, Mateusz Guzik =20 > > wrote: > > > > > > > > > > > I would expect soabort to result in a timeout/reset as opposed = to=20 > > regular > > > > > > connection close. > > > > > > > > > > > > Comments around soabort suggest it should not be used as a repl= acement > > > > > > for close, but maybe this is largely because of what the other = end=20 > > will > > > > > > see. That will need to be investigated. > > > > > > > > > > > > > > > > > I added some text regarding soabort to socket.9 in r266962 -- doe= s that > > > > > help clarify the situation? > > > > > > > > > > > > > Nope. :-) > > > > > > > > It is unclear if the only motivation here is making sure nobody else > > > > sees the socket when given thread calls soabort. This would be easi= ly > > > > guaranteed in here: fd allocation failed, fp with given socket was = never > > > > exposed to anyone. > > > > > > > > So, if you say soabort would work here just fine, I'm happy to use = it > > > > and blame you for problems. :-) > > >=20 > > > Hmm, I was hoping that jhb would chime in and save me from being on t= he > > > hook, but it does look like soabort() would be acceptable in this cas= e. > >=20 > > I think having the EMFILE/ENFILE case use the same exact logic as a lis= ten=20 > > queue overflow (i.e. soabort()) is correct. > >=20 >=20 > So I was playing with converting stuff and I think I hit the wall with > procdesc. >=20 > procdesc is instantianed after fork cannot fail anymore. I'm not so sure > we can finish up procdesc for a process which didn't fork yet, and I > seriously doubt we can just kill the new process to revert stuff (e.g. wh= at > about SIGCHLD or kqueue events which fire). We can create the procdesc-attached process in the stopped state. If fp installation went fine, the process is be released (pdfork(2) does not accept rfork(2) flags, so right now there is no issue of conflicting with RFSTOPPED). If installation failed, we can reparent the child to init, kill and release. The fork trampoline should do the right thing; if not, it could be changed to explicitely check for the termination conditions. >=20 > That said, like previously proposed plan would be: >=20 > where applicable: > 1. allocate fp > 2. do stuff > 3. install fp under free fd >=20 > in cases like procdesc: > 1. reserve fd > 2. allocate fp > 3. do stuff > 4. install fp >=20 > This introduces a special case to dup2 when newfd is reserved. Linux > fails in this case with, so I believe we can get away with it as well. >=20 > When it comes to avoiding taking filedesc lock multiple times for > succeedeing case, we can get away with the following: >=20 > 1. finit(..); > 2. ensure all writes are completed > 3. fdp->fd_ofiles[fd].fde_file =3D fp; >=20 > --=20 > Mateusz Guzik --D7UQ7r/72P+7HXeb Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJT+kAwAAoJEJDCuSvBvK1BluEP/16iGTotw1cTDHAoCLWBAoqi tX0yQ2KOVTzyVa7ZD4laj/rPyfZQu9KhEuFGjvleGNuxhcHfu/c1LyEGlsNzPZze uxy9Z2gOHCK+WX9sOZlrd34uEcpDPL3Ids3BFZzKzwKKvT0tTNRpCneXB1n7eMmo k73iJjtRDkidAGBjm3DadyVrkQr0u/+8gH45kIqUs/pDwXGOmdbwFPa5LBl0LR4p NVSFQ5PBzuMQadvwBVVhYQaLCYyELZ30XFfmAnNktIWTrCWzDgTLfF1eT4gLoPYQ oI8QTklY8AX48SNXpSDx90hOeh9mtPNJJVPYmZ1u9NJUiQhqZb8qEErDCYP89oZM IE0FuVdsQZ1Sg47MzkR6m9s4KQ4PXh2+ZUDxZnqNv6oGgtYmAhrk3TMUl5Chle0p jrdz3cdgsuRfnDyNFp6a+n9YW+RwVxXKvONREZ059sdGr+I9Z6CJCD3vO8oHEJeX NOnZpPCDdZBt/MKtrr5PcSEODAKsRj0diuD7zQkPdEMmm3EkX80KZkPNb2EGjeJ6 18O8hjfWJIjad0TZERxy+USHlkKVcLydW45MDC2Qa3hSkU1nzYhN5PJRggHDno78 vwb2kC+8eN08N3vVTXElgWHeBcfRU8AOBdx44naAPU0/REa3PVQ+MMfX6eGvjy3b ilP4NcUn3gU4SlQZPvoQ =G7Ir -----END PGP SIGNATURE----- --D7UQ7r/72P+7HXeb-- From owner-freebsd-arch@FreeBSD.ORG Thu Aug 28 17:02:58 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D65C2E00; Thu, 28 Aug 2014 17:02:58 +0000 (UTC) Received: from mail-lb0-x234.google.com (mail-lb0-x234.google.com [IPv6:2a00:1450:4010:c04::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 09E6F1266; Thu, 28 Aug 2014 17:02:57 +0000 (UTC) Received: by mail-lb0-f180.google.com with SMTP id w7so1219295lbi.25 for ; Thu, 28 Aug 2014 10:02:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=AIN+SpCXJTmKi8Qbs/CAA3Cr02G3MFKlt2jwi734a6c=; b=hXZv5lEzDqWVr/1gpXS/bucurwsu4EEBAokScuXwRBnCT4uRnQVPFSnk8PCnoRhor5 oTTP7Z++NrX4v3qdhNnADtNAM3TDx2rC0AukLmm9ZAD3oeRF/vqH2t9OIH6Z9Fgoe1pi wTptn+4D20mKfBmU88IYaCG8XOQn1QNfM8wV/ofy74xmj3XsWptBvFf5CL3IKV5l0UoN FGgazA0lFjkXpToPyaODMsLGUNg6SPU+Zo8Tne5FRQHxYwCkEB4qyVkvolu8mIba/HRa Qp0Qux6+RQoRE2yGN44tQW+F83te3AAz36EkW9OrtMA2Ym3PACR8EXpEhycTbPOCqPnb kEHg== MIME-Version: 1.0 X-Received: by 10.112.61.68 with SMTP id n4mr5354153lbr.91.1409245375509; Thu, 28 Aug 2014 10:02:55 -0700 (PDT) Sender: edschouten@gmail.com Received: by 10.152.144.2 with HTTP; Thu, 28 Aug 2014 10:02:55 -0700 (PDT) Date: Thu, 28 Aug 2014 19:02:55 +0200 X-Google-Sender-Auth: zZp9dGYxyBGLbliO0zYZO_0ZfgE Message-ID: Subject: Lock annotations: enable them for libpthread, libstdthreads From: Ed Schouten To: freebsd-arch@freebsd.org, freebsd-toolchain@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Aug 2014 17:02:59 -0000 Hello all, As some of you may know, Clang supports some form of thread safety analysis by means of annotations: http://clang.llvm.org/docs/ThreadSafetyAnalysis.html To summarize, it works as follows: - You annotate which functions in your code (in)directly acquire/release/depend on which locks. - You annotate global variables and structure members to specify which locks should be held when accessing them (guards). You can then invoke Clang with -Wthread-safety and it will throw a compiler error in case these invariants don't hold. For example, in case you acquire a lock in an unannotated function and don't unlock it before the function returns, it will generate a warning. Couple of random observations on this feature from my side: - It seems that the thread safety stuff was mainly designed with C++ in mind. So far I haven't been able to get guarding annotations working for mutexes residing in structures; only global mutexes. Still, lock/unlock flow analysis on these mutexes works fine, which should already be a game-changer. - We can even use this feature to mark functions like pthread_cond_wait() as requiring the lock to be held. That's pretty sweet. - In userspace, almost all of the code compiled out of the box with -Wthread-safety turned on. I found one actual bug with this enabled (see r270749) and had to patch up a small number of files in auditdistd and hastd to annotate functions that acquire/release locks (see synch.h). I've written a patch that adds #defines for the attributes that are relevant for us (read: things that are useful for C programs) to and patched up all functions in libpthread and libstdthreads to use these attributes where possible: http://80386.nl/pub/20140828-lock_annotate.txt One of my observations is that if we want to use this in C code seriously (which I hope), we would not use it in the same way as is done in C++ canonically. In kernel space it seems that our locking/unlocking strategy is a lot more asymmetric and non-trivial compared to high-level C++ programs. It is more common to have functions that pick up a lock and don't drop it or vice versa. Especially free()-like functions that expect the object to be locked. This is why I decided to go for a different naming scheme for the macros that is a bit shorter than what's used on the Clang page mentioned above. It is also a bit more accurate for our use case. Instead of it being "THE exclusive lock member function of a mutex", it is a function that just happens to "lock a mutex exclusively". My intent is to slowly push bits and pieces of this patch into HEAD within the next week. As the existing userspace only needs a very small number of patches to build with this flag, I am also going to add -Wthread-safety to WARNS=6, to make sure it won't regress. I'm also planning on MFC'ing at least the changes to , so it is at least possible to write portable code. Be sure to take a look at the diff, play around with it and let me know what you think. Thanks, -- Ed Schouten From owner-freebsd-arch@FreeBSD.ORG Thu Aug 28 17:34:18 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C805299B; Thu, 28 Aug 2014 17:34:18 +0000 (UTC) Received: from mail-la0-x233.google.com (mail-la0-x233.google.com [IPv6:2a00:1450:4010:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 265BB1744; Thu, 28 Aug 2014 17:34:17 +0000 (UTC) Received: by mail-la0-f51.google.com with SMTP id gl10so1330536lab.38 for ; Thu, 28 Aug 2014 10:34:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=+0FQhe0yUGT0+AA7klK5mULGs8jc7IL36ltiXtmYoE8=; b=b9GHw0qI/hvE2SLz/5XKZ2QKOah3fGOiDfWdSkSeuE0ZRZF+Rk7cWs5HoEo44Qp5I3 E8p7U6JVx6raa4akm165lTsIb0m17zm949zVJbHpwETN8pgzzwjMjL9kL0hDwcxj2MVn SolRERNK7JMVJ34rhsnGQp1xK0RYW3xzptLWhpKVtUTmgy/ea4BIWj/NqGguAv9YCvW/ l9bDWwOUtAUjVLFUECdNkMky2E09vOz/CywXSwatM7TpeLgW56PQ9h3uKRcO6AP/WFEW JPfsxI7bsYbxjarw4KJkPddZdm6UQeXIX0aaSKwch8+qyjCkI2/BM6TnuDdyBwsAReFt FNRg== MIME-Version: 1.0 X-Received: by 10.152.29.1 with SMTP id f1mr5778513lah.47.1409247255887; Thu, 28 Aug 2014 10:34:15 -0700 (PDT) Sender: edschouten@gmail.com Received: by 10.152.144.2 with HTTP; Thu, 28 Aug 2014 10:34:15 -0700 (PDT) In-Reply-To: <20140828171859.GA81757@troutmask.apl.washington.edu> References: <20140828171859.GA81757@troutmask.apl.washington.edu> Date: Thu, 28 Aug 2014 19:34:15 +0200 X-Google-Sender-Auth: nX6Ojn0CJ7ejjcr5ULxt0hEbOek Message-ID: Subject: Re: Lock annotations: enable them for libpthread, libstdthreads From: Ed Schouten To: Steve Kargl Content-Type: text/plain; charset=UTF-8 Cc: freebsd-toolchain@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Aug 2014 17:34:18 -0000 On 28 August 2014 19:18, Steve Kargl wrote: > What it is the performance penalty (time and memory)? That is, > buildworld already takes a looooonnnnng time on my system because > of clang. My gut feeling is that impact is minimal. Buildworlds seem to take approximately the same time, mainly because we don't have that many annotated objects/functions/expressions. I will do some measurements. -- Ed Schouten From owner-freebsd-arch@FreeBSD.ORG Thu Aug 28 17:37:09 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CE009B0F; Thu, 28 Aug 2014 17:37:09 +0000 (UTC) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "troutmask.apl.washington.edu", Issuer "troutmask.apl.washington.edu" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 91047176D; Thu, 28 Aug 2014 17:37:09 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.9/8.14.9) with ESMTP id s7SHIxeR081873 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 28 Aug 2014 10:18:59 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.9/8.14.9/Submit) id s7SHIxfo081872; Thu, 28 Aug 2014 10:18:59 -0700 (PDT) (envelope-from sgk) Date: Thu, 28 Aug 2014 10:18:59 -0700 From: Steve Kargl To: Ed Schouten Subject: Re: Lock annotations: enable them for libpthread, libstdthreads Message-ID: <20140828171859.GA81757@troutmask.apl.washington.edu> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-toolchain@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Aug 2014 17:37:09 -0000 On Thu, Aug 28, 2014 at 07:02:55PM +0200, Ed Schouten wrote: > > small number of patches to build with this flag, I am also going to > add -Wthread-safety to WARNS=6, to make sure it won't regress. I'm > also planning on MFC'ing at least the changes to , so it > is at least possible to write portable code. What it is the performance penalty (time and memory)? That is, buildworld already takes a looooonnnnng time on my system because of clang. -- Steve From owner-freebsd-arch@FreeBSD.ORG Thu Aug 28 19:22:16 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 209F0556; Thu, 28 Aug 2014 19:22:16 +0000 (UTC) Received: from mail-qg0-x22f.google.com (mail-qg0-x22f.google.com [IPv6:2607:f8b0:400d:c04::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C3AA01480; Thu, 28 Aug 2014 19:22:15 +0000 (UTC) Received: by mail-qg0-f47.google.com with SMTP id z60so1232375qgd.6 for ; Thu, 28 Aug 2014 12:22:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=UgNx2oE70RiXlomV3HGZiCXpMa9z2PlDKNhzNy1Mv2Q=; b=TmQk1GfowkTQTR7d9ZCdIIaL9T6SWOM2xHvNxDjjmqM5xR1XBoUuMADKLtKPH636Z2 f0TBIDsUjj2ep5McGhcdBYhjGcQNCG1aExzYygkCPjRd7pTIjM7EpwrnncahJ7aXhYia 9v6Doopb3DiIw1/EYfmgxb8fMzEkCtdo+jKnsacIG6ezll9/CCe4CAHVYF3/sYCOD7/N TqcPOoRlz+hlPhC0GUgB5LjE4aJ5JKfOVSU0DeP8woWSW/HzmojcAeKNZp8uF8uFeANL iPdqr9iO3CKWaFiLp6GL9hp/j6YvPYlxI/ks735UIdrtTCVdieHewk1e62QXhyteuYH5 mlDQ== MIME-Version: 1.0 X-Received: by 10.224.46.8 with SMTP id h8mr10068218qaf.6.1409253734659; Thu, 28 Aug 2014 12:22:14 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.39.139 with HTTP; Thu, 28 Aug 2014 12:22:14 -0700 (PDT) In-Reply-To: References: <20140828171859.GA81757@troutmask.apl.washington.edu> Date: Thu, 28 Aug 2014 12:22:14 -0700 X-Google-Sender-Auth: enEHJW-qBTlBVzDM18jUokyop10 Message-ID: Subject: Re: Lock annotations: enable them for libpthread, libstdthreads From: Adrian Chadd To: Ed Schouten Content-Type: text/plain; charset=UTF-8 Cc: freebsd-toolchain@freebsd.org, Steve Kargl , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Aug 2014 19:22:16 -0000 On 28 August 2014 10:34, Ed Schouten wrote: > On 28 August 2014 19:18, Steve Kargl wrote: >> What it is the performance penalty (time and memory)? That is, >> buildworld already takes a looooonnnnng time on my system because >> of clang. > > My gut feeling is that impact is minimal. Buildworlds seem to take > approximately the same time, mainly because we don't have that many > annotated objects/functions/expressions. I will do some measurements. Would you please test on something that isn't the latest generation hardware? Even just a core 2 duo would be fine. -a From owner-freebsd-arch@FreeBSD.ORG Sat Aug 30 05:33:30 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E5433EFB; Sat, 30 Aug 2014 05:33:30 +0000 (UTC) Received: from mail-la0-x22d.google.com (mail-la0-x22d.google.com [IPv6:2a00:1450:4010:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 430481D93; Sat, 30 Aug 2014 05:33:30 +0000 (UTC) Received: by mail-la0-f45.google.com with SMTP id pn19so3715307lab.4 for ; Fri, 29 Aug 2014 22:33:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=5O5zyU14ICNb0mAwlFS3aow3W2oDlwrfOEY094VwpOY=; b=ZrB4N5HAxLWw2XuhwM0MvbaOd0fqrUY9vzCElKhi/sxRiluBU9Hc4SaTxH2BNAN3wM De/CNgOXhi16NfQfXvTJyxFXv8ytWFEDWIWVbPIl5GTXH8kO+ml1CTcm+vE9rbfkCukG Go/4+lNr3ZxEZIckqQ0rDXokaZvHjQhzeoGQ9FaqgxzMSeJ6TNwpup9b/wFowNwBvivW hYrk4rc1MGZNPC+bRiKZRYK9++8gyfhqqLTYqJUXKhvgsM/rJFCwWEXkhgbVlEF466iQ QiQ3XFO4e2hWU0XHTeU+XYX73rwK96H/FfDZqFczB9LckUGVWDyvs+jNZL7Zvo8XbCuU N3Og== MIME-Version: 1.0 X-Received: by 10.152.9.42 with SMTP id w10mr15564169laa.9.1409376807716; Fri, 29 Aug 2014 22:33:27 -0700 (PDT) Sender: edschouten@gmail.com Received: by 10.152.144.2 with HTTP; Fri, 29 Aug 2014 22:33:27 -0700 (PDT) In-Reply-To: References: <20140828171859.GA81757@troutmask.apl.washington.edu> Date: Sat, 30 Aug 2014 07:33:27 +0200 X-Google-Sender-Auth: aWjFVkS6R5-j0PUf_BVS6yr6j8o Message-ID: Subject: Re: Lock annotations: enable them for libpthread, libstdthreads From: Ed Schouten To: Steve Kargl Content-Type: text/plain; charset=UTF-8 Cc: freebsd-toolchain@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 05:33:31 -0000 Hi, On 28 August 2014 19:34, Ed Schouten wrote: > My gut feeling is that impact is minimal. Buildworlds seem to take > approximately the same time, mainly because we don't have that many > annotated objects/functions/expressions. I will do some measurements. Some measurements: I've done five 'make buildworlds' with -Wthread-safety and five without. This was just a single-threaded build (no -j). Real time to build world: -Wthread-safety 1:12:55.60 1:12:48.81 1:12:56.33 1:12:44.58 1:12:48.14 -Wno-thread-safety 1:12:50.50 1:12:52.43 1:12:48.20 1:12:42.85 1:12:43.19 That said, this a pretty fast box (i7 4770) with 32 GB of RAM. Unfortunately, I currently don't have any system running FreeBSD that is significantly slower than that. So if you still have any concerns regarding build times, I would really appreciate it if you'd test this patch yourself. Thanks, -- Ed Schouten From owner-freebsd-arch@FreeBSD.ORG Sat Aug 30 16:22:20 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AAFDFC73; Sat, 30 Aug 2014 16:22:20 +0000 (UTC) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "troutmask.apl.washington.edu", Issuer "troutmask.apl.washington.edu" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6C9021C13; Sat, 30 Aug 2014 16:22:20 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.9/8.14.9) with ESMTP id s7UGMFar094743 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 30 Aug 2014 09:22:15 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.9/8.14.9/Submit) id s7UGMF6F094742; Sat, 30 Aug 2014 09:22:15 -0700 (PDT) (envelope-from sgk) Date: Sat, 30 Aug 2014 09:22:15 -0700 From: Steve Kargl To: Ed Schouten Subject: Re: Lock annotations: enable them for libpthread, libstdthreads Message-ID: <20140830162215.GA94722@troutmask.apl.washington.edu> References: <20140828171859.GA81757@troutmask.apl.washington.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-toolchain@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 16:22:20 -0000 On Sat, Aug 30, 2014 at 07:33:27AM +0200, Ed Schouten wrote: > > On 28 August 2014 19:34, Ed Schouten wrote: > > My gut feeling is that impact is minimal. Buildworlds seem to take > > approximately the same time, mainly because we don't have that many > > annotated objects/functions/expressions. I will do some measurements. > > Some measurements: I've done five 'make buildworlds' with > -Wthread-safety and five without. This was just a single-threaded > build (no -j). Real time to build world: > > -Wthread-safety > > 1:12:55.60 > 1:12:48.81 > 1:12:56.33 > 1:12:44.58 > 1:12:48.14 > > -Wno-thread-safety > > 1:12:50.50 > 1:12:52.43 > 1:12:48.20 > 1:12:42.85 > 1:12:43.19 > > That said, this a pretty fast box (i7 4770) with 32 GB of RAM. > Unfortunately, I currently don't have any system running FreeBSD that > is significantly slower than that. So if you still have any concerns > regarding build times, I would really appreciate it if you'd test this > patch yourself. > Thanks for testing. The above reduces my fear of feature creep where each change only adds a little bit more time to buildworld. -- Steve