From owner-svn-src-head@freebsd.org Fri Feb 3 07:51:34 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 77D32CCD05D; Fri, 3 Feb 2017 07:51:34 +0000 (UTC) (envelope-from Hartmut.Brandt@dlr.de) Received: from mailhost.dlr.de (mailhost.dlr.de [129.247.252.33]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailhost.dlr.de", Issuer "DLR CA - G02" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D9D5C1128; Fri, 3 Feb 2017 07:51:33 +0000 (UTC) (envelope-from Hartmut.Brandt@dlr.de) Received: from DLREXHUB02.intra.dlr.de (172.21.152.140) by mailhost.dlr.de (172.21.163.101) with Microsoft SMTP Server (TLS) id 14.3.319.2; Fri, 3 Feb 2017 08:51:18 +0100 Received: from DLREXMBX01.intra.dlr.de ([fe80::d198:77e5:d411:fccd]) by dlrexhub02.intra.dlr.de ([::1]) with mapi id 14.03.0319.002; Fri, 3 Feb 2017 08:51:24 +0100 From: To: CC: , , Subject: RE: svn commit: r313043 - head/sys/kern Thread-Topic: svn commit: r313043 - head/sys/kern Thread-Index: AQHSfIzUaHU0GSnLtUmrDysUntoftKFUYnQAgAD1jACAALH/gIAA4GAA Date: Fri, 3 Feb 2017 07:51:24 +0000 Message-ID: <611243783F62AF48AFB07BC25FA4B1061CEDA7DC@DLREXMBX01.intra.dlr.de> References: <201702011312.v11DC7WJ085025@repo.freebsd.org> <20170201180816.GF3334@FreeBSD.org> <611243783F62AF48AFB07BC25FA4B1061CED9FD9@DLREXMBX01.intra.dlr.de> <20170202192411.GK3334@FreeBSD.org> In-Reply-To: <20170202192411.GK3334@FreeBSD.org> Accept-Language: en-US, de-DE Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: Content-Type: multipart/mixed; boundary="_002_611243783F62AF48AFB07BC25FA4B1061CEDA7DCDLREXMBX01intra_" MIME-Version: 1.0 X-TM-AS-Product-Ver: SMEX-11.0.0.4283-8.100.1062-22862.005 X-TM-AS-Result: No--25.031400-5.000000-31 X-TM-AS-MatchedID: 150567-147015-700133-703829-106420-105700-702358-700767-7 09251-706290-700173-700324-707066-704852-706247-704421-703454-700398-711109 -843079-139006-708497-704746-106660-700107-709584-700970-186035-703747-7000 75-703788-702020-700476-700316-706891-705388-863828-701674-700486-700294-70 0693-711608-700516-702920-709512-701162-702367-701708-703712-704179-703283- 703969-704425-702037-706817-139705-701305-707027-705901-105250-703566-70147 5-187067-702560-106230-148035-148050-23116-42003 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Feb 2017 07:51:34 -0000 --_002_611243783F62AF48AFB07BC25FA4B1061CEDA7DCDLREXMBX01intra_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable It was attached to my mail, but maybe got removed somewhere. Here it is. It= does not use asio, but reproduces the same sequence of system calls. You s= tart it and the try to connect with telnet to port 10000. harti #include #include #include #include #include #include #include #include #include static void wait_loop(int kq, int sock) { struct kevent ev[32]; struct sockaddr_in addr; socklen_t socklen; for (;;) { int nev =3D kevent(kq, NULL, 0, ev, 32, NULL); if (nev < 1) err(1, "kevent"); for (int i =3D 0; i < nev; ++i) { if (ev[i].ident =3D=3D sock) { printf("accept\n"); int fd =3D accept(ev[i].ident, (struct sockaddr *)&addr, &socklen); if (fd =3D=3D -1) err(1, "accept"); } } } } int main() { struct sockaddr_in addr; /* open a TCP socket */ int kq =3D kqueue(); int sock =3D socket(PF_INET, SOCK_STREAM, 0); struct kevent ev[2]; EV_SET(&ev[0], sock, EVFILT_READ, EV_ADD | EV_CLEAR, 0, 0, NULL); EV_SET(&ev[1], sock, EVFILT_WRITE, EV_ADD | EV_CLEAR, 0, 0, NULL); int opt =3D 1; setsockopt(sock, SOL_SOCKET, SO_NOSIGPIPE, &opt, sizeof(opt)); if (kevent(kq, ev, 2, NULL, 0, NULL) =3D=3D -1) err(1, "kevent"); setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt)); memset(&addr, 0, sizeof(addr)); addr.sin_port =3D htons(10000); bind(sock, (struct sockaddr *)&addr, sizeof(addr)); listen(sock, 0x80); ioctl(sock, FIONBIO, &opt); if (kevent(kq, ev, 2, NULL, 0, NULL) =3D=3D -1) err(1, "kevent"); wait_loop(kq, sock); } -----Original Message----- From: Gleb Smirnoff [mailto:glebius@FreeBSD.org]=20 Sent: Thursday, February 02, 2017 8:24 PM To: Brandt, Hartmut Cc: src-committers@freebsd.org; svn-src-all@freebsd.org; svn-src-head@freeb= sd.org Subject: Re: svn commit: r313043 - head/sys/kern Hartmut, Thanks for explanation! Is there a test program available to reproduce th= e problem? I want to try the sequence on my branch. On Thu, Feb 02, 2017 at 08:29:20AM +0000, Hartmut.Brandt@dlr.de wrote: H> To be honest - I feared that when I saw your messages regarding this. He= re is my original message from july. Attached is also a small test program. H>=20 H> Hi, H>=20 H> I'm trying to use asio (that's boost::asio without boost) to handle list= ening sockets asynchronuosly. This appears not to work. There are also some= reports on the net about this problem. I was able to reproduce the problem= with a small C-programm that does the same steps as asio. The relevant seq= uence of system calls is: H>=20 H> kqueue() =3D 3 (0x3) H> socket(PF_INET,SOCK_STREAM,6) =3D 4 (0x4) H> setsockopt(0x4,0xffff,0x800,0x7fffffffea2c,0x4) =3D 0 (0x0) H> kevent(3,{ 4,EVFILT_READ,EV_ADD|EV_CLEAR,0x0,0x0,0x0 4,EVFILT_WRITE,EV_A= DD|EV_CLEAR,0x0,0x0,0x0 },2,0x0,0,0x0) =3D 0 (0x0) H> setsockopt(0x4,0xffff,0x4,0x7fffffffea2c,0x4) =3D 0 (0x0) H> bind(4,{ AF_INET 0.0.0.0:8080 },16) =3D 0 (0x0) H> listen(0x4,0x80) =3D 0 (0x0) H> ioctl(4,FIONBIO,0xffffea2c) =3D 0 (0x0) H> kevent(3,{ 4,EVFILT_READ,EV_ADD|EV_CLEAR,0x0,0x0,0x0 4,EVFILT_WRITE,EV_A= DD|EV_CLEAR,0x0,0x0,0x0 },2,0x0,0,0x0) =3D 0 (0x0) H> kevent(3,0x0,0,0x7fffffffe5a0,32,0x0) ERR#4 'Interrupted system call' H>=20 H> The problem here is that asio registers each file descriptor with EVFILT= _READ and EVFILT_WRITE as soon as it is opened (first kevent call).=20 H> After bringing the socket into the listening state and when async_accept= () is called it registers the socket a second time. According to the man pa= ge this is perfectly legal and can be used to modify the registration. H>=20 H> With this sequence of calls kevent() does not return when a connection i= s established successfully. H>=20 H> I tracked down the problem and the reason is in soo_kqfilter(). This is = called for the first EVFILT_READ registration and decides based on the SO_A= CCEPTCONN flag which filter operations to use solisten_filtops or soread_fi= ltops. In this case it chooses soread_filtops. H>=20 H> The second EVFILT_READ registration does not call soo_kqfilter() again, = but just updates the filter from the data and fflags field so the listening= socket ends up with the wrong filter operations. H>=20 H>=20 H>=20 H> -----Original Message----- H> From: Gleb Smirnoff [mailto:glebius@FreeBSD.org] H> Sent: Wednesday, February 01, 2017 7:08 PM H> To: Hartmut Brandt H> Cc: src-committers@freebsd.org; svn-src-all@freebsd.org;=20 H> svn-src-head@freebsd.org H> Subject: Re: svn commit: r313043 - head/sys/kern H>=20 H> On Wed, Feb 01, 2017 at 01:12:07PM +0000, Hartmut Brandt wrote: H> H> Author: harti H> H> Date: Wed Feb 1 13:12:07 2017 H> H> New Revision: 313043 H> H> URL: https://svnweb.freebsd.org/changeset/base/313043 H> H>=20 H> H> Log: H> H> Merge filt_soread and filt_solisten and decide what to do when chec= king H> H> for EVFILT_READ at the point of the check not when the event is reg= isters. H> H> This fixes a problem with asio when accepting a connection. H> H> =20 H> H> Reviewed by: kib@, Scott Mitchell H>=20 H> This goes into opposite direction with what I am doing: H>=20 H> https://reviews.freebsd.org/D9356 H>=20 H> Can you please explain the problem with asio when accepting a connection= ? H>=20 H> -- H> Totus tuus, Glebius. H>=20 H> #include H> #include H> #include H> #include H>=20 H> #include H> #include H> #include H> #include H> #include H>=20 H> static void H> wait_loop(int kq, int sock) H> { H> struct kevent ev[32]; H> struct sockaddr_in addr; H> socklen_t socklen; H>=20 H> for (;;) { H> int nev =3D kevent(kq, NULL, 0, ev, 32, NULL); H> if (nev < 1) H> err(1, "kevent"); H> for (int i =3D 0; i < nev; ++i) { H> if (ev[i].ident =3D=3D sock) { H> printf("accept\n"); H> int fd =3D accept(ev[i].ident, H> (struct sockaddr *)&addr, &socklen); H> if (fd =3D=3D -1) H> err(1, "accept"); H> } H> } H> } H> } H>=20 H> int H> main() H> { H> struct sockaddr_in addr; H>=20 H> /* open a TCP socket */ H> int kq =3D kqueue(); H>=20 H> int sock =3D socket(PF_INET, SOCK_STREAM, 0); H>=20 H> struct kevent ev[2]; H> EV_SET(&ev[0], sock, EVFILT_READ, EV_ADD | EV_CLEAR, 0, 0, NULL); H> EV_SET(&ev[1], sock, EVFILT_WRITE, EV_ADD | EV_CLEAR, 0, 0, NULL); H>=20 H> int opt =3D 1; H> setsockopt(sock, SOL_SOCKET, SO_NOSIGPIPE, &opt, sizeof(opt)); H>=20 H> if (kevent(kq, ev, 2, NULL, 0, NULL) =3D=3D -1) H> err(1, "kevent"); H>=20 H> setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt)); H>=20 H> memset(&addr, 0, sizeof(addr)); H> addr.sin_port =3D htons(10000); H>=20 H> bind(sock, (struct sockaddr *)&addr, sizeof(addr)); H> listen(sock, 0x80); H>=20 H> ioctl(sock, FIONBIO, &opt); H>=20 H> if (kevent(kq, ev, 2, NULL, 0, NULL) =3D=3D -1) H> err(1, "kevent"); H>=20 H> wait_loop(kq, sock); H> } --=20 Totus tuus, Glebius. --_002_611243783F62AF48AFB07BC25FA4B1061CEDA7DCDLREXMBX01intra_ Content-Type: text/plain; name="k.c" Content-Description: k.c Content-Disposition: attachment; filename="k.c"; size=1459; creation-date="Fri, 03 Feb 2017 07:47:17 GMT"; modification-date="Fri, 03 Feb 2017 07:47:17 GMT" Content-Transfer-Encoding: base64 I2luY2x1ZGUgPHN5cy9zb2NrZXQuaD4NCiNpbmNsdWRlIDxzeXMvdHlwZXMuaD4NCiNpbmNsdWRl IDxzeXMvZXZlbnQuaD4NCiNpbmNsdWRlIDxzeXMvZmlsaW8uaD4NCg0KI2luY2x1ZGUgPHN5cy9p b2N0bC5oPg0KI2luY2x1ZGUgPG5ldGluZXQvaW4uaD4NCiNpbmNsdWRlIDxzdGRpby5oPg0KI2lu Y2x1ZGUgPHN0cmluZy5oPg0KI2luY2x1ZGUgPGVyci5oPg0KDQpzdGF0aWMgdm9pZA0Kd2FpdF9s b29wKGludCBrcSwgaW50IHNvY2spDQp7DQoJc3RydWN0IGtldmVudCBldlszMl07DQoJc3RydWN0 IHNvY2thZGRyX2luIGFkZHI7DQoJc29ja2xlbl90IHNvY2tsZW47DQoNCglmb3IgKDs7KSB7DQoJ CWludCBuZXYgPSBrZXZlbnQoa3EsIE5VTEwsIDAsIGV2LCAzMiwgTlVMTCk7DQoJCWlmIChuZXYg PCAxKQ0KCQkJZXJyKDEsICJrZXZlbnQiKTsNCgkJZm9yIChpbnQgaSA9IDA7IGkgPCBuZXY7ICsr aSkgew0KCQkJaWYgKGV2W2ldLmlkZW50ID09IHNvY2spIHsNCgkJCQlwcmludGYoImFjY2VwdFxu Iik7DQoJCQkJaW50IGZkID0gYWNjZXB0KGV2W2ldLmlkZW50LA0KCQkJCSAgICAoc3RydWN0IHNv Y2thZGRyICopJmFkZHIsICZzb2NrbGVuKTsNCgkJCQlpZiAoZmQgPT0gLTEpDQoJCQkJCWVycigx LCAiYWNjZXB0Iik7DQoJCQl9DQoJCX0NCgl9DQp9DQoNCmludA0KbWFpbigpDQp7DQoJc3RydWN0 IHNvY2thZGRyX2luIGFkZHI7DQoNCgkvKiBvcGVuIGEgVENQIHNvY2tldCAqLw0KCWludCBrcSA9 IGtxdWV1ZSgpOw0KDQoJaW50IHNvY2sgPSBzb2NrZXQoUEZfSU5FVCwgU09DS19TVFJFQU0sIDAp Ow0KDQoJc3RydWN0IGtldmVudCBldlsyXTsNCglFVl9TRVQoJmV2WzBdLCBzb2NrLCBFVkZJTFRf UkVBRCwgRVZfQUREIHwgRVZfQ0xFQVIsIDAsIDAsIE5VTEwpOw0KCUVWX1NFVCgmZXZbMV0sIHNv Y2ssIEVWRklMVF9XUklURSwgRVZfQUREIHwgRVZfQ0xFQVIsIDAsIDAsIE5VTEwpOw0KDQoJaW50 IG9wdCA9IDE7DQoJc2V0c29ja29wdChzb2NrLCBTT0xfU09DS0VULCBTT19OT1NJR1BJUEUsICZv cHQsIHNpemVvZihvcHQpKTsNCg0KCWlmIChrZXZlbnQoa3EsIGV2LCAyLCBOVUxMLCAwLCBOVUxM KSA9PSAtMSkNCgkgICAgZXJyKDEsICJrZXZlbnQiKTsNCg0KCXNldHNvY2tvcHQoc29jaywgU09M X1NPQ0tFVCwgU09fUkVVU0VBRERSLCAmb3B0LCBzaXplb2Yob3B0KSk7DQoNCgltZW1zZXQoJmFk ZHIsIDAsIHNpemVvZihhZGRyKSk7DQoJYWRkci5zaW5fcG9ydCA9IGh0b25zKDEwMDAwKTsNCg0K CWJpbmQoc29jaywgKHN0cnVjdCBzb2NrYWRkciAqKSZhZGRyLCBzaXplb2YoYWRkcikpOw0KCWxp c3Rlbihzb2NrLCAweDgwKTsNCg0KCWlvY3RsKHNvY2ssIEZJT05CSU8sICZvcHQpOw0KDQoJaWYg KGtldmVudChrcSwgZXYsIDIsIE5VTEwsIDAsIE5VTEwpID09IC0xKQ0KCQllcnIoMSwgImtldmVu dCIpOw0KDQoJd2FpdF9sb29wKGtxLCBzb2NrKTsNCn0NCg== --_002_611243783F62AF48AFB07BC25FA4B1061CEDA7DCDLREXMBX01intra_--