Date: Tue, 27 Dec 2016 11:09:47 -0800 From: Alfred Perlstein <alfred@freebsd.org> To: Xin LI <delphij@gmail.com> Cc: =?utf-8?B?55ub5oWn5Y2O?= <hhsheng@corp.netease.com>, Hongjiang Zhang <honzhan@microsoft.com>, freebsd-arch@freebsd.org, Jilles Tjoelker <jilles@stack.nl> Subject: Re: question about fopen fd limit Message-ID: <5461BE82-34CF-4B46-B7A1-6F8E1B3246C8@freebsd.org> In-Reply-To: <CAGMYy3tQ0=7wEegf0W6Fq-Av%2BDTquc0su7Eg8o4OfEAmq3rCrg@mail.gmail.com> References: <2016122223570929089978@corp.netease.com> <CY1PR03MB1517A33F8F9E0BAC00C2E345B5950@CY1PR03MB1517.namprd03.prod.outlook.com> <2016122311014089280414@corp.netease.com> <CY1PR03MB151796BFAEA5DDB1CD6DB752B5950@CY1PR03MB1517.namprd03.prod.outlook.com> <2016122316484066524625@corp.netease.com> <db32b02b-2eb3-cee8-98cd-0940223f04d9@freebsd.org> <CAGMYy3tQ0=7wEegf0W6Fq-Av%2BDTquc0su7Eg8o4OfEAmq3rCrg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Makes sense, at the same time if we go to 4/2billion descriptors it shouldn'= t matter. Another trick would be to augment the inline function to check a v= ersion flag and if it's incorrect then call the libc function. This will cau= se some performance problems but only for really old software.=20 Sent from my iPhone > On Dec 27, 2016, at 9:52 AM, Xin LI <delphij@gmail.com> wrote: >=20 > -freebsd-net (bcc), +freebsd-arch >=20 > This would work but also comes with a lot of pain. >=20 > (But really - why do we implement these accessors, like fileno() and frien= ds as macros with knowledge of the sFILE layout? Will it be reasonable to s= tart converting these to functions as a first step, which would not break AB= I but allow us to do it in the future? Otherwise we would be just kicking t= he same can along the street forever...) >=20 > FILE is supposed to be fully opaque to application writers in my opinion. >=20 >> On Sat, Dec 24, 2016 at 5:19 PM, Alfred Perlstein <alfred@freebsd.org> wr= ote: >> Hello =E7=9B=9B=E6=85=A7=E5=8D=8E >>=20 >> Here's another trick that may work. >>=20 >> Use funopen(3) and provide your own read/write/seek and close functions f= or the high fds. >>=20 >> You can basically make "cookie" a struct that contains your "int sized" f= ds. >>=20 >> FILE * >> funopen(const void *cookie, int (*readfn)(void *, char *, int), int (= *writefn)(void *, const char *, int), >> fpos_t (*seekfn)(void *, fpos_t, int), int (*closefn)(void *)); >>=20 >>=20 >> If you need more help please make sure to email me directly so I can see y= our question. >>=20 >> -Alfred >>=20 >>=20 >>=20 >>> On 12/23/16 12:48 AM, =E7=9B=9B=E6=85=A7=E5=8D=8E wrote: >>> hi all, >>>=20 >>> Thank you for your advice ~ >>> solution 2 definitly broaden my horizons ~~but may be not a good choic= e for my project ~~LoL >>> i will try to mail freebsd-current mail list, if libc is as your descri= ption , may be i should modify it by myself ~~ >>> Thank you so much~ >>> Are u KingSoft's Dr Zhang ? nice to meet you !!!!! >>>=20 >>> = winson sheng >>> =20 >>>=20 >>> winson sheng >>> From: Hongjiang Zhang >>> Date: 2016-12-23 11:44 >>> To: =E7=9B=9B=E6=85=A7=E5=8D=8E; freebsd-net >>> Subject: RE: RE: question about fopen fd limit >>> Ok. I know. >>> There are two possible solutions: >>> Quick solution for short term: modify short to int in libc by yourself, b= uildworld and installworld. Pushing to modify libc may take a long time, esp= ecially only few people encounter this issue. You=E2=80=99d better send emai= l to freebsd-current to confirm whether they accept your suggestion. >>> Work around: You can first reserve a series of fd before opening TCP con= nections. For example, invoke open(=E2=80=9C/dev/null=E2=80=9D) for 10000 ti= mes to get 10000 fds. Those fd values are small enough to be held by =E2=80=9C= short=E2=80=9D. After that, start TCP connections. Once you need to fopen a f= ile, please call open(=E2=80=9Cxxx=E2=80=9D) instead, and then use dup2(old_= fd, new_fd) to exchange the two fd. The old_fd value is the one obtained by o= pen(=E2=80=9Cxxx=E2=80=9D), and new_fd is one in your reserved fd fields, an= d next please use fdopen(fd, mode). Here, you have to manage the reserved fd= s by yourself including open/close. >>> In my eyes: >>> is the quick method, and there is no modifications in your logic. >>> Needs you to maintain the reserved consecutive fields for fd by yourself= , which increased the complexity of your logic. >>> Thanks >>> Hongjiang Zhang >>> From: =E7=9B=9B=E6=85=A7=E5=8D=8E [mailto:hhsheng@corp.netease.com] >>> Sent: Friday, December 23, 2016 11:02 AM >>> To: Hongjiang Zhang <honzhan@microsoft.com>; freebsd-net <freebsd-net@fr= eebsd.org> >>> Subject: Re: RE: question about fopen fd limit >>> hi all, >>> not map TCP to FILE, you misunderstanding my meaning~ >>> for example, if my server tcp already holds 32000 connection >>> fopen only has 767 fd to use >>> the problem has no bussiness with tcp fd, BUT fopen ... >>> in some particular situlations , my server will open 1k+ FILE , tha= t will exceed the fileno limit, and overflow occur >>> my server can't open any file more ,that's the problem ~ >>> so i felt if bsd official could change FILE struct's fileno to a UN= SIGNED SHORT that may be an effecient and convenient solution just for my ca= se ? >>> UNSIGNED SHORT fileno is enough for me, and i don't wanna change a lo= t of FILE function that take FILE * as its argument ~ >>> Thank you ~~~ >>> winson sheng >>> =20 >>>=20 >>> winson sheng >>> From: Hongjiang Zhang >>> Date: 2016-12-23 10:17 >>> To: =E7=9B=9B=E6=85=A7=E5=8D=8E; freebsd-net >>> Subject: RE: question about fopen fd limit >>> Why do you need to map TCP fd to FILE? >>> It is difficult to modify FILE structure. If it is possible, let us fi= gure out some new designs to meet your requirement. >>> -----Original Message----- >>> From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.or= g] On Behalf Of ??? >>> Sent: Thursday, December 22, 2016 11:57 PM >>> To: freebsd-net <freebsd-net@freebsd.org> >>> Subject: question about fopen fd limit >>> hi all, >>> hi~ >>> we are from Chinese Game Develop Corp, Netease. >>> and One of our product using FreeBsd as its OS platform. >>> This Game has Millions of players online , and Each Server may holds= 25000+ tcp connection at the same time.Thanks to BSD and kqueue :) >>> for example, it's one of our server , netstat cmd to list connecti= ons overall... >>> netstat -an | grep 13396 (it's our listening port) | wc -l >>> 23221 >>> recently we do some performance optimize and promote this connect= limit to 28000+ or 30000+. >>> But we find Freebsd has a limit that this huge online number will tak= e 28000+ fd, and bsd FILE * struct's >>> fd only support to SHORT . such as .. >>> struct __sFILE { >>> ... >>> short _file; /* (*) fileno, if Unix descriptor, else -1 */ ... >>> so if our server want to fopen some file when we still hold this on= line number, the fd amount may easily exceed 32767, and fopen definitely ret= urn a err code. then the server will appear some fataly ERROR. >>> we do a simple test and confirm this situation. >>> then in fopen's code , we notice that we can use open to return a f= d instread of fopen to avoid this overflow, >>> as below >>> 68 /* >>> 1 * File descriptors are a full int, but _file is only a short. >>> 2 * If we get a valid file descriptor that is greater than >>> 3 * SHRT_MAX, then the fd will get sign-extended into an >>> 4 * invalid file descriptor. Handle this case by failing the >>> 5 * open. >>> 6 */ >>> BUT ... so many c lib FILE series function needs a FILE * pointer= as input argument, we can't convert all of them to fd, or it will be a rath= er suffering things to us. >>> and even in BSD 10 , it seems this short limit still there , but ot= her OS as debian , FILE strucnt's fileno is a int . >>> we found an unoffical patch easily change this fileno to unsigned ,= but we are a very stready project, we can't afford the risk to use an unoff= ical patch. >>> so, do you have any plan to change this fopen fd limit to UNSIGNED S= HORT or int in the future ? ushort is enough for us . >>> if you do , we are really glad and excited~~~~~~~if you don't ,it don= en't matter, plz give us a reply so that we may need to >>> find some other plan to resolved this suffering thing. >>> LoL, thank you !!!!! >>> yours sincerely >>> winson sheng >>> winson sheng >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> https://na01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flists= .freebsd.org%2Fmailman%2Flistinfo%2Ffreebsd-net&data=3D02%7C01%7Chonzhan%40m= icrosoft.com%7C4a9dfccbccd446be2f4a08d42a833fb0%7C72f988bf86f141af91ab2d7cd0= 11db47%7C1%7C0%7C636180190584478890&sdata=3DPAwJP5IXHy0WJwxbV7MB%2B8zvKheZUY= jhHx3ohFRSPZM%3D&reserved=3D0 >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>=20 >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5461BE82-34CF-4B46-B7A1-6F8E1B3246C8>