Date: Sat, 24 Dec 2016 17:19:56 -0800 From: Alfred Perlstein <alfred@freebsd.org> To: =?UTF-8?B?55ub5oWn5Y2O?= <hhsheng@corp.netease.com>, Hongjiang Zhang <honzhan@microsoft.com>, freebsd-net <freebsd-net@freebsd.org> Subject: Re: question about fopen fd limit Message-ID: <db32b02b-2eb3-cee8-98cd-0940223f04d9@freebsd.org> In-Reply-To: <2016122316484066524625@corp.netease.com> References: <2016122223570929089978@corp.netease.com> <CY1PR03MB1517A33F8F9E0BAC00C2E345B5950@CY1PR03MB1517.namprd03.prod.outlook.com> <2016122311014089280414@corp.netease.com> <CY1PR03MB151796BFAEA5DDB1CD6DB752B5950@CY1PR03MB1517.namprd03.prod.outlook.com> <2016122316484066524625@corp.netease.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hello 盛慧华 Here's another trick that may work. Use funopen(3) and provide your own read/write/seek and close functions for the high fds. You can basically make "cookie" a struct that contains your "int sized" fds. FILE * funopen(const void *cookie, int (*readfn)(void *, char *, int), int (*writefn)(void *, const char *, int), fpos_t (*seekfn)(void *, fpos_t, int), int (*closefn)(void *)); If you need more help please make sure to email me directly so I can see your question. -Alfred On 12/23/16 12:48 AM, 盛慧华 wrote: > hi all, > > Thank you for your advice ~ > solution 2 definitly broaden my horizons ~~but may be not a good choice for my project ~~LoL > i will try to mail freebsd-current mail list, if libc is as your description , may be i should modify it by myself ~~ > Thank you so much~ > Are u KingSoft's Dr Zhang ? nice to meet you !!!!! > > winson sheng > > > > winson sheng > > From: Hongjiang Zhang > Date: 2016-12-23 11:44 > To: 盛慧华; freebsd-net > Subject: RE: RE: question about fopen fd limit > Ok. I know. > There are two possible solutions: > Quick solution for short term: modify short to int in libc by yourself, buildworld and installworld. Pushing to modify libc may take a long time, especially only few people encounter this issue. You’d better send email to freebsd-current to confirm whether they accept your suggestion. > Work around: You can first reserve a series of fd before opening TCP connections. For example, invoke open(“/dev/null”) for 10000 times to get 10000 fds. Those fd values are small enough to be held by “short”. After that, start TCP connections. Once you need to fopen a file, please call open(“xxx”) instead, and then use dup2(old_fd, new_fd) to exchange the two fd. The old_fd value is the one obtained by open(“xxx”), and new_fd is one in your reserved fd fields, and next please use fdopen(fd, mode). Here, you have to manage the reserved fds by yourself including open/close. > > In my eyes: > is the quick method, and there is no modifications in your logic. > Needs you to maintain the reserved consecutive fields for fd by yourself, which increased the complexity of your logic. > > Thanks > Hongjiang Zhang > > From: 盛慧华 [mailto:hhsheng@corp.netease.com] > Sent: Friday, December 23, 2016 11:02 AM > To: Hongjiang Zhang <honzhan@microsoft.com>; freebsd-net <freebsd-net@freebsd.org> > Subject: Re: RE: question about fopen fd limit > > hi all, > > not map TCP to FILE, you misunderstanding my meaning~ > > for example, if my server tcp already holds 32000 connection > fopen only has 767 fd to use > > the problem has no bussiness with tcp fd, BUT fopen ... > > in some particular situlations , my server will open 1k+ FILE , that will exceed the fileno limit, and overflow occur > my server can't open any file more ,that's the problem ~ > > so i felt if bsd official could change FILE struct's fileno to a UNSIGNED SHORT that may be an effecient and convenient solution just for my case ? > UNSIGNED SHORT fileno is enough for me, and i don't wanna change a lot of FILE function that take FILE * as its argument ~ > > Thank you ~~~ > > winson sheng > > > > > winson sheng > > From: Hongjiang Zhang > Date: 2016-12-23 10:17 > To: 盛慧华; freebsd-net > Subject: RE: question about fopen fd limit > Why do you need to map TCP fd to FILE? > > It is difficult to modify FILE structure. If it is possible, let us figure out some new designs to meet your requirement. > > -----Original Message----- > From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org] On Behalf Of ??? > Sent: Thursday, December 22, 2016 11:57 PM > To: freebsd-net <freebsd-net@freebsd.org> > Subject: question about fopen fd limit > > hi all, > > hi~ > we are from Chinese Game Develop Corp, Netease. > and One of our product using FreeBsd as its OS platform. > This Game has Millions of players online , and Each Server may holds 25000+ tcp connection at the same time.Thanks to BSD and kqueue :) > > for example, it's one of our server , netstat cmd to list connections overall... > netstat -an | grep 13396 (it's our listening port) | wc -l > 23221 > > recently we do some performance optimize and promote this connect limit to 28000+ or 30000+. > But we find Freebsd has a limit that this huge online number will take 28000+ fd, and bsd FILE * struct's > fd only support to SHORT . such as .. > > struct __sFILE { > ... > short _file; /* (*) fileno, if Unix descriptor, else -1 */ ... > > so if our server want to fopen some file when we still hold this online number, the fd amount may easily exceed 32767, and fopen definitely return a err code. then the server will appear some fataly ERROR. > > we do a simple test and confirm this situation. > > then in fopen's code , we notice that we can use open to return a fd instread of fopen to avoid this overflow, > as below > > 68 /* > 1 * File descriptors are a full int, but _file is only a short. > 2 * If we get a valid file descriptor that is greater than > 3 * SHRT_MAX, then the fd will get sign-extended into an > 4 * invalid file descriptor. Handle this case by failing the > 5 * open. > 6 */ > > > BUT ... so many c lib FILE series function needs a FILE * pointer as input argument, we can't convert all of them to fd, or it will be a rather suffering things to us. > > and even in BSD 10 , it seems this short limit still there , but other OS as debian , FILE strucnt's fileno is a int . > > we found an unoffical patch easily change this fileno to unsigned , but we are a very stready project, we can't afford the risk to use an unoffical patch. > > so, do you have any plan to change this fopen fd limit to UNSIGNED SHORT or int in the future ? ushort is enough for us . > if you do , we are really glad and excited~~~~~~~if you don't ,it donen't matter, plz give us a reply so that we may need to > find some other plan to resolved this suffering thing. > LoL, thank you !!!!! > > yours sincerely > winson sheng > > > winson sheng > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freebsd.org%2Fmailman%2Flistinfo%2Ffreebsd-net&data=02%7C01%7Chonzhan%40microsoft.com%7C4a9dfccbccd446be2f4a08d42a833fb0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636180190584478890&sdata=PAwJP5IXHy0WJwxbV7MB%2B8zvKheZUYjhHx3ohFRSPZM%3D&reserved=0 > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?db32b02b-2eb3-cee8-98cd-0940223f04d9>