From owner-freebsd-arch@freebsd.org Tue Dec 27 18:38:28 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44B50C93649 for ; Tue, 27 Dec 2016 18:38:28 +0000 (UTC) (envelope-from lew@perftech.com) Received: from smtp-gw.pt.net (smtp-gw.pt.net [206.210.194.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp-gw.pt.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0A20B1F7D for ; Tue, 27 Dec 2016 18:38:27 +0000 (UTC) (envelope-from lew@perftech.com) X-ASG-Debug-ID: 1482862740-09411a12f98e2df0001-RYubVt Received: from mail.pt.net (mail.pt.net [206.210.194.11]) by smtp-gw.pt.net with ESMTP id aYXADG1kmeGgXuPp (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 27 Dec 2016 12:19:00 -0600 (CST) X-Barracuda-Envelope-From: lew@perftech.com X-Barracuda-Effective-Source-IP: mail.pt.net[206.210.194.11] X-Barracuda-Apparent-Source-IP: 206.210.194.11 Received: from localhost (localhost [IPv6:::1]) by mail.pt.net (Postfix) with ESMTP id 73E03840263; Tue, 27 Dec 2016 12:19:00 -0600 (CST) Received: from mail.pt.net ([IPv6:::1]) by localhost (mail.pt.net [IPv6:::1]) (amavisd-new, port 10032) with ESMTP id onOqH_35Qxbc; Tue, 27 Dec 2016 12:18:59 -0600 (CST) Received: from localhost (localhost [IPv6:::1]) by mail.pt.net (Postfix) with ESMTP id D58138402ED; Tue, 27 Dec 2016 12:18:59 -0600 (CST) X-Virus-Scanned: amavisd-new at pt.net Received: from mail.pt.net ([IPv6:::1]) by localhost (mail.pt.net [IPv6:::1]) (amavisd-new, port 10026) with ESMTP id nlgk8vrbKjMD; Tue, 27 Dec 2016 12:18:59 -0600 (CST) Received: from vpn199-8.pt.net (vpn199-8.pt.net [206.210.199.8]) (Authenticated sender: lew@pt.net) by mail.pt.net (Postfix) with ESMTPSA id 81E66840263; Tue, 27 Dec 2016 12:18:59 -0600 (CST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: question about fopen fd limit From: Lewis Donzis X-ASG-Orig-Subj: Re: question about fopen fd limit In-Reply-To: Date: Tue, 27 Dec 2016 13:19:01 -0500 Cc: Xin LI Content-Transfer-Encoding: quoted-printable Message-Id: <7A472344-E4DA-452A-AC81-9CA67CD0B26C@perftech.com> References: <2016122223570929089978@corp.netease.com> <2016122311014089280414@corp.netease.com> <2016122316484066524625@corp.netease.com> To: freebsd-arch@freebsd.org X-Mailer: Apple Mail (2.3124) X-Barracuda-Connect: mail.pt.net[206.210.194.11] X-Barracuda-Start-Time: 1482862740 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://smtp-gw.pt.net:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 9208 X-Virus-Scanned: by bsmtpd at pt.net X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 1.32 X-Barracuda-Spam-Status: No, SCORE=1.32 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests=BSF_RULE7568M, HTTP_ESCAPED_HOST, URI_HEX, URI_NOVOWEL X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.35391 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.32 URI_HEX URI: URI hostname has long hexadecimal sequence 0.00 HTTP_ESCAPED_HOST URI: Uses %-escapes inside a URL's hostname 0.50 URI_NOVOWEL URI: URI hostname has long non-vowel sequence 0.50 BSF_RULE7568M Custom Rule 7568M X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2016 18:38:28 -0000 I agree, and, in my humble opinion, the FILE structure should never have = made any assumptions about the value of an fd, especially that it would = fit in a short when an fd is defined as an int. For that matter, = fileno() is defined as returning an int, but the implementation is = technically returning a short in a non-threaded environment. I would support changing the __sFILE structure=E2=80=99s _file member to = an int and probably also changing _flags to int or unsigned since it = would end up getting padded for alignment anyway. It would also be more = efficient. > On Dec 27, 2016, at 12:52 PM, Xin LI wrote: >=20 > -freebsd-net (bcc), +freebsd-arch >=20 > This would work but also comes with a lot of pain. >=20 > (But really - why do we implement these accessors, like fileno() and > friends as macros with knowledge of the sFILE layout? Will it be > reasonable to start converting these to functions as a first step, = which > would not break ABI but allow us to do it in the future? Otherwise we > would be just kicking the same can along the street forever...) >=20 > FILE is supposed to be fully opaque to application writers in my = opinion. >=20 > On Sat, Dec 24, 2016 at 5:19 PM, Alfred Perlstein > wrote: >=20 >> Hello =E7=9B=9B=E6=85=A7=E5=8D=8E >>=20 >> Here's another trick that may work. >>=20 >> Use funopen(3) and provide your own read/write/seek and close = functions >> for the high fds. >>=20 >> You can basically make "cookie" a struct that contains your "int = sized" >> fds. >>=20 >> FILE * >> funopen(const void *cookie, int (*readfn)(void *, char *, int), = int >> (*writefn)(void *, const char *, int), >> fpos_t (*seekfn)(void *, fpos_t, int), int (*closefn)(void = *)); >>=20 >>=20 >> If you need more help please make sure to email me directly so I can = see >> your question. >>=20 >> -Alfred >>=20 >>=20 >>=20 >> On 12/23/16 12:48 AM, =E7=9B=9B=E6=85=A7=E5=8D=8E wrote: >>=20 >>> hi all, >>>=20 >>> Thank you for your advice ~ >>> solution 2 definitly broaden my horizons ~~but may be not a good = choice >>> for my project ~~LoL >>> i will try to mail freebsd-current mail list, if libc is as your >>> description , may be i should modify it by myself ~~ >>> Thank you so much~ >>> Are u KingSoft's Dr Zhang ? nice to meet you !!!!! >>>=20 >>>=20 >>> winson sheng >>>=20 >>>=20 >>> winson sheng >>> From: Hongjiang Zhang >>> Date: 2016-12-23 11:44 >>> To: =E7=9B=9B=E6=85=A7=E5=8D=8E; freebsd-net >>> Subject: RE: RE: question about fopen fd limit >>> Ok. I know. >>> There are two possible solutions: >>> Quick solution for short term: modify short to int in libc by = yourself, >>> buildworld and installworld. Pushing to modify libc may take a long = time, >>> especially only few people encounter this issue. You=E2=80=99d = better send email to >>> freebsd-current to confirm whether they accept your suggestion. >>> Work around: You can first reserve a series of fd before opening TCP >>> connections. For example, invoke open(=E2=80=9C/dev/null=E2=80=9D) = for 10000 times to get >>> 10000 fds. Those fd values are small enough to be held by = =E2=80=9Cshort=E2=80=9D. After >>> that, start TCP connections. Once you need to fopen a file, please = call >>> open(=E2=80=9Cxxx=E2=80=9D) instead, and then use dup2(old_fd, = new_fd) to exchange the two >>> fd. The old_fd value is the one obtained by open(=E2=80=9Cxxx=E2=80=9D= ), and new_fd is one >>> in your reserved fd fields, and next please use fdopen(fd, mode). = Here, you >>> have to manage the reserved fds by yourself including open/close. >>> In my eyes: >>> is the quick method, and there is no modifications in your logic. >>> Needs you to maintain the reserved consecutive fields for fd by = yourself, >>> which increased the complexity of your logic. >>> Thanks >>> Hongjiang Zhang >>> From: =E7=9B=9B=E6=85=A7=E5=8D=8E [mailto:hhsheng@corp.netease.com] >>> Sent: Friday, December 23, 2016 11:02 AM >>> To: Hongjiang Zhang ; freebsd-net < >>> freebsd-net@freebsd.org> >>> Subject: Re: RE: question about fopen fd limit >>> hi all, >>> not map TCP to FILE, you misunderstanding my meaning~ >>> for example, if my server tcp already holds 32000 connection >>> fopen only has 767 fd to use >>> the problem has no bussiness with tcp fd, BUT fopen ... >>> in some particular situlations , my server will open 1k+ FILE , = that >>> will exceed the fileno limit, and overflow occur >>> my server can't open any file more ,that's the problem ~ >>> so i felt if bsd official could change FILE struct's fileno to a >>> UNSIGNED SHORT that may be an effecient and convenient solution just = for my >>> case ? >>> UNSIGNED SHORT fileno is enough for me, and i don't wanna change a = lot >>> of FILE function that take FILE * as its argument ~ >>> Thank you ~~~ >>> winson sheng >>>=20 >>>=20 >>> winson sheng >>> From: Hongjiang Zhang >>> Date: 2016-12-23 10:17 >>> To: =E7=9B=9B=E6=85=A7=E5=8D=8E; freebsd-net >>> Subject: RE: question about fopen fd limit >>> Why do you need to map TCP fd to FILE? >>> It is difficult to modify FILE structure. If it is possible, let us >>> figure out some new designs to meet your requirement. >>> -----Original Message----- >>> From: owner-freebsd-net@freebsd.org = [mailto:owner-freebsd-net@freebsd.org] >>> On Behalf Of ??? >>> Sent: Thursday, December 22, 2016 11:57 PM >>> To: freebsd-net >>> Subject: question about fopen fd limit >>> hi all, >>> hi~ >>> we are from Chinese Game Develop Corp, Netease. >>> and One of our product using FreeBsd as its OS platform. >>> This Game has Millions of players online , and Each Server may = holds >>> 25000+ tcp connection at the same time.Thanks to BSD and kqueue :) >>> for example, it's one of our server , netstat cmd to list >>> connections overall... >>> netstat -an | grep 13396 (it's our listening port) | wc -l >>> 23221 >>> recently we do some performance optimize and promote this = connect >>> limit to 28000+ or 30000+. >>> But we find Freebsd has a limit that this huge online number will = take >>> 28000+ fd, and bsd FILE * struct's >>> fd only support to SHORT . such as .. >>> struct __sFILE { >>> ... >>> short _file; /* (*) fileno, if Unix descriptor, else -1 */ ... >>> so if our server want to fopen some file when we still hold this >>> online number, the fd amount may easily exceed 32767, and fopen = definitely >>> return a err code. then the server will appear some fataly ERROR. >>> we do a simple test and confirm this situation. >>> then in fopen's code , we notice that we can use open to return = a fd >>> instread of fopen to avoid this overflow, >>> as below >>> 68 /* >>> 1 * File descriptors are a full int, but _file is only a short. >>> 2 * If we get a valid file descriptor that is greater than >>> 3 * SHRT_MAX, then the fd will get sign-extended into an >>> 4 * invalid file descriptor. Handle this case by failing the >>> 5 * open. >>> 6 */ >>> BUT ... so many c lib FILE series function needs a FILE * = pointer >>> as input argument, we can't convert all of them to fd, or it will be = a >>> rather suffering things to us. >>> and even in BSD 10 , it seems this short limit still there , but >>> other OS as debian , FILE strucnt's fileno is a int . >>> we found an unoffical patch easily change this fileno to = unsigned , >>> but we are a very stready project, we can't afford the risk to use = an >>> unoffical patch. >>> so, do you have any plan to change this fopen fd limit to = UNSIGNED >>> SHORT or int in the future ? ushort is enough for us . >>> if you do , we are really glad and excited~~~~~~~if you don't ,it >>> donen't matter, plz give us a reply so that we may need to >>> find some other plan to resolved this suffering thing. >>> LoL, thank you !!!!! >>> yours sincerely >>> winson sheng >>> winson sheng >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> https://na01.safelinks.protection.outlook.com/?url=3Dhttps%3A% >>> 2F%2Flists.freebsd.org%2Fmailman%2Flistinfo%2Ffreebsd >>> -net&data=3D02%7C01%7Chonzhan%40microsoft.com%7C4a9dfccbccd4 >>> 46be2f4a08d42a833fb0%7C72f988bf86f141af91ab2d7cd011db47%7C1% >>> 7C0%7C636180190584478890&sdata=3DPAwJP5IXHy0WJwxbV7MB%2B8 >>> zvKheZUYjhHx3ohFRSPZM%3D&reserved=3D0 >>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>=20 >>=20 >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>=20 > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org"