Date: Tue, 4 Apr 1995 03:21:53 -0500 (CDT) From: Mike Pritchard <pritc003@maroon.tc.umn.edu> To: wpaul@freefall.cdrom.com (Bill Paul) Cc: CVS-commiters@freefall.cdrom.com, cvs-lib@freefall.cdrom.com Subject: Re: cvs commit: src/lib/libc/rpc rpc_dtablesize.c Message-ID: <199504040821.DAA08121@mpp.com> In-Reply-To: <199504040553.WAA04169@freefall.cdrom.com> from "Bill Paul" at Apr 3, 95 10:53:24 pm
next in thread | previous in thread | raw e-mail | index | archive | help
> A temporary fix for this is to clamp the value returned by _rpc_dtablesize() > at FD_SETSIZE (as defined in <sys/types.h> (256)). I suppose the Right > Thing would be to provide some mechanism for select() to dynamically > adjust itself to handle FD_SETSIZE values larger than 256, but it's a > bit late in the game for that. Hopefully 256 file descriptors will be enough > to keep RPC happy for now. I've done some work directly related to this in the past. You run into problems anytime the user is able to raise the open file limit higher than FD_SETSIZE. Just so everyone knows, the problem is that whenever the open file limit is higher than FD_SETSIZE, and select() is called and given an "nfds" parmeter > FD_SETSIZE, select winds up reading past the fdset array and finds bogus bits set and usually blows the system call off with EBADF. Programs really shouldn't be calling select with a "nfds" parameter of "getdtablesize()", they should use FD_SETSIZE instead. On the system I was working on, we had users who wanted to be able to have 10,000+ open files at a time (don't ask me why - I personally think that if you require that many open files you must be doing something really stupid/inefficient). After a lot of heated discussion, we decided to set FD_SETSIZE to 1024 so that most compiled programs would be able to handle almost any "reasonable" open file limit they might be run with and document the fact that higher limits would probably require changes to allow the program to correctly in all cases when using the higher limit. In order to allow programs with > 1024 open files to use a select like function, we opted to implement the SYS5.4 "poll" system call. You those of you who aren't familiar with poll(), it allows you to pass in an array of file descriptors that you want polled (for the same conditions that select checks). This has the advantage of allowing the user to check file descriptors > FD_SETSIZE, and since most programs are usually only interested in a few file descriptors anyways, no extra kernel overhead is involved scanning a bitmask full of zeros that you aren't interested in. System library routines can then be changed to use poll() so that they will function no matter how many open files the calling program may have opened. Any program that requires more than FD_SETSIZE open files needs to be aware of the fact that they need to use poll() instead of select. Select() could also be implemented as a library routine that calls poll() if so desired. Standard I/O also needs to allow support of the higher number of open file. I think that the current FreeBSD stdio does dynamic allocation of the FILE structure entries at run time so I don't think that this is a problem. POSIX doesn't say anything about select/poll, either. But POSIX does take a stand on several issues on the maximum number of open files. One of them being the fact that the maximum number of open files per process can't change over the life of the process, which FreeBSD currently violates. The fix for this is to allow a "new" limit to be set that is inherited by any new child processes that are created, but the limit for the process that set the limit is not changed. POSIX also has some things to say about the OPEN_MAX symbol (if it is defined then certain things must be a certain way, and that if it is not defined then certain things can work another way, I forget the exact details), and something about a minimum number of open files (OPEN_MIN?). I don't have a POSIX manual handy, so I can't be anymore specific right now. The ANSI C spec also has something to say about OPEN_MAX. The whole problem with this is that if you really want to be able to support large numbers of open files correctly and in a POSIX blessed manner, you have to bite the bullet and make some decisions on which way you want the open file limit handled. -- Mike Pritchard pritc003@maroon.tc.umn.edu "Go that way. Really fast. If something gets in your way, turn"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199504040821.DAA08121>