Date: Tue, 10 Apr 2001 11:37:37 -0700 From: Alfred Perlstein <bright@wintelcom.net> To: Bill Paul <wpaul@FreeBSD.ORG> Cc: hackers@FreeBSD.ORG Subject: Re: sendfile() vs. gigabit ethernet Message-ID: <20010410113736.U15938@fw.wintelcom.net> In-Reply-To: <20010410180514.E229D37B422@hub.freebsd.org>; from wpaul@FreeBSD.ORG on Tue, Apr 10, 2001 at 11:05:14AM -0700 References: <20010410180514.E229D37B422@hub.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
* Bill Paul <wpaul@FreeBSD.ORG> [010410 11:05] wrote: > People are constantly asking me about the performance of gigabit ethernet > cards with FreeBSD. I'm constantly wishing they would all go to hell -- > directly to hell, without passing "GO" or collecting $200 -- but their > persistence has led me to ask a few questions of my own. One thing everyone > notices right away is that some of FreeBSD's defaults aren't terribly > optimal for gigabit ethernet, the main one being socket buffer sizes. > At least as far as TCP is concerned, you can do one of two things: > > - Change your application to call setsockopt() and increase the SO_SNDBUF > and SO_RCVBUF values. > - Use sysctl to change the net.inet.tcp.sendspace and net.inet.tcp.recvspace > values. > > The former option affectly only a given application, while the latter > affects the entire system. Most people will bump the sendspace and > recvspace variables since it's easier than modifying their application. > Changing the default from 16K to 64K usually has a drastic effect on > performance with something like ttcp, which is what I typically use > for testing. You can also use the -B flag to ttcp to have it increase > the SNDBUF/RCVBUF values just for itself and achieve the same effect. > > But then what usually happens is people will attempt to perform a file > transfer test using, of all things. FTP, and suddenly they notice that > performance doesn't really improve all that much. I could never understand > just why this was so, until I realized that /usr/libexec/ftpd in FreeBSD > now uses sendfile(). The ttcp utility on the other hand uses write(2) > (as does netperf for that matter). > > So my question is: has anyone done any tests with sendfile() over a > high speed network link and tried to vary the sendspace/recvspace > defaults to improve performance? If so, does it make any difference? > My feeling is that it should, but from what I can tell, sendfile() > only passes data to sosend() in page-sized chunks, and this is somehow > negating the effect of increasing the socket buffer size. Am I right, > or am I just missing something obvious, as usual? Have you considered that the client side doesn't use "receivefile" and therefore is probably an order of magnitude slower? What you might want to try is this ftp> get remote_large_file /dev/null As far as the tcp send/recv space stuff, I agree that the default is kind of low. It's really about time we offered some tunables based on system configuration, for example: depending on ram available: scale: maxusers, nmbclusters turn on: vfs.vmiodirenable when traversing the tcb hashes, detect long chain runs and possibly raise spl/aquire a lock to realloc them to a larger size. I'm sure there's numerous other ssytem things to tune (amount of sysv ipc primatives available) based on the amount of ram. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] Represent yourself, show up at BABUG http://www.babug.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20010410113736.U15938>