Date: Fri, 6 Nov 1998 10:05:10 -0800 (PST) From: Matthew Dillon <dillon@apollo.backplane.com> To: David Greenman <dg@root.com> Cc: cvs-committers@FreeBSD.ORG, cvs-all@FreeBSD.ORG Subject: Re: sendfile.2 (was Re: cvs commit: ...) Message-ID: <199811061805.KAA00657@apollo.backplane.com> References: <199811061603.IAA18481@implode.root.com>
next in thread | previous in thread | raw e-mail | index | archive | help
: Matt, please try out the attached patches that provide non-blocking support
:for sendfile(2).
: It works like this: If the socket is set non-blocking, then sendfile(2)
:will send as much as it can and if there is still more to send, it will
:return EAGAIN and return the number of bytes written to the socket in *sbytes.
:The application will need to keep track of the number of bytes written and
:adjust the file offset accordingly for subsequant sendfile(2) calls. Further,
:if headers/trailers are specified, then the application will need to figure
:out the mess and rebuild the iovecs as appropriate.
:
:-DG
:
:David Greenman
:Co-founder/Principal Architect, The FreeBSD Project
I've tested the patch, it appears to work nicely!
My (new) test code can be had at:
http://www.backplane.com/FreeBSD/
I also had a thought and experiemented with the SO_SNDLOWAT TCP
option. It actually seems to make a significant difference in
certain cases.
First, in the non-blocking sendfile() case, the advantages are obvious
since increasing SO_SNDLOWAT greatly decreases the number of calls
to sendfile() and select() from user mode. A definite 7x improvement
in efficiency is realized. Here changing SO_SNDLOWAT makes a big
difference.
# time ./tcpsendfile lander 5202 -r8 /usr/lib/libc_pic.a -3
count = 1493
0.030u 0.045s 0:09.01 0.7% 6+126k 0+0io 0pf+0w
# time ./tcpsendfile lander 5202 -r8 /usr/lib/libc_pic.a -3 -l16384
count = 434
0.019u 0.038s 0:09.02 0.4% 14+255k 0+0io 0pf+0w
# time ./tcpsendfile lander 5202 -r8 /usr/lib/libc_pic.a -3 -l32768
count = 259
0.008u 0.048s 0:09.01 0.4% 14+255k 0+0io 0pf+0w
In the blocking write() case, SO_SNDLOWAT doesn't seem to matter as much.
# time ./tcpsendfile lander 5202 -r8 /usr/lib/libc_pic.a -2 -l4096
count = 160
0.023u 0.378s 0:09.01 4.3% 11+293k 0+0io 0pf+0w
# time ./tcpsendfile lander 5202 -r8 /usr/lib/libc_pic.a -2 -l32768
count = 160
0.014u 0.365s 0:09.03 3.9% 10+279k 0+0io 0pf+0w
In the blocking sendfile() case, where there is no return to
usermode, changing SO_SNDLOWAT has a less pronounced effect. It's hard
to tell since the user & system times are so low because I only have
a 10BaseT network at home (and no idle -current machines at BEST to
play with at the moment :-().
# time ./tcpsendfile lander 5202 -r8 /usr/lib/libc_pic.a -1 -l4096
count = 8
0.020u 0.030s 0:09.02 0.5% 8+148k 0+0io 0pf+0w
# time ./tcpsendfile lander 5202 -r8 /usr/lib/libc_pic.a -1 -l32768
count = 8
0.026u 0.026s 0:09.02 0.4% 12+218k 0+0io 0pf+0w
It is obvious that proper tuning of SO_SNDLOWAT in the non-blocking
select() case can be important, and just as obvious that it makes no
significant difference in other cases.
--
The interesting thing about this is that the number of context switches
does not seem to be the deciding factor... otherwise I would have expected
the blocking write() case to have a more significant spread. Instead,
it appears to be the actual switch to and from user mode that is causing
the greatest overhead. The block-write() case does not return to usermode
while the non-blocking sendfile() case does. Either that, or select()ing
on a single descriptor is more expensive then I thought it was.
-Matt
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe cvs-all" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199811061805.KAA00657>