Date: Mon, 31 Dec 2001 15:53:18 -0800 From: Terry Lambert <tlambert2@mindspring.com> To: "Louis A. Mamakos" <louie@TransSys.COM> Cc: Julian Elischer <julian@elischer.org>, Matthew Dillon <dillon@apollo.backplane.com>, Mike Silbersack <silby@silby.com>, Josef Karthauser <joe@tao.org.uk>, Tomas Svensson <tsn@gbdev.net>, freebsd-hackers@FreeBSD.ORG Subject: Re: FreeBSD performing worse than Linux? Message-ID: <3C30FA6E.231CCE0C@mindspring.com> References: <Pine.BSF.4.21.0112311225150.94344-100000@InterJet.elischer.org> <200112312327.fBVNRt719835@whizzo.transsys.com>
next in thread | previous in thread | raw e-mail | index | archive | help
"Louis A. Mamakos" wrote: > > An underlying issue here is why applications decide to set TCP_NODELAY > options on sockets, rather than just letting Nagle's algorithm do > the right thing. I recall some handwaving about this in the X server > some years ago to make mouse movements "smoother". OK, I've been making an assumption here that is perhaps incorrect, but is consistant with the traces posted so far: I've assumed that the data is lost between the ethernet side of the dongle and the USB side of the dongle. If the FreeBSD USB driver is really a dog (I have a hard time buying that) or the dongle USB hardware is superior to that in the PC being used (I have a hard time buying that one, either), then it's just possible that the data losses are FreeBSD's, and not a result of dongle buffer overrun. You'd need a breakout box for USB, and a USB system without the problem (Linux?) to prove it via dumps, though. That said: There is no problem with turning on TCP_NODELAY. If anything, the resulting data sent through the dongle would be *less* bursty (being application paced), and tend to be *less* likely to fill up any pool in the middle to the point of overflow. This really has all the earmarks of a pool congestion (not a link congestion) problem in the dongle, to my mind. > For the problem at hand, if both the client and server machines didn't > do TCP_NODLEAY, then there'd only be one packet smaller than the > TCP MSS in flight between the transmitter and receiver at any one > time. I think that poking OpenSSH to not set the TCP_NODELAY option > "fixed" this problem. I think it masked it. If you had a less CPU-intensive test application, I think you would see that you can still overrun the buffer. The masking is a natural effect of decreasing the overall payload encapsulation overhead. But increase the amount of payload, and you should still see the problem rear its ugly head, since the link rate differential is so large on either side of the dongle. A fugly way of handling the problem would be to initiate a flow control mechanism between the dongle and the FreeBSD ethernet driver. This is tantamount to the hardware flow control required for the MNP modem knock-offs by Microcomm competitors who used flow control as a means of keeping the pool size manageable, so that they could really cheap out on putting RAM in the modem, and undercut Microcomm's prices. For this to work, you'd have to have cooperative firmware in the dongles; I think that taking that approach is a lost cause, unless we are downloading our own firmware. I think we will have to tackle it another way. Teaching the driver about TCP windows, and delaying transmissions until 50% of the receive window is satisfied would probably work, but is a very unsatisfactory approach (even though we know that transmit over the link is not going to have the same problem as receive, due to the speed differential across the dongle). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3C30FA6E.231CCE0C>