Date: Sat, 18 Jan 2003 14:27:30 -0800 From: Terry Lambert <tlambert2@mindspring.com> To: Lars Eggert <larse@ISI.EDU> Cc: Scott Mitchell <scott.mitchell@mail.com>, Jay Sern Liew <jay@americanhorizonsbank.com>, freebsd-hackers@FreeBSD.ORG Subject: Re: 5400RPM|7200RPM Cable bottleneck? Message-ID: <3E29D4D2.47A5E909@mindspring.com> References: <1042849118.3e289d5ee4458@webmail.americanhorizonsbank.com> <20030118105015.GB783@fishballoon.dyndns.org> <3E2998FD.1010904@isi.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
Lars Eggert wrote: > > If by 'cable' you mean a cable modem providing at best a few Mb/s > > bandwidth, then I doubt the speed of your disk will have any impact > > whatsoever. Even the crappiest ATA disk will be able to deliver a few > > MB/s -- in the worst case that's still an order of magnitude more than you > > can stuff down your cable modem. > > I've tried NFS mounting ISI servers at home over PPTP over a cable modem > connection, and it's painfully slow - much slower than the bandwidth of > the cable pipe. NFS isn't well tuned for high-RTT environments (in my > case, 20ms). > > If you get anything reasonable working, I'd be very interested in seeing > your NFS parameters. Or, if you settle on anything else (AFS, etc.), I'd > be interested to hear how that compares. The up-channel speed on a cable modem is usually in the neighborhood of 1.5 times the amount of bandwidth needed for just the ACKs for the data coming down the down-channel; that leaves very little bandwidth left over for NFS requests to the server, particularly if there are a lot of data transfers taking place from the server to the client (client reads). If you are doing any level of data transfers from the client to the server (client writes), you should expect your performance to go to hell. The only real fix for this is a symmetric pipe size, where the upchannel is not a lot smaller than the down-channel (the intent of a cable modem is to push data *to* you; that it has to accept data *from* you in order to do this is merely a large inconvenience for the cable company, who'd prefer that you were a passive content recipient, rather than a content generator). One workaround is to split ACK traffic from data going back, logically, and then delay the ACKs. This additional latency will modify the receive window size down on the remote end's router, and thus reserve additional bandwidth for the data you want to send up, out of the available upchannel bandwidth. Another is to schedule traffic, prioritizing outbound data traffic. Neither of these approaches are easily implemented in FreeBSD (or any OS, for that matter, because you are supposed to be a passive data consumer, according to the cable companies model of the world, which they got from their television and pay per view businesses); the closest you can really come is the Alt-Q implementation for FreeBSD. Even then, it operates by doing over-throttling, so you willl lose larger than necessary down channel speed, to obtain up-channel, since it does not take into account the idea of an assymetric pipe size. Actually, Julian Elisher has some code that can do ACK delaying, which could account for pipe assymetry; also some of the work done by Jennifer Yang could be applied to this problem, as well, using ACK-pacing to avoid hysteresis swings in the data rates that might happen with Julian's code. In any case, the problem is your cable modem, and the protocol you are using to access the remote data. You could go to an HTTP model, where you simply pull a very large chunk of data down via a single initial request when the link is (relatively) idle. This is actually the model that the cable company expects its customers to use, so it will be relatively fast, compared to an NFS copy of the same file. You could also deal with this by caching writes; to do this requires a distributed cache coherency protocol between the clients and the servers; it saves the up-channel by grouping up-channel writes in a cache zone, and then accepting cache notifications over the wire from the server. The only way for this to work in an NFS context is to go to NFSv4, which has other problems. If you can, spring for a symmetric pipe, instead (e.g SDSL); otherwise, you have a lot of work ahead of you, one way or another. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3E29D4D2.47A5E909>