Date: Wed, 22 Jul 2009 16:07:34 -0700 (PDT) From: Matthew Dillon <dillon@apollo.backplane.com> To: Chris Hedley <freebsd-current@chrishedley.com> Cc: freebsd-current@freebsd.org Subject: Re: Linux NFS ate my bge Message-ID: <200907222307.n6MN7YhU012788@apollo.backplane.com> References: <alpine.BSF.2.00.0907222232310.3132@teapot.cbhnet>
next in thread | previous in thread | raw e-mail | index | archive | help
:requests.
:
:I've (re)discovered this now that I've recently got my assorted config
:problems largely ironed out and have my FreeBSD box up to date again, I'm
:reminded of an unresolved problem from way back, which is that my bge card
:collapses after being subjected to a large amount of NFS traffic coming
:from my Linux box, e.g. recompiling KDE on a discless workstation, which
:has been responsible for three embuggerances so far today.
If you are using a NFS UDP mount, try using a NFS TCP mount instead.
This could very well instantly fix your issues even if it does not
solve the underlying bugs.
With a UDP mount the server can get a barrage of IP fragmented UDP
packets, particularly from linux clients. While I don't know any
specific with regards to bugs in the bge driver itself, I do know that
for a UDP mount to operate adequately over a GigE network the NFS server
needs about a 2 Megabyte socket buffer to receive the bursts.
This is not something that would typically be seen between two FreeBSD
boxes as FreeBSD's asynchronous NFS client traffic (mainly read-ahead)
is limited by available synchronous nfsiod threads, and further limited
by default mount options, internal #defined concurrency limits, and
by the out-of-order transmission of the read ahead rpcs caused by the
way the nfsiod threads operate (which results in out-of-order processing
on the server side and stalls the linear reads done by the client).
Linux clients, on the other-hand can generate an enormous number of
concurrent RPCs.
Use of a TCP mount instead of a UDP mount solves the sockbuf and
IP fragmentation issues. The TCP connection will not use fragmented IP
packets, will not blow away the server's receive-side sockbuf, and
does a much better job dealing with any packet loss, to boot.
-Matt
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200907222307.n6MN7YhU012788>