Date: Sat, 20 Oct 2012 22:53:31 +0300 From: Nikolay Denev <ndenev@gmail.com> To: Outback Dingo <outbackdingo@gmail.com> Cc: "freebsd-hackers@freebsd.org Hackers" <freebsd-hackers@freebsd.org>, Rick Macklem <rmacklem@uoguelph.ca>, Ivan Voras <ivoras@freebsd.org> Subject: Re: NFS server bottlenecks Message-ID: <A16B8A34-D365-4A4B-9EB4-F2419B6BC430@gmail.com> In-Reply-To: <CAKYr3zzvb%2BiJzX8zmxUeH1ZjNEnc1FuuE5SdmYUAgQH84O64Mg@mail.gmail.com> References: <CAF-QHFWY0drcrUpo7GGD1zQNSDWsEeB_LHAjEbUKrX2ovQHNxw@mail.gmail.com> <191784842.2570110.1350737132305.JavaMail.root@erie.cs.uoguelph.ca> <CAF-QHFXB=yfD2EPoQf4C8YyX=0BA0Awndg0QNsWO8_rq=StHhQ@mail.gmail.com> <CAKYr3zzvb%2BiJzX8zmxUeH1ZjNEnc1FuuE5SdmYUAgQH84O64Mg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Oct 20, 2012, at 10:45 PM, Outback Dingo <outbackdingo@gmail.com> = wrote: > On Sat, Oct 20, 2012 at 3:28 PM, Ivan Voras <ivoras@freebsd.org> = wrote: >> On 20 October 2012 14:45, Rick Macklem <rmacklem@uoguelph.ca> wrote: >>> Ivan Voras wrote: >>=20 >>>> I don't know how to interpret the rise in context switches; as this = is >>>> kernel code, I'd expect no context switches. I hope someone else = can >>>> explain. >>>>=20 >>> Don't the mtx_lock() calls spin for a little while and then context >>> switch if another thread still has it locked? >>=20 >> Yes, but are in-kernel context switches also counted? I was assuming >> they are light-weight enough not to count. >>=20 >>> Hmm, I didn't look, but were there any tests using UDP mounts? >>> (I would have thought that your patch would mainly affect UDP = mounts, >>> since that is when my version still has the single LRU queue/mutex. >>=20 >> Another assumption - I thought UDP was the default. >>=20 >>> As I think you know, my concern with your patch would be correctness >>> for UDP, not performance.) >>=20 >> Yes. >=20 > Ive got a similar box config here, with 2x 10GB intel nics, and 24 2TB > drives on an LSI controller. > Im watching the thread patiently, im kinda looking for results, and > answers, Though Im also tempted to > run benchmarks on my system also see if i get similar results I also > considered that netmap might be one > but not quite sure if it would help NFS, since its to hard to tell if > its a network bottle neck, though it appears > to be network related. >=20 >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to = "freebsd-hackers-unsubscribe@freebsd.org" Doesn't look like network issue to me. =46rom my observations it's more = like some overhead in nfs and arc. The boxes easily push 10G with simple iperf test. Running two iperf test over each port of the dual ported 10G nics gives = 960MB/sec regardles which machine is the server. Also, I've seen over 960Gb/sec over NFS with this setup, but I can't = understand what type of workload was able to do this. At some point I was able to do this with simple dd, then after a reboot = I was no longer to push this traffic. I'm thinking something like ARC/kmem fragmentation might be the issue? =20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A16B8A34-D365-4A4B-9EB4-F2419B6BC430>