Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 20 Oct 2012 22:53:31 +0300
From:      Nikolay Denev <ndenev@gmail.com>
To:        Outback Dingo <outbackdingo@gmail.com>
Cc:        "freebsd-hackers@freebsd.org Hackers" <freebsd-hackers@freebsd.org>, Rick Macklem <rmacklem@uoguelph.ca>, Ivan Voras <ivoras@freebsd.org>
Subject:   Re: NFS server bottlenecks
Message-ID:  <A16B8A34-D365-4A4B-9EB4-F2419B6BC430@gmail.com>
In-Reply-To: <CAKYr3zzvb%2BiJzX8zmxUeH1ZjNEnc1FuuE5SdmYUAgQH84O64Mg@mail.gmail.com>
References:  <CAF-QHFWY0drcrUpo7GGD1zQNSDWsEeB_LHAjEbUKrX2ovQHNxw@mail.gmail.com> <191784842.2570110.1350737132305.JavaMail.root@erie.cs.uoguelph.ca> <CAF-QHFXB=yfD2EPoQf4C8YyX=0BA0Awndg0QNsWO8_rq=StHhQ@mail.gmail.com> <CAKYr3zzvb%2BiJzX8zmxUeH1ZjNEnc1FuuE5SdmYUAgQH84O64Mg@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help

On Oct 20, 2012, at 10:45 PM, Outback Dingo <outbackdingo@gmail.com> =
wrote:

> On Sat, Oct 20, 2012 at 3:28 PM, Ivan Voras <ivoras@freebsd.org> =
wrote:
>> On 20 October 2012 14:45, Rick Macklem <rmacklem@uoguelph.ca> wrote:
>>> Ivan Voras wrote:
>>=20
>>>> I don't know how to interpret the rise in context switches; as this =
is
>>>> kernel code, I'd expect no context switches. I hope someone else =
can
>>>> explain.
>>>>=20
>>> Don't the mtx_lock() calls spin for a little while and then context
>>> switch if another thread still has it locked?
>>=20
>> Yes, but are in-kernel context switches also counted? I was assuming
>> they are light-weight enough not to count.
>>=20
>>> Hmm, I didn't look, but were there any tests using UDP mounts?
>>> (I would have thought that your patch would mainly affect UDP =
mounts,
>>> since that is when my version still has the single LRU queue/mutex.
>>=20
>> Another assumption - I thought UDP was the default.
>>=20
>>> As I think you know, my concern with your patch would be correctness
>>> for UDP, not performance.)
>>=20
>> Yes.
>=20
> Ive got a similar box config here, with 2x 10GB intel nics, and 24 2TB
> drives on an LSI controller.
> Im watching the thread patiently, im kinda looking for results, and
> answers, Though Im also tempted to
> run benchmarks on my system also see if i get similar results I also
> considered that netmap might be one
> but not quite sure if it would help NFS, since its to hard to tell if
> its a network bottle neck, though it appears
> to be network related.
>=20
>> _______________________________________________
>> freebsd-hackers@freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
>> To unsubscribe, send any mail to =
"freebsd-hackers-unsubscribe@freebsd.org"

Doesn't look like network issue to me. =46rom my observations it's more =
like some overhead in nfs and arc.
The boxes easily push 10G with simple iperf test.
Running two iperf test over each port of the dual ported 10G nics gives =
960MB/sec regardles which machine is the server.
Also, I've seen over 960Gb/sec over NFS with this setup, but I can't =
understand what type of workload was able to do this.
At some point I was able to do this with simple dd, then after a reboot =
I was no longer to push this traffic.
I'm thinking something like ARC/kmem fragmentation might be the issue?
=20




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A16B8A34-D365-4A4B-9EB4-F2419B6BC430>