Date: Sun, 10 Mar 2013 20:04:06 -0400 (EDT) From: Rick Macklem <rmacklem@uoguelph.ca> To: Andre Oppermann <andre@freebsd.org> Cc: freebsd-net@freebsd.org, Jack Vogel <jfvogel@gmail.com>, Garrett Wollman <wollman@bimajority.org> Subject: Re: Limits on jumbo mbuf cluster allocation Message-ID: <564543211.3749080.1362960246372.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <513D07A4.4010503@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Andre Oppermann wrote:
> On 10.03.2013 07:04, Garrett Wollman wrote:
> > <<On Fri, 8 Mar 2013 12:13:28 -0800, Jack Vogel <jfvogel@gmail.com>
> > said:
> >
> >> Yes, in the past the code was in this form, it should work fine
> >> Garrett,
> >> just make sure
> >> the 4K pool is large enough.
> >
> > [Andre Oppermann's patch:]
> >>> if (adapter->max_frame_size <= 2048)
> > adapter-> rx_mbuf_sz = MCLBYTES;
> >>> - else if (adapter->max_frame_size <= 4096)
> >>> + else
> > adapter-> rx_mbuf_sz = MJUMPAGESIZE;
> >>> - else if (adapter->max_frame_size <= 9216)
> >>> - adapter->rx_mbuf_sz = MJUM9BYTES;
> >>> - else
> >>> - adapter->rx_mbuf_sz = MJUM16BYTES;
> >
> > So I tried exactly this, and it certainly worked insofar as only 4k
> > clusters were allocated, but NFS performance went down precipitously
> > (to fewer than 100 ops/s where normally it would be doing 2000
> > ops/s). I took a tcpdump while it was in this state, which I will
> > try
> > to make some sense of when I get back to the office. (It wasn't
> > livelocked; in fact, the server was mostly idle, but responses would
> > take seconds rather than milliseconds -- assuming the client could
> > even successfully mount the server at all, which the Debian
> > automounter frequently refused to do.)
>
> This is very weird and unlikely to come from the 4k mbufs by itself
> considering they are in heavy use in the write() path. Such a high
> delay smells like an issue in either the driver dealing with multiple
> mbufs per packet or nfs having a problem with it.
>
I am not aware of anything within the NFS server that would care. The
code simply believes the m_len field.
--> However, this is a good way to reduce server load. At 100ops/sec
I'd think you shouldn't have any server resource exhaustion issues.
--> Problem solved ;-);-)
rick
> > I ended up reverting back to the old kernel (which I managed to lose
> > the sources for), and once I get my second server up, I will try to
> > do
> > some more testing to see if I can identify the source of the
> > problem.
>
> --
> Andre
>
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?564543211.3749080.1362960246372.JavaMail.root>