Date: Wed, 16 Jul 2014 11:00:48 -0700 From: Adrian Chadd <adrian@freebsd.org> To: hiren panchasara <hiren.panchasara@gmail.com> Cc: "freebsd-net@freebsd.org" <net@freebsd.org> Subject: Re: UDP sendto() returning ENOBUFS - "No buffer space available" Message-ID: <CAJ-VmokEiZMpdfNjs%2B-C9pmRcjOOjjNGTvM88muh940sr7SmPw@mail.gmail.com> In-Reply-To: <CALCpEUE7OtbXjVTk2C8%2BV7fjOKutuNq04BTo0SN42YEgX81k-Q@mail.gmail.com> References: <CALCpEUE7OtbXjVTk2C8%2BV7fjOKutuNq04BTo0SN42YEgX81k-Q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi! So the UDP transmit path is udp_usrreqs->pru_send() == udp_send() -> udp_output() -> ip_output() udp_output() does do a M_PREPEND() which can return ENOBUFS. ip_output can also return ENOBUFS. it doesn't look like the socket code (eg sosend_dgram()) is doing any buffering - it's just copying the frame and stuffing it up to the driver. No queuing involved before the NIC. So a _well behaved_ driver will return ENOBUFS _and_ not queue the frame. However, it's entirely plausible that the driver isn't well behaved - the intel drivers screwed up here and there with transmit queue and failure to queue vs failure to transmit. So yeah, try tweaking the tx ring descriptor for the driver your'e using and see how big a bufring it's allocating. -a On 16 July 2014 01:58, hiren panchasara <hiren.panchasara@gmail.com> wrote: > Return values in sendto() manpage says: > > [ENOBUFS] The system was unable to allocate an internal buffer. > The operation may succeed when buffers become avail- > able. > > [ENOBUFS] The output queue for a network interface was full. > This generally indicates that the interface has > stopped sending, but may be caused by transient con- > gestion. > > If I hit the first condition, it should reflect as failures in > "netstat -m". Is that a correct assumption? > > I want to understand what happens when/if we hit the second condition. > And how to prevent that from happening. > Is it just application's job to rate-limit data it sends to the n/w > interface card so that it doesn't saturate? > Does kernel do any sort of queuing in the case of ENOBUFS? OR does the > message just gets dropped? > > For an application sending a lot of UDP data and returning ENOBUFS, > what all udp and other tunables I should tweak? I can only think of: > - number of tx ring descriptors - increasing this will get us more txds. > - kern.ipc.maxsockbuf: Increasing this will increase buffer size > allocated for sockets. > > what else? > > Any comments/suggestions/corrections? > > cheers, > Hiren > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-VmokEiZMpdfNjs%2B-C9pmRcjOOjjNGTvM88muh940sr7SmPw>