Skip site navigation (1)Skip section navigation (2) 
Date:      Tue, 28 Nov 2000 09:41:56 -0800
From:      Lars Eggert <larse@ISI.EDU>
To:        Alfred Perlstein <bright@wintelcom.net>
Cc:        questions@FreeBSD.ORG
Subject:   Re: send(2) and resuming after ENOBUFS
Message-ID:  <3A23EE64.D50FD149@isi.edu>
References:  <3A2326E6.AD835284@isi.edu> <20001127220700.W8051@fw.wintelcom.net>
next in thread | previous in thread | raw e-mail | index | archive | help 

[-- Attachment #1 --]
Alfred Perlstein wrote:
> 
> * Lars Eggert <larse@ISI.EDU> [001127 19:31] wrote:
...
> > My question is: How do I detect that the socket buffer has sufficiently
> > emptied to resume sending? I tried setting the SO_SNDLOWAT to my message
> > size and checking with select(2) if the socket is writable. However,
> > select(2) always returns telling me it is, even though a following send(2)
> > results in ENOBUFS. This may in fact be a bug in FreeBSD, since the
> > setsockopt(2) man page seems to indicate that this should work:
> 
> ENOBUFS doesn't indicate that the socketbuffer is full
> 
>      55 ENOBUFS No buffer space available.  An operation on a socket or pipe
>              was not performed because the system lacked sufficient buffer
>              space or because a queue was full.
> 
> opposed to:
> 
>      35 EAGAIN Resource temporarily unavailable.  This is a temporary condi-
>              tion and later calls to the same routine may complete normally.
> 
> but when you xref against write(2) you'll see:
> 
>      [EAGAIN]           The file was marked for non-blocking I/O, and no data
>                         could be written immediately.
> 
> What you need to do is increase the amount of bufferspace available in
> the kernel because your program is exhausing all the network resources
> available to it.
> 
> Raise maxusers or nmbclusters.

I am running this with maxusers=128, and as far as I can tell, mbufs are
not the problem:

[root@hbo: /nfs/ruby/larse/projects/thesis/bin] netstat -m
139/2192/10240 mbufs in use (current/peak/max):
        132 mbufs allocated to data
        7 mbufs allocated to packet headers
128/1166/2560 mbuf clusters in use (current/peak/max)
2880 Kbytes allocated to network (37% of mb_map in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines

Looking at the send(2) man page, I agree that ENOBUFS doesn't mean that the
send buffer is full:

 [ENOBUFS]     The system was unable to allocate an internal buffer.
               The operation may succeed when buffers become avail-
               able.

 [ENOBUFS]     The output queue for a network interface was full.
               This generally indicates that the interface has
               stopped sending, but may be caused by transient con-
               gestion.

However, since a too low maxusers is not the problem, I'm still looking for
a way of detecting when to resume sending after ENOBUFS, without
spinning...

Thanks,
Lars
-- 
Lars Eggert <larse@isi.edu>                 Information Sciences Institute
http://www.isi.edu/larse/                University of Southern California
[-- Attachment #2 --]
0#	*H
010	+0	*H
00A#0
	*H
010	UZA10UWestern Cape10UDurbanville10
U
Thawte10UCertificate Services1(0&UPersonal Freemail RSA 1999.9.160
000824203008Z
010824203008Z0T10
UEggert1
0U*Lars10ULars Eggert10	*H
	
larse@isi.edu00
	*H
0\p9޻ H;v֐r∩6"C?mxfJf7I[3CF́L	I
-zHRVA怤2]0-bL)%X>nӅw0u0*+e!000L2uMyffBNUbNJJcdZ2s0U0
larse@isi.edu0U00U#0`fUXFa#Ì0
	*H
_3	F=%nWY-HXD9UOc6ܰwf@uܶNԄR?Pr}E1֮23mFhySwM_h|d yR=$P 00}0
	*H
010	UZA10UWestern Cape10U	Cape Town10U
Thawte Consulting1(0&UCertification Services Division1$0"UThawte Personal Freemail CA1+0)	*H
	personal-freemail@thawte.com0
990916140140Z
010915140140Z010	UZA10UWestern Cape10UDurbanville10
U
Thawte10UCertificate Services1(0&UPersonal Freemail RSA 1999.9.1600
	*H
0iZz]!#rLK~r$BRW{azr98e^eyvL>hput,O	1ArƦ]D.Mօ>lx~@эWs0FO7050U00U#0rIs4Uvr~wƲ0
	*H
kY1rr`HU{gapm¥7؝(V\uoƑlfq|ko!6-	-mƃRt\~
orzg,ksnΝc)	~U100010	UZA10UWestern Cape10UDurbanville10
U
Thawte10UCertificate Services1(0&UPersonal Freemail RSA 1999.9.16#0	+0	*H
	1	*H
0	*H
	1
001128174156Z0#	*H
	1L6<ȐNf,ԍ0R	*H
	1E0C0
*H
0*H
0+0
*H
@0
*H
(0
	*H
3N_`KՓh3ZE7-L=z_g	Щ͇-GrdᴦA}~I{m8i?]^3_9T@ut#N.{?\kc

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3A23EE64.D50FD149>