Date: Tue, 28 Nov 2000 09:41:56 -0800 From: Lars Eggert <larse@ISI.EDU> To: Alfred Perlstein <bright@wintelcom.net> Cc: questions@FreeBSD.ORG Subject: Re: send(2) and resuming after ENOBUFS Message-ID: <3A23EE64.D50FD149@isi.edu> References: <3A2326E6.AD835284@isi.edu> <20001127220700.W8051@fw.wintelcom.net>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format.
--------------ms6F4E439A79416446E9D433E5
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Alfred Perlstein wrote:
>
> * Lars Eggert <larse@ISI.EDU> [001127 19:31] wrote:
...
> > My question is: How do I detect that the socket buffer has sufficiently
> > emptied to resume sending? I tried setting the SO_SNDLOWAT to my message
> > size and checking with select(2) if the socket is writable. However,
> > select(2) always returns telling me it is, even though a following send(2)
> > results in ENOBUFS. This may in fact be a bug in FreeBSD, since the
> > setsockopt(2) man page seems to indicate that this should work:
>
> ENOBUFS doesn't indicate that the socketbuffer is full
>
> 55 ENOBUFS No buffer space available. An operation on a socket or pipe
> was not performed because the system lacked sufficient buffer
> space or because a queue was full.
>
> opposed to:
>
> 35 EAGAIN Resource temporarily unavailable. This is a temporary condi-
> tion and later calls to the same routine may complete normally.
>
> but when you xref against write(2) you'll see:
>
> [EAGAIN] The file was marked for non-blocking I/O, and no data
> could be written immediately.
>
> What you need to do is increase the amount of bufferspace available in
> the kernel because your program is exhausing all the network resources
> available to it.
>
> Raise maxusers or nmbclusters.
I am running this with maxusers=128, and as far as I can tell, mbufs are
not the problem:
[root@hbo: /nfs/ruby/larse/projects/thesis/bin] netstat -m
139/2192/10240 mbufs in use (current/peak/max):
132 mbufs allocated to data
7 mbufs allocated to packet headers
128/1166/2560 mbuf clusters in use (current/peak/max)
2880 Kbytes allocated to network (37% of mb_map in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines
Looking at the send(2) man page, I agree that ENOBUFS doesn't mean that the
send buffer is full:
[ENOBUFS] The system was unable to allocate an internal buffer.
The operation may succeed when buffers become avail-
able.
[ENOBUFS] The output queue for a network interface was full.
This generally indicates that the interface has
stopped sending, but may be caused by transient con-
gestion.
However, since a too low maxusers is not the problem, I'm still looking for
a way of detecting when to resume sending after ENOBUFS, without
spinning...
Thanks,
Lars
--
Lars Eggert <larse@isi.edu> Information Sciences Institute
http://www.isi.edu/larse/ University of Southern California
--------------ms6F4E439A79416446E9D433E5
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature
MIIIIwYJKoZIhvcNAQcCoIIIFDCCCBACAQExCzAJBgUrDgMCGgUAMAsGCSqGSIb3DQEHAaCC
BfQwggLYMIICQaADAgECAgMDIwUwDQYJKoZIhvcNAQEEBQAwgZQxCzAJBgNVBAYTAlpBMRUw
EwYDVQQIEwxXZXN0ZXJuIENhcGUxFDASBgNVBAcTC0R1cmJhbnZpbGxlMQ8wDQYDVQQKEwZU
aGF3dGUxHTAbBgNVBAsTFENlcnRpZmljYXRlIFNlcnZpY2VzMSgwJgYDVQQDEx9QZXJzb25h
bCBGcmVlbWFpbCBSU0EgMTk5OS45LjE2MB4XDTAwMDgyNDIwMzAwOFoXDTAxMDgyNDIwMzAw
OFowVDEPMA0GA1UEBBMGRWdnZXJ0MQ0wCwYDVQQqEwRMYXJzMRQwEgYDVQQDEwtMYXJzIEVn
Z2VydDEcMBoGCSqGSIb3DQEJARYNbGFyc2VAaXNpLmVkdTCBnzANBgkqhkiG9w0BAQEFAAOB
jQAwgYkCgYEAz1yfcNs53rvhuw8gSDvr2+/snP8GduYY7x7WkJdyvcwb4oipNpWYIkMGP214
Zv1KrgvntGaG+jeugAGQt0n64VusgcIzQ6QDRtnMgdQDTAkVSQ2eLRSQka+nAPx6SFKJg79W
EEHmgKQBMtZdMBYtYv/mTOcpm7jTJVg+7W6n04UCAwEAAaN3MHUwKgYFK2UBBAEEITAfAgEA
MBowGAIBBAQTTDJ1TXlmZkJOVWJOSkpjZFoyczAYBgNVHREEETAPgQ1sYXJzZUBpc2kuZWR1
MAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUiKvxYINmVfTkWMdGHcBhvSPXw4wwDQYJKoZI
hvcNAQEEBQADgYEAi65fM/jSCaPhRoA9JW5X2FktSFhE5zkIpFVPpv33GWPPNrncsK13HfZm
s0B1rNy2vU7UhFI/vsJQgBJyffkLFgMCjp3uRZvBBjGD1q4yjDO5yfMMjquqBpZtRp5op3lT
d01faA58ZCB5sxCb0ORSxvXR8tc9DJO0JIpQILa6vIAwggMUMIICfaADAgECAgELMA0GCSqG
SIb3DQEBBAUAMIHRMQswCQYDVQQGEwJaQTEVMBMGA1UECBMMV2VzdGVybiBDYXBlMRIwEAYD
VQQHEwlDYXBlIFRvd24xGjAYBgNVBAoTEVRoYXd0ZSBDb25zdWx0aW5nMSgwJgYDVQQLEx9D
ZXJ0aWZpY2F0aW9uIFNlcnZpY2VzIERpdmlzaW9uMSQwIgYDVQQDExtUaGF3dGUgUGVyc29u
YWwgRnJlZW1haWwgQ0ExKzApBgkqhkiG9w0BCQEWHHBlcnNvbmFsLWZyZWVtYWlsQHRoYXd0
ZS5jb20wHhcNOTkwOTE2MTQwMTQwWhcNMDEwOTE1MTQwMTQwWjCBlDELMAkGA1UEBhMCWkEx
FTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTEUMBIGA1UEBxMLRHVyYmFudmlsbGUxDzANBgNVBAoT
BlRoYXd0ZTEdMBsGA1UECxMUQ2VydGlmaWNhdGUgU2VydmljZXMxKDAmBgNVBAMTH1BlcnNv
bmFsIEZyZWVtYWlsIFJTQSAxOTk5LjkuMTYwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGB
ALNpWpfU0BYLerXFXekhnCNyzRJMS/d+z8f7ynIk9EJSrFeV43theheE5/1yOTiUtOrtZaeS
Bl694GX2GbuUeXZMPrlocHWEHPQRdAC8BSxPCQMXMcz0QdRyxqZd4ohEsIsuxE3x8NaFPmzz
lZR4kX5A6ZzRjRVXjsJz5TDeRvVPAgMBAAGjNzA1MBIGA1UdEwEB/wQIMAYBAf8CAQAwHwYD
VR0jBBgwFoAUcknCczTGVfQLdnKBfnf0h+fGsg4wDQYJKoZIhvcNAQEEBQADgYEAa8ZZ6TH6
6bbssQPY33Jy/pFgSOrGVd178GeOxmFw523CpTfYnbcXKFYFi91cdW/GkZDGbGZxE9AQfGuR
b4bgITYtwdfqsgmtzy1txoNSm/u7/pyHnfy36XSS5FyXrvx+rMoNb3J6Zyxrc/WG+Z31AG70
HQfOnZ6CYynvkwl+Vd4xggH3MIIB8wIBATCBnDCBlDELMAkGA1UEBhMCWkExFTATBgNVBAgT
DFdlc3Rlcm4gQ2FwZTEUMBIGA1UEBxMLRHVyYmFudmlsbGUxDzANBgNVBAoTBlRoYXd0ZTEd
MBsGA1UECxMUQ2VydGlmaWNhdGUgU2VydmljZXMxKDAmBgNVBAMTH1BlcnNvbmFsIEZyZWVt
YWlsIFJTQSAxOTk5LjkuMTYCAwMjBTAJBgUrDgMCGgUAoIGxMBgGCSqGSIb3DQEJAzELBgkq
hkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTAwMTEyODE3NDE1NlowIwYJKoZIhvcNAQkEMRYE
FID5qZtMNjyoEMiQTuBmLIft1I0MMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYI
KoZIhvcNAwICAgCAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgFAMA0GCCqGSIb3DQMCAgEoMA0G
CSqGSIb3DQEBAQUABIGAM5+QTl9gA+FL/6/Vk+TM0mgzWkWDqjct9qmzukw9etHc5l/G9bmY
3QbqEh9n+AmMhtCpzQjNhy1Hcr2jDM7MZKLhtKZBsn1+q0kBe204aby4BoXNP6JdXjMc4n8E
X8z+ORlUQIbQdb4QycC1lHTOI4Hnudl/E/hOLns/XJFrY8k=
--------------ms6F4E439A79416446E9D433E5--
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3A23EE64.D50FD149>