From owner-freebsd-questions Tue Nov 28 9:42: 2 2000 Delivered-To: freebsd-questions@freebsd.org Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by hub.freebsd.org (Postfix) with ESMTP id 6438737B400 for ; Tue, 28 Nov 2000 09:41:57 -0800 (PST) Received: from isi.edu (hbo.isi.edu [128.9.160.75]) by boreas.isi.edu (8.9.3/8.9.3) with ESMTP id JAA11481; Tue, 28 Nov 2000 09:41:56 -0800 (PST) Message-ID: <3A23EE64.D50FD149@isi.edu> Date: Tue, 28 Nov 2000 09:41:56 -0800 From: Lars Eggert Organization: USC Information Sciences Institute X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en, de MIME-Version: 1.0 To: Alfred Perlstein Cc: questions@FreeBSD.ORG Subject: Re: send(2) and resuming after ENOBUFS References: <3A2326E6.AD835284@isi.edu> <20001127220700.W8051@fw.wintelcom.net> Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms6F4E439A79416446E9D433E5" Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG This is a cryptographically signed message in MIME format. --------------ms6F4E439A79416446E9D433E5 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Alfred Perlstein wrote: > > * Lars Eggert [001127 19:31] wrote: ... > > My question is: How do I detect that the socket buffer has sufficiently > > emptied to resume sending? I tried setting the SO_SNDLOWAT to my message > > size and checking with select(2) if the socket is writable. However, > > select(2) always returns telling me it is, even though a following send(2) > > results in ENOBUFS. This may in fact be a bug in FreeBSD, since the > > setsockopt(2) man page seems to indicate that this should work: > > ENOBUFS doesn't indicate that the socketbuffer is full > > 55 ENOBUFS No buffer space available. An operation on a socket or pipe > was not performed because the system lacked sufficient buffer > space or because a queue was full. > > opposed to: > > 35 EAGAIN Resource temporarily unavailable. This is a temporary condi- > tion and later calls to the same routine may complete normally. > > but when you xref against write(2) you'll see: > > [EAGAIN] The file was marked for non-blocking I/O, and no data > could be written immediately. > > What you need to do is increase the amount of bufferspace available in > the kernel because your program is exhausing all the network resources > available to it. > > Raise maxusers or nmbclusters. I am running this with maxusers=128, and as far as I can tell, mbufs are not the problem: [root@hbo: /nfs/ruby/larse/projects/thesis/bin] netstat -m 139/2192/10240 mbufs in use (current/peak/max): 132 mbufs allocated to data 7 mbufs allocated to packet headers 128/1166/2560 mbuf clusters in use (current/peak/max) 2880 Kbytes allocated to network (37% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines Looking at the send(2) man page, I agree that ENOBUFS doesn't mean that the send buffer is full: [ENOBUFS] The system was unable to allocate an internal buffer. The operation may succeed when buffers become avail- able. [ENOBUFS] The output queue for a network interface was full. This generally indicates that the interface has stopped sending, but may be caused by transient con- gestion. However, since a too low maxusers is not the problem, I'm still looking for a way of detecting when to resume sending after ENOBUFS, without spinning... Thanks, Lars -- Lars Eggert Information Sciences Institute http://www.isi.edu/larse/ University of Southern California --------------ms6F4E439A79416446E9D433E5 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIIIIwYJKoZIhvcNAQcCoIIIFDCCCBACAQExCzAJBgUrDgMCGgUAMAsGCSqGSIb3DQEHAaCC BfQwggLYMIICQaADAgECAgMDIwUwDQYJKoZIhvcNAQEEBQAwgZQxCzAJBgNVBAYTAlpBMRUw EwYDVQQIEwxXZXN0ZXJuIENhcGUxFDASBgNVBAcTC0R1cmJhbnZpbGxlMQ8wDQYDVQQKEwZU aGF3dGUxHTAbBgNVBAsTFENlcnRpZmljYXRlIFNlcnZpY2VzMSgwJgYDVQQDEx9QZXJzb25h bCBGcmVlbWFpbCBSU0EgMTk5OS45LjE2MB4XDTAwMDgyNDIwMzAwOFoXDTAxMDgyNDIwMzAw OFowVDEPMA0GA1UEBBMGRWdnZXJ0MQ0wCwYDVQQqEwRMYXJzMRQwEgYDVQQDEwtMYXJzIEVn Z2VydDEcMBoGCSqGSIb3DQEJARYNbGFyc2VAaXNpLmVkdTCBnzANBgkqhkiG9w0BAQEFAAOB jQAwgYkCgYEAz1yfcNs53rvhuw8gSDvr2+/snP8GduYY7x7WkJdyvcwb4oipNpWYIkMGP214 Zv1KrgvntGaG+jeugAGQt0n64VusgcIzQ6QDRtnMgdQDTAkVSQ2eLRSQka+nAPx6SFKJg79W EEHmgKQBMtZdMBYtYv/mTOcpm7jTJVg+7W6n04UCAwEAAaN3MHUwKgYFK2UBBAEEITAfAgEA MBowGAIBBAQTTDJ1TXlmZkJOVWJOSkpjZFoyczAYBgNVHREEETAPgQ1sYXJzZUBpc2kuZWR1 MAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUiKvxYINmVfTkWMdGHcBhvSPXw4wwDQYJKoZI hvcNAQEEBQADgYEAi65fM/jSCaPhRoA9JW5X2FktSFhE5zkIpFVPpv33GWPPNrncsK13HfZm s0B1rNy2vU7UhFI/vsJQgBJyffkLFgMCjp3uRZvBBjGD1q4yjDO5yfMMjquqBpZtRp5op3lT d01faA58ZCB5sxCb0ORSxvXR8tc9DJO0JIpQILa6vIAwggMUMIICfaADAgECAgELMA0GCSqG SIb3DQEBBAUAMIHRMQswCQYDVQQGEwJaQTEVMBMGA1UECBMMV2VzdGVybiBDYXBlMRIwEAYD VQQHEwlDYXBlIFRvd24xGjAYBgNVBAoTEVRoYXd0ZSBDb25zdWx0aW5nMSgwJgYDVQQLEx9D ZXJ0aWZpY2F0aW9uIFNlcnZpY2VzIERpdmlzaW9uMSQwIgYDVQQDExtUaGF3dGUgUGVyc29u YWwgRnJlZW1haWwgQ0ExKzApBgkqhkiG9w0BCQEWHHBlcnNvbmFsLWZyZWVtYWlsQHRoYXd0 ZS5jb20wHhcNOTkwOTE2MTQwMTQwWhcNMDEwOTE1MTQwMTQwWjCBlDELMAkGA1UEBhMCWkEx FTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTEUMBIGA1UEBxMLRHVyYmFudmlsbGUxDzANBgNVBAoT BlRoYXd0ZTEdMBsGA1UECxMUQ2VydGlmaWNhdGUgU2VydmljZXMxKDAmBgNVBAMTH1BlcnNv bmFsIEZyZWVtYWlsIFJTQSAxOTk5LjkuMTYwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGB ALNpWpfU0BYLerXFXekhnCNyzRJMS/d+z8f7ynIk9EJSrFeV43theheE5/1yOTiUtOrtZaeS Bl694GX2GbuUeXZMPrlocHWEHPQRdAC8BSxPCQMXMcz0QdRyxqZd4ohEsIsuxE3x8NaFPmzz lZR4kX5A6ZzRjRVXjsJz5TDeRvVPAgMBAAGjNzA1MBIGA1UdEwEB/wQIMAYBAf8CAQAwHwYD VR0jBBgwFoAUcknCczTGVfQLdnKBfnf0h+fGsg4wDQYJKoZIhvcNAQEEBQADgYEAa8ZZ6TH6 6bbssQPY33Jy/pFgSOrGVd178GeOxmFw523CpTfYnbcXKFYFi91cdW/GkZDGbGZxE9AQfGuR b4bgITYtwdfqsgmtzy1txoNSm/u7/pyHnfy36XSS5FyXrvx+rMoNb3J6Zyxrc/WG+Z31AG70 HQfOnZ6CYynvkwl+Vd4xggH3MIIB8wIBATCBnDCBlDELMAkGA1UEBhMCWkExFTATBgNVBAgT DFdlc3Rlcm4gQ2FwZTEUMBIGA1UEBxMLRHVyYmFudmlsbGUxDzANBgNVBAoTBlRoYXd0ZTEd MBsGA1UECxMUQ2VydGlmaWNhdGUgU2VydmljZXMxKDAmBgNVBAMTH1BlcnNvbmFsIEZyZWVt YWlsIFJTQSAxOTk5LjkuMTYCAwMjBTAJBgUrDgMCGgUAoIGxMBgGCSqGSIb3DQEJAzELBgkq hkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTAwMTEyODE3NDE1NlowIwYJKoZIhvcNAQkEMRYE FID5qZtMNjyoEMiQTuBmLIft1I0MMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYI KoZIhvcNAwICAgCAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgFAMA0GCCqGSIb3DQMCAgEoMA0G CSqGSIb3DQEBAQUABIGAM5+QTl9gA+FL/6/Vk+TM0mgzWkWDqjct9qmzukw9etHc5l/G9bmY 3QbqEh9n+AmMhtCpzQjNhy1Hcr2jDM7MZKLhtKZBsn1+q0kBe204aby4BoXNP6JdXjMc4n8E X8z+ORlUQIbQdb4QycC1lHTOI4Hnudl/E/hOLns/XJFrY8k= --------------ms6F4E439A79416446E9D433E5-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message