From owner-freebsd-arch Mon Mar 3 14:44:28 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4989A37B401 for ; Mon, 3 Mar 2003 14:44:26 -0800 (PST) Received: from perrin.int.nxad.com (internal.ext.nxad.com [69.1.70.251]) by mx1.FreeBSD.org (Postfix) with ESMTP id C9EAD43FDF for ; Mon, 3 Mar 2003 14:44:25 -0800 (PST) (envelope-from sean@perrin.int.nxad.com) Received: by perrin.int.nxad.com (Postfix, from userid 1001) id 772AF2107C; Mon, 3 Mar 2003 14:44:18 -0800 (PST) Date: Mon, 3 Mar 2003 14:44:18 -0800 From: Sean Chittenden To: freebsd-arch@FreeBSD.ORG Subject: Should sendfile() to return ENOBUFS? Message-ID: <20030303224418.GU79234@perrin.int.nxad.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="1hKfHPzOXWu1rh0v" Content-Disposition: inline User-Agent: Mutt/1.4i X-PGP-Key: finger seanc@FreeBSD.org X-PGP-Fingerprint: 3849 3760 1AFE 7B17 11A0 83A6 DD99 E31F BC84 B341 X-Web-Homepage: http://sean.chittenden.org/ Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --1hKfHPzOXWu1rh0v Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I've got a cluster of busy servers and have now exhausted the number of sf_buf's available. top(1) reports that applications using sendfile(2) are spending quite a bit of time in the 'sfbufa' state blocking even though the socket is non-blocking. I'd consider this a pretty nice bug and that sendfile(2) should return ENOBUFS instead of blocking on a non-blocking call. Right now if sf_buf_alloc() returns NULL, it is assumed that the call was sent a signal and was interrupted. So I have a two fold question: 1) Should sendfile(2) block on a non-blocking socket when there are no sf_buf's available? I don't think it should. sendfile(2) should return ENOBUFS and let the user land process continue working even though the kernel is constrained for sf_buf's. 2) Will changing the sendfile() call to return ENOBUFs break source compatibility across sendfile() implementations? I'm pretty amazed that other performance mongers haven't run across this problem and noticed this before under high load conditions. If there is a precedent, I'm tempted to suggest that we break ranks because: *) the current behavior is pretty clearly not "the right thing" *) is pretty clearly broken, and *) is causing me no end of headaches (using writev() is a terrible alternative to sendfile()). Comments? -sc --=20 Sean Chittenden --1hKfHPzOXWu1rh0v Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Comment: Sean Chittenden iD8DBQE+Y9rC3ZnjH7yEs0ERAl99AJ47H8wu0FBJ3Qqv9shnyMrqjgZEnQCglUXX 6/MGUFEN5qdDAFaq+pJsuxc= =Hmv+ -----END PGP SIGNATURE----- --1hKfHPzOXWu1rh0v-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message