From owner-freebsd-net@FreeBSD.ORG Sun Jul 3 15:00:23 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 21F51106568C; Sun, 3 Jul 2011 15:00:23 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-fx0-f44.google.com (mail-fx0-f44.google.com [209.85.161.44]) by mx1.freebsd.org (Postfix) with ESMTP id DAC548FC12; Sun, 3 Jul 2011 15:00:13 +0000 (UTC) Received: by fxe6 with SMTP id 6so3709271fxe.17 for ; Sun, 03 Jul 2011 08:00:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:cc:subject:sender:date:message-id:user-agent:mime-version :content-type; bh=Ji0U38n0k/QJgxAWciSzBwjLh3W/5Kqi+TSb4HtxjaU=; b=K/REo8aCDIRt7jk/oFERdK6RXsdIHu173FrFgYkuFD+WKgprN73RqFG7lHgORgWxPG 5YKLSDEgoXUApcWFnKwwqDEjDU1QyMXQRxnv/p1LAs7dxNlkq7T5UfykC9fkwLwwz0Vk ZKwL0OxIsfXaJajrgSC5MOu6a5Gwot5NXsyBU= Received: by 10.223.4.136 with SMTP id 8mr1802397far.16.1309705212535; Sun, 03 Jul 2011 08:00:12 -0700 (PDT) Received: from localhost ([95.69.173.122]) by mx.google.com with ESMTPS id k17sm1233376fah.13.2011.07.03.08.00.10 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 03 Jul 2011 08:00:11 -0700 (PDT) From: Mikolaj Golub To: freebsd-net@freebsd.org Sender: Mikolaj Golub Date: Sun, 03 Jul 2011 18:00:09 +0300 Message-ID: <867h7zxvd2.fsf@kopusha.home.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Kostik Belousov , Pawel Jakub Dawidek , Andre Oppermann Subject: soreceive_stream: issues with O_NONBLOCK X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jul 2011 15:00:23 -0000 Hi, Trying soreceive_stream I found that many applications (like firefox, pidgin, gnus) might hang in soreceive_stream/sbwait. It was shown up that the issue was with O_NONBLOCK connections -- they blocked in recv() when should not have been. This can be checked with this simple test: http://people.freebsd.org/~trociny/test_nonblock.c In soreceive_stream we have the following code that looks wrong: 1968 /* Socket buffer is empty and we shall not block. */ 1969 if (sb->sb_cc == 0 && 1970 ((sb->sb_flags & SS_NBIO) || (flags & (MSG_DONTWAIT|MSG_NBIO)))) { 1971 error = EAGAIN; 1972 goto out; 1973 } It should check so->so_state agains SS_NBIO, not sb->sb_flags. But just changing this is not enough. This check is called too early -- before checking that socket state is not SBS_CANTRCVMORE. As a result, if the peer closes the connection recv() returns EAGAIN instead of 0. See this example: http://people.freebsd.org/~trociny/test_close.c So I moved the "nonblock" check below SBS_CANTRCVMORE check and ended up with this patch: http://people.freebsd.org/~trociny/uipc_socket.c.soreceive_stream.patch It works for me fine. Also, this part looks wrong: 1958 /* We will never ever get anything unless we are connected. */ 1959 if (!(so->so_state & (SS_ISCONNECTED|SS_ISDISCONNECTED))) { 1960 /* When disconnecting there may be still some data left. */ 1961 if (sb->sb_cc > 0) 1962 goto deliver; 1963 if (!(so->so_state & SS_ISDISCONNECTED)) 1964 error = ENOTCONN; 1965 goto out; 1966 } Why we check in 1959 that state is not SS_ISDISCONNECTED? If it is valid then the check at 1963 is useless becase it will be always true. Shouldn't it be something like below? if (!(so->so_state & (SS_ISCONNECTED|SS_ISCONNECTING))) { /* When disconnecting there may be still some data left. */ if (sb->sb_cc > 0) goto deliver; error = ENOTCONN; goto out; } (I don't see why we souldn't set ENOTCONN if the state is SS_ISDISCONNECTED). And the last :-). Currently, to try soreceive_stream one need to rebuild kernel with TCP_SORECEIVE_STREAM and then set tunable net.inet.tcp.soreceive_stream. Why do we need TCP_SORECEIVE_STREAM option? Wouldn't tunable be enough? It would simplify trying soreceive_stream by users and we might have more testing/feedback. -- Mikolaj Golub