From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 15:21:21 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 247453FE; Fri, 27 Feb 2015 15:21:21 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F11A1A59; Fri, 27 Feb 2015 15:21:20 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 90323B915; Fri, 27 Feb 2015 10:21:19 -0500 (EST) From: John Baldwin To: freebsd-net@freebsd.org Subject: Re: Accessing socket APIs soon after accept Date: Fri, 27 Feb 2015 10:20:52 -0500 Message-ID: <4258761.a1pROkzJe6@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: <1421428048190.48193@netapp.com> References: <1421339375968.94209@netapp.com> <1421428048190.48193@netapp.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 27 Feb 2015 10:21:19 -0500 (EST) Cc: Adrian Chadd , "Quattlebaum, Ryan" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 15:21:21 -0000 On Friday, January 16, 2015 05:07:28 PM Quattlebaum, Ryan wrote: > Hi, Adrian. Thanks for taking a look at this. > > We're using FreeBSD 8.2 and httpd-2.4.10 with arp-1.5.1 and apr-util-1.5.4. > > The problem we're seeing is pretty intermittent, so I hope this test case > can shine a little bit of light on the > problem. We tried debugging this on our own by adding calls to > getsockname() right after the accept call (in > srclib/apr/network_io/unix/sockets.c: apr_socket_accept()) and logging the > output. That's where we saw invalid data. > > I took a look at the source code for the TCP syncache module and the accept > syscall. It looks like the new child socket is available for the > application to accept after the call to sonewconn returns, but the address > information isn't set until further down in the function. Wouldn't this > open a window where an application could accept on a socket that the > syncache code isn't done configuring? This is a bug in 8.x it seems. It was fixed in HEAD in this commit: ------------------------------------------------------------------------ r261242 | gnn | 2014-01-28 15:28:32 -0500 (Tue, 28 Jan 2014) | 10 lines Decrease lock contention within the TCP accept case by removing the INP_INFO lock from tcp_usr_accept. As the PR/patch states this was following the advice already in the code. See the PR below for a full disucssion of this change and its measured effects. PR: 183659 Submitted by: Julian Charbon Reviewed by: jhb In particular, that commit changed the syncache code to not place the socket in the queue until the end of the function via soisconnected(). You can probably merge the tcp_syncache.c portion of that change back to 8.x without any ill effects and it should fix your problem. -- John Baldwin