From owner-freebsd-net@FreeBSD.ORG Thu May 2 16:03:22 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id ED69A882 for ; Thu, 2 May 2013 16:03:22 +0000 (UTC) (envelope-from realrichardsharpe@gmail.com) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) by mx1.freebsd.org (Postfix) with ESMTP id 84CE91682 for ; Thu, 2 May 2013 16:03:22 +0000 (UTC) Received: by mail-wi0-f179.google.com with SMTP id l13so766634wie.12 for ; Thu, 02 May 2013 09:03:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:content-transfer-encoding; bh=b9lRJMzukmMiIhbCW0MRzZ6ZQFeOzcbPqCZ+55wjQIA=; b=dG4GRYAEBEp0nLWffXfJDXG0D1AkN9kG/rS1E/tMI13kUbT8il0w0QY+rmOBAuWKPq LQw8ZZCYSQzkaN1a+GbKOtEmRFqwpbyI24jIEvB6N9rV9jaPpIlmcoyDlKdp9NHcTQqT QyMjKy7nb75DJddJm+eEDEii+Wrv/35KbVNVpMTOkD7QtU6hxyG1ghFyuKdkNMzMJ2wx 8jLIAwMnFVaOIqCSZYDDWLB1Z3uP3xgynzC12QQQHvUb6iGA/VyZFp5Ribm4JP2DyjDx 27qXpqh3wSz4dabmeTPw/qhYyVv8HNJ0PHDlXcmBOjoKryIrDaOFVVz4Yk60DygZGaaY XvBg== MIME-Version: 1.0 X-Received: by 10.180.21.167 with SMTP id w7mr8731060wie.2.1367510601592; Thu, 02 May 2013 09:03:21 -0700 (PDT) Received: by 10.194.179.194 with HTTP; Thu, 2 May 2013 09:03:21 -0700 (PDT) In-Reply-To: <51827DAA.2020009@vangyzen.net> References: <5181ECDF.1040905@mu.org> <51827DAA.2020009@vangyzen.net> Date: Thu, 2 May 2013 09:03:21 -0700 Message-ID: Subject: Re: Seeing EINVAL from writev on 8.0 to a non-blocking socket even though the data seems to hit the wire From: Richard Sharpe To: Eric van Gyzen Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, Alfred Perlstein X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 16:03:23 -0000 On Thu, May 2, 2013 at 7:52 AM, Eric van Gyzen wrote: > On 05/02/2013 08:48, Richard Sharpe wrote: >> On Wed, May 1, 2013 at 9:34 PM, Alfred Perlstein wrote: >>> On 5/1/13 8:03 PM, Richard Sharpe wrote: >>>> Hi folks, >>>> >>>> I am checking to see if there are any known bugs with respect to this >>>> in FreeBSD 8.0. >>>> >>>> Situation is that Samba 3.6.6 uses writev to a non-blocking socket to >>>> get the SMB2 requests on the wire. >>>> >>>> Intermittently, we see the writev return EINVAL even though the data >>>> has gotten on the wire. This I have verified by grabbing a capture and >>>> comparing the SMB Sequence number in the last outgoing packet on the >>>> wire vs the in-memory contents when we get EINVAL. >>>> >>>> Sometimes it occurs on a four-element IOVEC, sometimes we get EAGAIN >>>> on the four-element IOVEC and then we get EINVAL when retrying on a >>>> smaller IOVEC. >>>> >>>> Where should I look to check if there is some path where this might be >>>> happening? Is this even the correct mailing list? >>>> >>> What does the iovec look like when you get EINVAL? Can you sanity check >>> it? Is there anything special about it? (zero length vecs?) >>> >>> I think there are a few "maxvals" that if overrun cause EINVAL to be >>> returned. example is if your iovec is somehow huge or has many, many >>> elements. >> Can anyone tell me the call graph down to the TCP code? >> > > writev kern/sys_generic.c > kern_writev > dofilewrite > fo_write in sys/file.h > soo_write in kern/sys_socket.c > sosend in kern/uipc_socket.c > sosend_generic > tcp_usr_send in netinet/tcp_usrreq.c Thanks for that. The only places it seems that EINVAL could be coming from are the last two, and they seem to be valid error cases and one of those seem like it could never happen in this path (kern_writev sends down flags set to zero). Since it has the feel of a race, I need to find out of we have a race with signal handlers in some way and whether or not we are doing something weird with control messages or something on the socket. --=20 Regards, Richard Sharpe (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE)