From owner-freebsd-net@FreeBSD.ORG Sat May 4 20:30:36 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 25EE215A; Sat, 4 May 2013 20:30:36 +0000 (UTC) (envelope-from realrichardsharpe@gmail.com) Received: from mail-we0-x233.google.com (mail-we0-x233.google.com [IPv6:2a00:1450:400c:c03::233]) by mx1.freebsd.org (Postfix) with ESMTP id 8E882E84; Sat, 4 May 2013 20:30:35 +0000 (UTC) Received: by mail-we0-f179.google.com with SMTP id t9so2132498wey.10 for ; Sat, 04 May 2013 13:30:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:content-transfer-encoding; bh=HttCSscRMIXITaNGFKnLc9cjxU5DaNwfHF337vdAvqs=; b=FrylsITfTcBgZJCnNNxCoY0nvhh2233MA6idktxAiybT8EFQXNJDD2sTAoW6Lj42Ok 4QwjpJ022qjk/rEYvdwqJ/jFaruxHK4aQU5NIyYL2HZy4UWTVY0FiqWeZ7rZtx582Nk1 c0qWJs6gPdicWcbAoNuyDqAUWPZA8w3cfwMDrWHrxgH0p4ZLiLJjif632u5dAh0VNr2s QY1O4hYTk11yFMNh11bJhpBFvCVfAbizqSQyHeqPhMThZBAvswI8nhMvNhZsL16jtolb ak1ILFBqlpUtTctNQPo4l54pqD+WRgijSCFDhhkyZvaFwkVXAY1sSBnZ1mJiL9A5g2bq vqzw== MIME-Version: 1.0 X-Received: by 10.180.90.203 with SMTP id by11mr2937227wib.10.1367699434709; Sat, 04 May 2013 13:30:34 -0700 (PDT) Received: by 10.194.179.194 with HTTP; Sat, 4 May 2013 13:30:34 -0700 (PDT) In-Reply-To: References: Date: Sat, 4 May 2013 13:30:34 -0700 Message-ID: Subject: Re: Is there any way to limit the amount of data in an mbuf chain submitted to a driver? From: Richard Sharpe To: Adrian Chadd Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 20:30:36 -0000 On Sat, May 4, 2013 at 10:39 AM, Adrian Chadd wrote: > On 4 May 2013 06:52, Richard Sharpe wrote: >> Hi folks, >> >> I understand better why I am seeing EINVAL intermittently when sending >> data from Samba via SMB2. >> >> The ixgbe driver, for TSO reasons, limits the amount of data that can >> be DMA'd to 65535 bytes. It returns EINVAL for any mbuf chain larger >> than that. >> >> The SO_SNDBUF for that socket is set to 131972. Mostly there is less >> than 64kiB of space available, so that is all TCP etc can put into the >> socket in one chain of mbufs. However, every now and then there is >> more than 65535 bytes available in the socket buffers, and we have an >> SMB packet that is larger than 65535 bytes, and we get hit. >> >> To confirm this I am going to set SO_SNDBUF back to the default of >> 65536 and test again. My repros are very reliable. >> >> However, I wondered if my only way around this if I want to continue >> to use SO_SNDBUF sizes larger than 65536 is to fragment large mbuf >> chains in the driver? > > Hm, is this is a problem without TSO? We are using the card without TSO, so I am thinking of changing that limit to 131072 and retesting. I am currently testing with SO_SNDBUF=3D32768 and have not hit the problem. > Is the problem that the NIC can't handle a frame that big, or a buffer th= at big? > Ie - if you handed the hardware two descriptors of 64k each, for the > same IP datagram, will it complain? I can't find any documentation, but it seems that with TSO it cannot handle a frame that big. Actually, since we are not using TSO, there really should not be a problem with larger frames. > Or do you need to break it up into two separate IP datagrams, facing > the driver, with a maximum size of 64k each? Not sure, but it looks like we need to do that. --=20 Regards, Richard Sharpe (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE)