From owner-freebsd-net@FreeBSD.ORG Sat May 4 17:39:19 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D10FF885 for ; Sat, 4 May 2013 17:39:19 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wg0-x22f.google.com (mail-wg0-x22f.google.com [IPv6:2a00:1450:400c:c00::22f]) by mx1.freebsd.org (Postfix) with ESMTP id 6D6857CC for ; Sat, 4 May 2013 17:39:19 +0000 (UTC) Received: by mail-wg0-f47.google.com with SMTP id e11so2412616wgh.26 for ; Sat, 04 May 2013 10:39:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=hA/LkLXebx9X8ljUIeQpoiWcV6Lit/5OxJgbgz2ymaU=; b=ssSaupNn26eI8FE2UV+5g3Ama0TEYOIef9NFQOGMLwa8blB1oMlOn2kLLHCLWZCwRy ds5yYPbZJFN5JWUaDdBp5bbgHiUo9FDH/eslJVbCZGh8dOzGkN+6lWMW8slgqMIiDdKA jMR4BuWzNWeQnQUC5a3KwIXQALiR6NDjErZigJOIuWoCfXfnb+6oGpJ69lYn96isADaz CnkYnerdwdv5t9gP9biSheBSgYqudg5qlaukCyRzR86awElISiUZSXszUQqx9M4bmT8I pAO9Boppc78NV7987dEtttsNh0IdYvEP1cU+sexf9eEq9hULQxzcfzE73jEi1iuNTy08 4Vaw== MIME-Version: 1.0 X-Received: by 10.180.149.200 with SMTP id uc8mr2646873wib.3.1367689158081; Sat, 04 May 2013 10:39:18 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.217.58.138 with HTTP; Sat, 4 May 2013 10:39:17 -0700 (PDT) In-Reply-To: References: Date: Sat, 4 May 2013 10:39:17 -0700 X-Google-Sender-Auth: vEznjvlbMgXa2LjS5-Ki0hYdsdY Message-ID: Subject: Re: Is there any way to limit the amount of data in an mbuf chain submitted to a driver? From: Adrian Chadd To: Richard Sharpe Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 17:39:19 -0000 On 4 May 2013 06:52, Richard Sharpe wrote: > Hi folks, > > I understand better why I am seeing EINVAL intermittently when sending > data from Samba via SMB2. > > The ixgbe driver, for TSO reasons, limits the amount of data that can > be DMA'd to 65535 bytes. It returns EINVAL for any mbuf chain larger > than that. > > The SO_SNDBUF for that socket is set to 131972. Mostly there is less > than 64kiB of space available, so that is all TCP etc can put into the > socket in one chain of mbufs. However, every now and then there is > more than 65535 bytes available in the socket buffers, and we have an > SMB packet that is larger than 65535 bytes, and we get hit. > > To confirm this I am going to set SO_SNDBUF back to the default of > 65536 and test again. My repros are very reliable. > > However, I wondered if my only way around this if I want to continue > to use SO_SNDBUF sizes larger than 65536 is to fragment large mbuf > chains in the driver? Hm, is this is a problem without TSO? Is the problem that the NIC can't handle a frame that big, or a buffer that big? Ie - if you handed the hardware two descriptors of 64k each, for the same IP datagram, will it complain? Or do you need to break it up into two separate IP datagrams, facing the driver, with a maximum size of 64k each? Adrian