From owner-freebsd-net@FreeBSD.ORG Wed May 6 15:33:40 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D089FA29 for ; Wed, 6 May 2015 15:33:40 +0000 (UTC) Received: from kerio.tuxis.nl (alcyone.saas.tuxis.net [31.3.111.19]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D7BC11613 for ; Wed, 6 May 2015 15:33:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=tuxis.nl; s=mail; h=from:subject:date:message-id:to:mime-version:content-type:in-reply-to: references; bh=wg3HAHEDc8WSe9ytGmDg/AWfMkNKzdnuB8Yysw1HghY=; b=XMPr0aWePCWpSKVka5yxdMP8IizimaWiRUyLRuD4u8VBaIP4s82jzg/GJb7s39YZIYoFy/Zgk34Vw Jh4/ATPUL5Np8/XTJPwvdTaKI6navDhoRHS3vdR5Wx2CuMUsQDtPyWTrGy4MyZTjngZCHxunNkYst7 SXYGSbBxuoTkNahhwd10cs1HYkW4Hav++fWM7p4LjmKppsW124Eux5JN6b7pwM1bbcguf8XcTWFsfr tx1iNurghmT8jK3od7DkzTb1lm4kt6q8lkfEzH7BxpEJhokO0H7wdOIy7K/23+rC33saUXPEQEoUke ZxnJFA6NVFmL7dpLRqGgsLp84wGmOxw== X-Footer: dHV4aXMubmw= Received: from [31.3.104.222] ([31.3.104.222]) (authenticated user mark@tuxis.nl) by kerio.tuxis.nl (Kerio Connect 8.5.0 RC 1) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128 bits)) for freebsd-net@freebsd.org; Wed, 6 May 2015 17:03:25 +0200 Message-ID: <554A2D3D.3060408@tuxis.nl> Date: Wed, 06 May 2015 17:03:25 +0200 From: Mark Schouten User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Icedove/31.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Frequent hickups on the networking layer References: <137094161.27589033.1430255162390.JavaMail.root@uoguelph.ca> <5540889A.5030904@tuxis.nl> <21824.58754.452182.195043@hergotha.csail.mit.edu> In-Reply-To: <21824.58754.452182.195043@hergotha.csail.mit.edu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 May 2015 15:33:40 -0000 Hi, On 04/29/2015 04:06 PM, Garrett Wollman wrote: > If you're using one of the drivers that has this problem, then yes, > keeping your layer-2 MTU/MRU below 4096 will probably cause it to use > 4k (page-sized) clusters instead, which are perfectly safe. > > As a side note, at least on the hardware I have to support, Infiniband > is limited to 4k MTU -- so I have one "jumbo" network with 4k frames > (that's bridged to IB) and one with 9k frames (that everything else > uses). So I was thinking, a customer of mine runs mostly the same setup, and has no issues at all. The only difference, MTU of 1500 vs MTU of 9000. I also created a graph in munin, graphing the number of mbuf_jumbo requests and failures. I find that when lots of writes occur to the iscsi-layer, the number of failed requests grow, and so so the number of errors on the ethernet interface. See attached images. My customer is also not suffering from crashing ctld-daemons, which crashes every other minute in my setup. So tonight I'm going to switch to an MTU of 1500, I'll let you know if that helped. Regards, Mark Schouten