From owner-freebsd-stable@FreeBSD.ORG Wed Jul 9 14:53:54 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 82B8DB6B for ; Wed, 9 Jul 2014 14:53:54 +0000 (UTC) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 406C0277C for ; Wed, 9 Jul 2014 14:53:54 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id EAD2E1FE02D; Wed, 9 Jul 2014 16:53:52 +0200 (CEST) Message-ID: <53BD5788.40308@selasky.org> Date: Wed, 09 Jul 2014 16:54:00 +0200 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Bob Healey , FreeBSD Stable Mailing List Subject: Re: Interactions with mxge, pf, nfsd, and the kernel References: <2035928323.8860293.1404849282793.JavaMail.root@uoguelph.ca> <53BD5607.9080406@rpi.edu> In-Reply-To: <53BD5607.9080406@rpi.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 14:53:54 -0000 On 07/09/14 16:47, Bob Healey wrote: > These machines primary purpose is to provide nfs services to an HPC > cluster (12 hosts, 384 cores) on a gigabit interconnection. mxge0 is a > 10G link to an HP Procurve 2910al with the hpc nodes connected to it, > existing in RFC 1918 space. bce1 is connected to the public network. > bce0 is only used for accessing the IPMI chipset. 9K packets are > enabled in what appears to be a futile attempt to improve nfs and mpi > performance within the cluster. I can revert back to 1500 byte packets, > and just chalk this up to being things mere mortals are not to meddle with. > > Bob Healey > Systems Administrator > Biocomputation and Bioinformatics Constellation > and Molecularium > healer@rpi.edu > (518) 276-4407 > Hi, Have you tried 4K packets instead of 9K ones? I understand the mbufs are special and must be contiguous in physical memory, so memory allocation fragmentation can lead up to starvation of mbufs. --HPS