From owner-freebsd-current@FreeBSD.ORG Wed May 20 05:57:29 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AC92C1065673 for ; Wed, 20 May 2009 05:57:29 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.233]) by mx1.freebsd.org (Postfix) with ESMTP id 772C68FC08 for ; Wed, 20 May 2009 05:57:29 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: by rv-out-0506.google.com with SMTP id k40so96016rvb.43 for ; Tue, 19 May 2009 22:57:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=QDcADHy2j+TtqEH/mNXYLxbFAWquDEE+Fm72qfJpxFc=; b=KoR7tqgUU6lbtTpq7Gn+mZN2TsGCobBWlxsDuSkFTydNJzwDwQsCnUIqnVRKBDNAJA UYKTZL9yqovjYX+IY3qE7AKEQNqU18NWJ/xbG+OWpDM50flgnUSK7H1DEqZVtioYgEpx 4fTybT8IryObq/8PicdSqXppaTkzsET7A3DzY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=GpWONDQXHef661LPQwGZh8TpnIucjy/o6+bp9t10iHsEDh5OMx6oV6AZAdDb4TnqvL Wj3ANtTR7LoQFOYZAF6QIWpTXobxEnWerg6ZcAePomQ6DPCKfddpHMAID7g5dVbHkI0r FzMWBp4fybp7q9EErkeCvxXXMDLqBZqXaa0d0= Received: by 10.142.163.13 with SMTP id l13mr359961wfe.284.1242799049243; Tue, 19 May 2009 22:57:29 -0700 (PDT) Received: from michelle.cdnetworks.co.kr ([114.111.62.249]) by mx.google.com with ESMTPS id k2sm14362264rvb.10.2009.05.19.22.57.26 (version=SSLv3 cipher=RC4-MD5); Tue, 19 May 2009 22:57:27 -0700 (PDT) Received: by michelle.cdnetworks.co.kr (sSMTP sendmail emulation); Wed, 20 May 2009 15:07:23 +0900 From: Pyun YongHyeon Date: Wed, 20 May 2009 15:07:23 +0900 To: Pascal Braun Message-ID: <20090520060723.GH9043@michelle.cdnetworks.co.kr> References: <4A127C56.6040502@continum.net> <20090519104133.GF4697@michelle.cdnetworks.co.kr> <4A12B0B8.5020507@continum.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="24zk1gE8NUlDmwG9" Content-Disposition: inline In-Reply-To: <4A12B0B8.5020507@continum.net> User-Agent: Mutt/1.4.2.3i Cc: freebsd-current@freebsd.org Subject: Re: Problems with jumbo frames on nfe X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 May 2009 05:57:29 -0000 --24zk1gE8NUlDmwG9 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, May 19, 2009 at 03:14:32PM +0200, Pascal Braun wrote: > Pyun YongHyeon wrote: > >On Tue, May 19, 2009 at 11:31:02AM +0200, Pascal Braun wrote: > >>Hi, > >> > >>I'm currently testing jumbo frames on a Sunfire 4540 running with > >>FreeBSD 8-Current (build on April 8th). While using the nfe driver I'm > >>having some unexpected problems if i try to bring up the interface with > >>MTU sizes greater than 1970 bytes. > >> > >>The error message (in dmesg) is: > >>nfe2: initialization failed: no memory for rx buffers > >> > > > >It means you've run out of jumbo clusters. Check the output of > >"netstat -m" and see how many jumbo cluster requests were denied. > > 771/1149/1920 mbufs in use (current/cache/total) > 770/680/1450/25600 mbuf clusters in use (current/cache/total/max) > 770/510 mbuf+clusters out of packet secondary zone in use (current/cache) > 0/9/9/12800 4k (page size) jumbo clusters in use (current/cache/total/max) > 0/0/0/16384 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) > 1732K/1683K/3416K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0/0/0 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > There are no denied requests. MTU size was about 1800. > But I have to add, that i cant even get the interface up if the mtu size > is above 1970. > > >>Does anyone have any ideas how to get jumbo frames working? > >> > > > >How about increasing 9K jumbo clusters(kern.ipc.nmbjumbo9) with > >sysctl(8)? > > sunfire# sysctl -a | grep jumbo > kern.ipc.nmbjumbo16: 3200 > kern.ipc.nmbjumbo9: 16384 > kern.ipc.nmbjumbop: 12800 > > I increased kern.ipc.nmbjumbo9 to 16384 (was 6400) but that didn't seem > to help. Do you have another idea? > Ok, would you try attached patch? --24zk1gE8NUlDmwG9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="nfe.jumbo.patch" Index: sys/dev/nfe/if_nfe.c =================================================================== --- sys/dev/nfe/if_nfe.c (revision 192411) +++ sys/dev/nfe/if_nfe.c (working copy) @@ -1153,7 +1153,7 @@ /* Create DMA tag for jumbo Rx buffers. */ error = bus_dma_tag_create(sc->nfe_parent_tag, - PAGE_SIZE, 0, /* alignment, boundary */ + 1, 0, /* alignment, boundary */ BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ --24zk1gE8NUlDmwG9--