From owner-freebsd-current@FreeBSD.ORG Thu Jul 19 00:22:26 2007 Return-Path: X-Original-To: current@freebsd.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7077916A402 for ; Thu, 19 Jul 2007 00:22:26 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.182]) by mx1.freebsd.org (Postfix) with ESMTP id 36D0713C478 for ; Thu, 19 Jul 2007 00:22:26 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: by wa-out-1112.google.com with SMTP id j37so455486waf for ; Wed, 18 Jul 2007 17:22:25 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:received:received:date:from:to:cc:subject:message-id:reply-to:references:mime-version:content-type:content-disposition:in-reply-to:user-agent; b=Q9GVhQGEdPH1UUyOeqVpTJCbokpz4Oy2dZvYXViLrZvZa7/Dnf2QzQIbkJdR+lVP/J1jxi1rLEA2EBu6xx9n38W3I//68LMhMVZtOpP25V577HqbOuj5jAx5EVNbY6k4lk9591hWztSNcdp6jJ+go9WQjIXdbOLARRJU6PFHJ9o= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:date:from:to:cc:subject:message-id:reply-to:references:mime-version:content-type:content-disposition:in-reply-to:user-agent; b=paUhq20ASRqR8U7gzYUJ8iSUthmxf3CN0fTy7HIXriAue5XmUEkcxwO2E5gk0SHFRTJO3zi3zPjsf+pjgxI7MCb2PY2tHvBzfdosSGn+KCPGQh9mjoWLCj+c7tXt6n26bkB3WOEL7FnkaJDaDbM8jb6AvBOvFGNvPZ6D4e4SB+U= Received: by 10.114.177.1 with SMTP id z1mr1122599wae.1184804545872; Wed, 18 Jul 2007 17:22:25 -0700 (PDT) Received: from michelle.cdnetworks.co.kr ( [211.53.35.84]) by mx.google.com with ESMTPS id l36sm2145765waf.2007.07.18.17.22.23 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 18 Jul 2007 17:22:24 -0700 (PDT) Received: from michelle.cdnetworks.co.kr (localhost.cdnetworks.co.kr [127.0.0.1]) by michelle.cdnetworks.co.kr (8.13.5/8.13.5) with ESMTP id l6J0MJWm042601 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 19 Jul 2007 09:22:19 +0900 (KST) (envelope-from pyunyh@gmail.com) Received: (from yongari@localhost) by michelle.cdnetworks.co.kr (8.13.5/8.13.5/Submit) id l6J0MIr8042600; Thu, 19 Jul 2007 09:22:18 +0900 (KST) (envelope-from pyunyh@gmail.com) Date: Thu, 19 Jul 2007 09:22:18 +0900 From: Pyun YongHyeon To: David Christensen Message-ID: <20070719002218.GA42405@cdnetworks.co.kr> References: <09BFF2FA5EAB4A45B6655E151BBDD9030483F161@NT-IRVA-0750.brcm.ad.broadcom.com> <20070718021839.GA37935@cdnetworks.co.kr> <09BFF2FA5EAB4A45B6655E151BBDD9030483F437@NT-IRVA-0750.brcm.ad.broadcom.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <09BFF2FA5EAB4A45B6655E151BBDD9030483F437@NT-IRVA-0750.brcm.ad.broadcom.com> User-Agent: Mutt/1.4.2.1i Cc: current@freebsd.org Subject: Re: Getting/Forcing Greater than 4KB Buffer Allocations X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2007 00:22:26 -0000 On Wed, Jul 18, 2007 at 11:50:02AM -0700, David Christensen wrote: > > On Tue, Jul 17, 2007 at 04:54:31PM -0700, David Christensen wrote: > > > I'm investigating a problem with my bce driver which > > occurs when I ask > > > for a jumbo > > > mbuf cluster (through m_cljget()). When I map the memory for DMA I > > > normally > > > get 3 memory segments (4KB + 4KB + 1KB) on my system, but > > on another > > > user's > > > system he's seeing 2 memory segments (8KB + 1KB). Is there a > > > configuration > > > option that allows this or some other tuning variable > > involved? The > > > system is a > > > Xeon dual-core processor and has 8GB of RAM, running an > > AMD64 version of > > > the kernel. > > > > > > > I've briefly looked over bus_dma usage on bce(4). It seems that you > > told bus_dma the the dma map could be made up of BCE_MAX_SEGMENTS > > segments, where a dma segment could be MJUM9BYTES bytes. If you want > > just two segments you may have to use 2 instead of BCE_MAX_SEGMENTS. > > If the hardware can support up to BCE_MAX_SEGMENTS dma segments on Rx > > descriptors you should be prepared to handle that number of dma > > segments too(e.g. You don't know how may dma segments would be > > returned by bus_dma, you just know the upper bound as you specified > > in bus_dma_tag_create()). > > If the hardware can handle just up to 4KB for a dma segment you > > should tell bus_dma the restriction of the dma segment. > > If you have to get a single dma segment that covers MJUM9BYTES bytes > > due to the limitation of the hardware you may have to use local > > allocator. > > > > Thanks Pyun but I'm really just looking for a way to test that I can > handle the number of segments I've advertised that I can support. I > believe my code is correct but when all I see are allocations of 3 > segments I just can't prove it. I was hoping that running a utility > such as "stress" would help fragment memory and force more variable > responses but that hasn't happened yet. > It seems you've used the following code to create jumbo dma tag. /* * Create a DMA tag for RX mbufs. */ if (bus_dma_tag_create(sc->parent_tag, 1, BCE_DMA_BOUNDARY, sc->max_bus_addr, BUS_SPACE_MAXADDR, NULL, NULL, MJUM9BYTES, BCE_MAX_SEGMENTS, MJUM9BYTES, ^^^^^^^^^^ 0, NULL, NULL, &sc->rx_mbuf_tag)) { BCE_PRINTF("%s(%d): Could not allocate RX mbuf DMA tag!\n", __FILE__, __LINE__); rc = ENOMEM; goto bce_dma_alloc_exit; } If you want to have > 9 dma segements change maxsegsz(MJUM9BYTES) to 1024. bus_dma honors maxsegsz argument so you wouldn't get a dma segments larger than maxsegsz. With MJUM9BYTES maxsegsz you would get up to 4 dma segments on systems with 4K PAGE_SIZE.(You would have got up to 3 dma segements if you used PAGE_SIZE alignment argument.) > Dave > > -- Regards, Pyun YongHyeon