From owner-freebsd-current@FreeBSD.ORG Sat Jan 14 15:28:15 2006 Return-Path: X-Original-To: current@freebsd.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E8A9016A41F for ; Sat, 14 Jan 2006 15:28:15 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 65B3243D46 for ; Sat, 14 Jan 2006 15:28:12 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [192.168.254.11] (junior.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.4/8.13.4) with ESMTP id k0EFSACQ047766; Sat, 14 Jan 2006 08:28:10 -0700 (MST) (envelope-from scottl@samsco.org) Message-ID: <43C9188B.3050300@samsco.org> Date: Sat, 14 Jan 2006 08:28:11 -0700 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20051230 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Steve Kargl References: <20060114052117.GA16773@xor.obsecurity.org> <20060114055347.GA45580@troutmask.apl.washington.edu> In-Reply-To: <20060114055347.GA45580@troutmask.apl.washington.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.4 required=3.8 tests=ALL_TRUSTED autolearn=failed version=3.1.0 X-Spam-Checker-Version: SpamAssassin 3.1.0 (2005-09-13) on pooker.samsco.org Cc: current@freebsd.org, Kris Kennaway Subject: Re: Extreme contigmalloc() slowness with mpt driver X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Jan 2006 15:28:16 -0000 Steve Kargl wrote: > On Sat, Jan 14, 2006 at 12:21:17AM -0500, Kris Kennaway wrote: > >>I have an amd64 machine with 16GB of RAM that takes ages to boot (~40 >>minutes on 7.0). This is because the mpt driver takes 20 minutes to >>attach (with 2 instances). This in turn is because the following code >>from dev/mpt/mpt_pci.c:mpt_dma_mem_alloc() takes about 5 seconds to >>execute, and it is run 256 times in a loop: >> >> error = bus_dmamap_create(mpt->buffer_dmat, 0, &req->dmap); >> >>When I set vm.old_contigmalloc=1, the system boots without delay. >> >>This points to a bug in contigmalloc. >> > > > This is probably related to my recent reports of extremely > slow probing of fxp0. I have 12 GB on a Tyan K8S Pro and > fxp0 takes on the order of 7 minutes to probe. > Yep, that's the same reason. THe issue here is that bus_dmamap_create is using contigmalloc to allocate bounce pages for the device. At the request of Soeren, I recently upped the max limit on bounce pages from 512 to 4096. Before that, drivers would quickly reach the max and then move on. Now that the max is a lot higher, I guess it points to a scalability problem in the page search algorithm of contigmalloc. Scott