From owner-freebsd-amd64@FreeBSD.ORG Thu Oct 27 14:03:40 2005 Return-Path: X-Original-To: freebsd-amd64@FreeBSD.ORG Delivered-To: freebsd-amd64@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 69DBA16A41F; Thu, 27 Oct 2005 14:03:40 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id DEAE143D48; Thu, 27 Oct 2005 14:03:39 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [192.168.254.204] ([192.168.254.204]) (authenticated bits=0) by pooker.samsco.org (8.13.4/8.13.4) with ESMTP id j9RE3cJs073160; Thu, 27 Oct 2005 08:03:38 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <4360DE3C.6010209@samsco.org> Date: Thu, 27 Oct 2005 08:03:40 -0600 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.8) Gecko/20050615 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jacques Caron References: <6.2.3.4.0.20051025171333.03a15490@pop.interactivemediafactory.net> <6.2.3.4.0.20051026131012.03a80a20@pop.interactivemediafactory.net> <435F8E06.9060507@samsco.org> <08A81034-AB5D-4BFC-8F53-21501073D674@FreeBSD.ORG> <435FA542.3030209@samsco.org> <6.2.3.4.0.20051026180325.03ad7558@wheresmymailserver.com> <42A1B51D-5A5E-4849-96D0-BC5C2DD1AE97@FreeBSD.ORG> <6.2.3.4.0.20051027121824.03d34dd8@wheresmymailserver.com> In-Reply-To: <6.2.3.4.0.20051027121824.03d34dd8@wheresmymailserver.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.4 required=3.8 tests=ALL_TRUSTED autolearn=failed version=3.1.0 X-Spam-Checker-Version: SpamAssassin 3.1.0 (2005-09-13) on pooker.samsco.org Cc: freebsd-amd64@FreeBSD.ORG, SXren Schmidt Subject: Re: busdma dflt_lock on amd64 > 4 GB X-BeenThere: freebsd-amd64@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the AMD64 platform List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Oct 2005 14:03:40 -0000 Jacques Caron wrote: > Hi, > > At 20:01 26/10/2005, Søren Schmidt wrote: > ATA does all the tag/map creates/allocs/loads for the SGlist and > >> simple workspace stuff at channel attach time, there are NO further >> creates or allocs after that. So that is exactly what I would expect >> the ALLOCNOW flags to make busdma support, if that doesn't work >> busdma needs to be fixed IMNHO. > > > So I tested with ALLOCNOW set on all 4 tags, and the end result is > exactly the same, ch->dma->load fails (due to EINPROGRESS, most > probably), and a panic is trigerred by busdma dflt_lock . And I agree > that this is a busdma bug, as the busdma manpage says, in the > description of bus_dmamap_load: > > EINPROGRESS The mapping has been deferred for lack of > resources. The callback will be called as > soon as > resources are available. Callbacks are > serviced in > FIFO order. DMA maps created from DMA tags that > are allocated with the BUS_DMA_ALLOCNOW flag > will > never return this status for a load operation. > > So either the busdma API needs to be changed and ALLOCNOW no longer > means that EINPROGRESS cannot happen (and that might break other > things), or busdma needs to be fixed to conform to the documentation. > > Interestingly, if I remove the ALLOCNOW in all tag creations in ata, then: > - ata attaches during boot are a bit longer (like fxp which is really long) > - 320 bounces pages are allocated > - I can do apparently do everything I want and not panic the machine. > - I suppose that since there's plenty of available bounce pages, I > should not have problems with ENOMEM. > > I'll go with that configuration for now, but that's most probably not > the correct fix, just a workaround. The correct fix is to make busdma > work as documented. > > Here I see several things to look into: > - make sure enough ressources are actually allocated at tag create time > if ALLOCNOW is set (this does not necessarily mean allocating maxsize > additional pages, some logic about already required pages needs to be > added I think) > - let bus_dmamap_create allocate additional bounce pages if needed even > if ALLOCNOW was set initially (i.e. don't use BUS_DMA_MIN_ALLOC_COMP), > though I'm not sure the page allocation logic there is entirely correct > either, as it will allocate additional pages even if there are already > enough > - in _bus_dmamap_load_buffer, consider ALLOCNOW like the undocumented > NOWAIT, i.e. return ENOMEM if enough ressources aren't available (which > I believe wouldn't be correct either: ALLOCNOW should prevent > EINPROGRESS but also ENOMEM at map load time and should return ENOMEM > immediately if there aren't enough ressources, not later). > > Another thing to look into is why bounce page allocation is that long > (at least I believe that's what is taking so long), but I haven't looked > at that bit at all yet. > > I also wonder if allocating bounce pages on <=4G configurations is > really needed/useful? There are probably situations where this is > needed, but I'm not sure I understand why. > > Jacques. > > It sounds like the only real bug in busdma is that it gets confused by ALLOCNOW and doesn't allow enough pages to be allocated when bus_dmamap_create() is called. I'll look into this. Scott