Date: Wed, 26 Oct 2005 14:00:12 +0200 From: =?ISO-8859-1?Q?S=F8ren_Schmidt?= <sos@FreeBSD.ORG> To: Jacques Caron <jc@oxado.com> Cc: scottl@FreeBSD.ORG, freebsd-amd64@FreeBSD.ORG Subject: Re: busdma dflt_lock on amd64 > 4 GB Message-ID: <E1A9BF00-252D-4E62-9A73-029612581FD3@FreeBSD.ORG> In-Reply-To: <6.2.3.4.0.20051026131012.03a80a20@pop.interactivemediafactory.net> References: <6.2.3.4.0.20051025171333.03a15490@pop.interactivemediafactory.net> <6.2.3.4.0.20051026131012.03a80a20@pop.interactivemediafactory.net>
index | next in thread | previous in thread | raw e-mail
On 26/10/2005, at 13:33, Jacques Caron wrote:
> Hi all,
>
> Continuing on this story... [I took the liberty of CC'ing Scott and
> Soren], pr is amd64/87977 though it finally isn't amd64-specific
> but >4GB-specific.
>
> There is really a big problem somewhere between ata and bus_dma for
> boxes with more than 4 GB RAM and more than 2 ata disks:
> * bounce buffers will be needed
> * ata will have bus_dma allocate bounce buffers:
> hw.busdma.zone1.total_bpages: 32
> hw.busdma.zone1.free_bpages: 32
> hw.busdma.zone1.reserved_bpages: 0
> hw.busdma.zone1.active_bpages: 0
> hw.busdma.zone1.total_bounced: 27718
> hw.busdma.zone1.total_deferred: 0
> hw.busdma.zone1.lowaddr: 0xffffffff
> hw.busdma.zone1.alignment: 2
> hw.busdma.zone1.boundary: 65536
>
> * if I do a dd with a bs=256000, 16 bounce pages will be used (most
> of the time). As long as I stay on the same disk, no more pages
> will be used.
> * as soon as I access another disk (e.g. with another dd with the
> same bs=256000), another set of 16 pages will be used (bus_dma tags
> and maps are allocated on a per-channel basis), and all 32 bounce
> pages will be used (most of the time)
> * and if I try to access a third disk, more bounce pages are needed
> and:
> - one of ata_dmaalloc calls to bus_dma_tag_create has ALLOCNOW set
> - busdma_machdep will not allocate more bounce pages in that case
> (the limit is imposed by maxsize in that situation, which has
> already been reached)
> - ata_dmaalloc will fail
> - but some other bus_dma_tag_create call without ALLOCNOW set will
> still cause bounce pages to be allocated, but deferred, and the non-
> existent lockfunc to be called, and panic.
>
> Adding the standard lockfunc will (probably) solve the panic issue,
> but there will still be a problem with DMA in ata.
>
> The same problems most probably exist with many other drivers.
>
> I think we thus have two issues:
> - providing a lockfunc in nearly all bus_dma_tag_create calls (or
> have a better default than a panic)
> - allocating more bounce pages when needed in the ALLOCNOW case
> (with a logic similar to that used to allocate bounce pages in the
> non-ALLOCNOW case)
>
> Thoughts?
The below patch makes ATA always use the ALLOCNOW flag which actually
was intended.
How to patch busdma I'll let scottl decide...
Index: ata-dma.c
===================================================================
RCS file: /home/ncvs/src/sys/dev/ata/ata-dma.c,v
retrieving revision 1.138
diff -u -r1.138 ata-dma.c
--- ata-dma.c 6 Oct 2005 15:44:07 -0000 1.138
+++ ata-dma.c 26 Oct 2005 11:56:15 -0000
@@ -102,13 +102,13 @@
BUS_SPACE_MAXADDR_32BIT, BUS_SPACE_MAXADDR,
NULL, NULL, ch->dma->max_iosize,
ATA_DMA_ENTRIES, ch->dma->segsize,
- 0, NULL, NULL, &ch->dma->dmatag))
+ BUS_DMA_ALLOCNOW, NULL, NULL, &ch->dma-
>dmatag))
goto error;
if (bus_dma_tag_create(ch->dma->dmatag, PAGE_SIZE, PAGE_SIZE,
BUS_SPACE_MAXADDR_32BIT, BUS_SPACE_MAXADDR,
NULL, NULL, MAXTABSZ, 1, MAXTABSZ,
- 0, NULL, NULL, &ch->dma->sg_tag))
+ BUS_DMA_ALLOCNOW, NULL, NULL, &ch->dma-
>sg_tag))
goto error;
if (bus_dma_tag_create(ch->dma->dmatag,ch->dma->alignment,ch-
>dma->boundary,
@@ -135,7 +135,7 @@
if (bus_dma_tag_create(ch->dma->dmatag, PAGE_SIZE, 64 * 1024,
BUS_SPACE_MAXADDR_32BIT, BUS_SPACE_MAXADDR,
NULL, NULL, MAXWSPCSZ, 1, MAXWSPCSZ,
- 0, NULL, NULL, &ch->dma->work_tag))
+ BUS_DMA_ALLOCNOW, NULL, NULL, &ch->dma-
>work_tag))
goto error;
if (bus_dmamem_alloc(ch->dma->work_tag, (void **)&ch->dma-
>work, 0,
Søren Schmidt
sos@FreeBSD.org
home |
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E1A9BF00-252D-4E62-9A73-029612581FD3>