Date: Wed, 26 Oct 2005 18:56:28 +0200 From: Jacques Caron <jc@oxado.com> To: Scott Long <scottl@samsco.org> Cc: freebsd-amd64@FreeBSD.ORG, sos@FreeBSD.ORG Subject: Re: busdma dflt_lock on amd64 > 4 GB Message-ID: <6.2.3.4.0.20051026183714.03b48e20@wheresmymailserver.com> In-Reply-To: <435FAEE1.6000506@samsco.org> References: <6.2.3.4.0.20051025171333.03a15490@pop.interactivemediafactory.net> <6.2.3.4.0.20051026131012.03a80a20@pop.interactivemediafactory.net> <435F8E06.9060507@samsco.org> <6.2.3.4.0.20051026163501.03b7d3e8@wheresmymailserver.com> <435FAEE1.6000506@samsco.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Scott, Thank you for your time on this issue... At 18:29 26/10/2005, Scott Long wrote: >I sent a long email on this on Dec 14, 2004. I'll pull it up and >forward it out. What I really should do is publish a definitive article >on the whole topic. Definitely! And add a link in the bus_dma man page, or expand its contents... >As for 'limitations as to what can happen in the callback', there are >none if you use the correct code structure. Isn't the callback called from an interrupt context of some kind? (if you think I don't know a thing about kernel internals you've guessed right!) Would it be possible, for instance, to move this whole bunch of code: /* start ATAPI operation */ if (ch->hw.command(request->device, ATA_PACKET_CMD, 0, 0, ATA_F_DMA)) { ata_prtdev(request->device, "error issuing ATAPI packet command\n"); request->result = EIO; break; } /* wait for ready to write ATAPI command block */ { int timeout = 5000; /* might be less for fast devices */ while (timeout--) { int reason = ATA_IDX_INB(ch, ATA_IREASON); int status = ATA_IDX_INB(ch, ATA_STATUS); if (((reason & (ATA_I_CMD | ATA_I_IN)) | (status & (ATA_S_DRQ | ATA_S_BUSY))) == ATAPI_P_CMDOUT) break; DELAY(20); } if (timeout <= 0) { ata_prtdev(request->device,"timeout waiting for ATAPI ready\n"); request->result = EIO; break; } } /* this seems to be needed for some (slow) devices */ DELAY(10); /* output actual command block */ ATA_IDX_OUTSW_STRM(ch, ATA_DATA, (int16_t *)request->u.atapi.ccb, (request->device->param->config & ATA_PROTO_MASK) == ATA_PROTO_ATAPI_12 ? 6 : 8); /* start DMA engine */ if (ch->dma->start(ch)) { request->result = EIO; break; } into a callback? It's mainly all the DELAY's that seem out of place in a callback to me... Note for Soren: there is a good explanation at the end of http://freebsd.active-venture.com/arch-handbook/isa-driver-busmem.html of callback integration strategies... >>>No. Some tags specifically should not permit deferals. >> >>How do they do that? Setting BUS_DMA_ALLOCNOW in the tag, or >>BUS_DMA_NOWAIT in the map_load, or both, or something else? > >They set it by using NULL as the lockfunc. Errrr... I'm lost here. I understood that you should use NULL as the lockfunc if and only if you know there won't be any deferrals (and I know by experience now that not having a lockfunc and having a deferral cause a panic!). So having NULL as the lockfunc is a consequence of not having any deferrals, it won't prevent them. >Actually, one map per tag is not common. If the ATA driver supported >tagged queuing (which I assume that it will someday for SATAII, yes?) I'm waiting for it eagerly, I have quite a bunch of hardware all set for it :-) >If a map is being created for every drive in the system, and the result >is that not enough bounce pages are being reserved for all three drives >to operate concurrently, then there might be a bug in busdma. We should >discuss this offline. Maps are created for every busy drive in the system, but I believe they each belong to a separate tag (all tags and maps are created on the fly for each I/O operation, as I understand it). If three drives are (really) busy at the same time, a panic is guaranteed if bounce buffers are needed. I'm trying to figure out since yesterday where the problem is (busdma or ata), but I think it's "somewhere in between", with each part assuming that the other does things some way or another. >>In this case 32 bounce pages (out of 8 GB RAM) for 6 disks seems >>like a very tight bottleneck to me. > >If that's all that is needed to saturate non-tagged ATA, then there is >nothing wrong with that. It might be enough to saturate one or two non-tagged ATA drives, but certainly not 6... Anyway, right now, it's not even saturation I have to deal with, but just a brutal panic :-( Jacques.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6.2.3.4.0.20051026183714.03b48e20>