Date: Thu, 26 Jul 2012 01:29:54 +0300 From: Andriy Gapon <avg@FreeBSD.org> To: Scott Long <scottl@samsco.org> Cc: freebsd-scsi@FreeBSD.org, freebsd-hackers@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: geom <-> cam disk Message-ID: <50107362.7050709@FreeBSD.org> In-Reply-To: <23011628-17F1-49A0-A41E-E7A8A8E3EA64@samsco.org> References: <501056C4.3080806@FreeBSD.org> <23011628-17F1-49A0-A41E-E7A8A8E3EA64@samsco.org>
next in thread | previous in thread | raw e-mail | index | archive | help
on 26/07/2012 00:14 Scott Long said the following:
> Once the bio is put into the bioq from da_strategy, the CAM scheduler is
> called. It may or may not wind up calling dastart right away; if the simq or
> devq is frozen, or if the devq has been exhausted, then the io will be
> deferred until later and the call stack will unwind back into g_down. The
> bioq can therefore accumulate many bio's before being drained. Draining will
> usually happen from the camisr, at which point you can potentially have i/o
> being initiated from both the camisr and the g_down threads in parallel. The
Uh-hah. Thank you for the answer. I didn't think of the case of
frozen/exhausted queues and also didn't hit in my tests.
Now I am starting to understand the logic in xpt_run_dev_allocq.
BTW, I think that it would be nice if the GEOM work-processing could re-use the
CAM model.
That is, try to execute GEOM bio transformations in the original thread as much
as possible, defer work to the GEOM thread as the last resort.
> monolithic locking in CAM right now prevents this from actually happening,
> though that's a topic that needs to be revisited.
> On Jul 25, 2012, at 1:27 PM, Andriy Gapon wrote:
>
>>
>>
>> Preamble. I am trying to understand in detail how things work at GEOM <->
>> "CAM disk" boundary. I am looking at scsi_da and ata_da which seem to be
>> twins in this respect.
>>
>> I got an impression that the bioq_disksort calls in the strategy methods
>> and the related queues are completely useless in the GEOM single-threaded
>> world. There is only one thread, g_down, that can call a strategy method,
>> the method enqueues a bio, then calls a schedule function and through
>> xpt_schedule the call flow continues to a start method which dequeues the
>> bio and off it goes. I currently can see how a bio queue can accumulate
>> more than one bio.
>>
>> What am I missing? :-) I will be very glad to learn more about this layer
>> if anyone is willing to educate me. Thank you in advance.
>>
>> P.S. I wrote a very simple to DTrace script to my "theory" experimentally
>> and my testing with various workloads didn't disprove the theory so far
>> (which doesn't mean that it is correct, of course).
>>
>> The script: fbt::bioq_disksort:entry /args[0]->queue.tqh_first == 0/ {
>> @["empty"] = count(); }
>>
>> fbt::bioq_disksort:entry /args[0]->queue.tqh_first != 0/ { @["non-empty"] =
>> count(); }
>>
>> It works on all bioq_disksort calls, but I stressing only ada disks at the
>> moment. -- Andriy Gapon
>>
>
--
Andriy Gapon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?50107362.7050709>