Date: Wed, 25 Jul 2012 14:14:02 -0700 From: Scott Long <scottl@samsco.org> To: Andriy Gapon <avg@freebsd.org> Cc: freebsd-scsi@freebsd.org, freebsd-hackers@freebsd.org, freebsd-geom@freebsd.org Subject: Re: geom <-> cam disk Message-ID: <23011628-17F1-49A0-A41E-E7A8A8E3EA64@samsco.org> In-Reply-To: <501056C4.3080806@FreeBSD.org> References: <501056C4.3080806@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Once the bio is put into the bioq from da_strategy, the CAM scheduler is called. It may or may not wind up calling dastart right away; if the simq or devq is frozen, or if the devq has been exhausted, then the io will be deferred until later and the call stack will unwind back into g_down. The bioq can therefore accumulate many bio's before being drained. Draining will usually happen from the camisr, at which point you can potentially have i/o being initiated from both the camisr and the g_down threads in parallel. The monolithic locking in CAM right now prevents this from actually happening, though that's a topic that needs to be revisited.
Scott
On Jul 25, 2012, at 1:27 PM, Andriy Gapon wrote:
>
>
> Preamble. I am trying to understand in detail how things work at GEOM <-> "CAM
> disk" boundary. I am looking at scsi_da and ata_da which seem to be twins in
> this respect.
>
> I got an impression that the bioq_disksort calls in the strategy methods and the
> related queues are completely useless in the GEOM single-threaded world.
> There is only one thread, g_down, that can call a strategy method, the method
> enqueues a bio, then calls a schedule function and through xpt_schedule the call
> flow continues to a start method which dequeues the bio and off it goes.
> I currently can see how a bio queue can accumulate more than one bio.
>
> What am I missing? :-)
> I will be very glad to learn more about this layer if anyone is willing to
> educate me.
> Thank you in advance.
>
> P.S. I wrote a very simple to DTrace script to my "theory" experimentally and my
> testing with various workloads didn't disprove the theory so far (which doesn't
> mean that it is correct, of course).
>
> The script:
> fbt::bioq_disksort:entry
> /args[0]->queue.tqh_first == 0/
> {
> @["empty"] = count();
> }
>
> fbt::bioq_disksort:entry
> /args[0]->queue.tqh_first != 0/
> {
> @["non-empty"] = count();
> }
>
> It works on all bioq_disksort calls, but I stressing only ada disks at the moment.
> --
> Andriy Gapon
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?23011628-17F1-49A0-A41E-E7A8A8E3EA64>