From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 25 22:07:20 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 28E09106564A; Wed, 25 Jul 2012 22:07:20 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-wg0-f50.google.com (mail-wg0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 36DE48FC0C; Wed, 25 Jul 2012 22:07:19 +0000 (UTC) Received: by wgbds11 with SMTP id ds11so1164488wgb.31 for ; Wed, 25 Jul 2012 15:07:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=xCjRclWupUBbHNW28bshcuJldym+nph0RS8wi0HH9UM=; b=ULFogl4bT0/BjFx1VnUjvWWa1w2ihMH21rpgcF331r7/H7Q1PlCqQZ2elhO8yZYauG j9VsRMyoFL8ixWtPDTk/FF8mG8mwSZV2b0C/uiz0d/xXlKdBzoppGZT9ZrI5u5xtuSbB EYeyvurVb/Z9ESA7plDv5aITI8s54fvIdPx64vs+e47a3oQ9sj0bcoroKIY4dN9zs78n uZH2VHKpiSeTxIWk2mYUSR7qu8UC9Vv2wYlCbn9ftOHvNL6dRnrWGWLHDKJUkE08MVtJ ZuyyIVOo1mFyxy6ObaFiiyKC2EQT7EEYe3uQwVhPv7Hk0CdtzMvSw7m1vDkzOPe91x7t 1kKA== Received: by 10.180.78.4 with SMTP id x4mr8035938wiw.19.1343254038100; Wed, 25 Jul 2012 15:07:18 -0700 (PDT) Received: from pc.mavhome.dp.ua (pc.mavhome.dp.ua. [212.86.226.226]) by mx.google.com with ESMTPS id bc2sm7301435wib.0.2012.07.25.15.07.16 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 25 Jul 2012 15:07:17 -0700 (PDT) Sender: Alexander Motin Message-ID: <50106E5F.4030402@FreeBSD.org> Date: Thu, 26 Jul 2012 01:08:31 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120621 Thunderbird/13.0.1 MIME-Version: 1.0 To: Andriy Gapon References: <501056C4.3080806@FreeBSD.org> In-Reply-To: <501056C4.3080806@FreeBSD.org> Content-Type: text/plain; charset=x-viet-vps; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-scsi@FreeBSD.org, freebsd-hackers@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: geom <-> cam disk X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jul 2012 22:07:20 -0000 On 25.07.2012 23:27, Andriy Gapon wrote: > Preamble. I am trying to understand in detail how things work at GEOM <-> "CAM > disk" boundary. I am looking at scsi_da and ata_da which seem to be twins in > this respect. > > I got an impression that the bioq_disksort calls in the strategy methods and the > related queues are completely useless in the GEOM single-threaded world. > There is only one thread, g_down, that can call a strategy method, the method > enqueues a bio, then calls a schedule function and through xpt_schedule the call > flow continues to a start method which dequeues the bio and off it goes. > I currently can see how a bio queue can accumulate more than one bio. > > What am I missing? :-) > I will be very glad to learn more about this layer if anyone is willing to > educate me. > Thank you in advance. > > P.S. I wrote a very simple to DTrace script to my "theory" experimentally and my > testing with various workloads didn't disprove the theory so far (which doesn't > mean that it is correct, of course). > > The script: > fbt::bioq_disksort:entry > /args[0]->queue.tqh_first == 0/ > { > @["empty"] = count(); > } > > fbt::bioq_disksort:entry > /args[0]->queue.tqh_first != 0/ > { > @["non-empty"] = count(); > } > > It works on all bioq_disksort calls, but I stressing only ada disks at the moment. Different controllers have different command queueing limitations. If you are testing with ahci(4) driver and modern disks, then their 32 command slots per port can be enough for many workloads to enqueue all commands to the hardware and leave queue empty as you've described. But if you take harder workload, or controller/ device without command queueing support, extra requests will be accumulated on that bioq and sorted there. -- Alexander Motin