From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 19:07:58 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DDD4C106566B; Mon, 22 Mar 2010 19:07:57 +0000 (UTC) (envelope-from pisymbol@gmail.com) Received: from mail-pv0-f182.google.com (mail-pv0-f182.google.com [74.125.83.182]) by mx1.freebsd.org (Postfix) with ESMTP id 976828FC1B; Mon, 22 Mar 2010 19:07:57 +0000 (UTC) Received: by pvc7 with SMTP id 7so1725188pvc.13 for ; Mon, 22 Mar 2010 12:07:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=IAbwQDXKWD9NlEl/9OqmLYW9cBIGQ2VBgkLOBd+Tk4Y=; b=nhMIq7dphzai8YsLQY2aGid9Rt8VCoTK/veYPs7rDiu0MWDg1v4aNttBUygAelvpLb aPAaKk85PCqWL2aNBdNbVCy04T8PdvkB9kYXWYu6gpE0Ko3pWRS/WvH++XNuDSc0yVON mcMsvA0/pTu32F9rCm6OCnsdq2y0gA4rPNGJo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=QIyl9hwMBbqWg9rep6rl6Y4BrR6gG3k1vmhvrzJcBUUQB4CEpswh33WNWs5UO+2G/w 9URo3ZMrL5LmXt/IbFQ5ucHGbr/ouVIKs9CqoxaACn4XtlhH39Er7Kz1gqrC8de38dOO ou4ewQmsBFPrIKjg9ayOXpAccKERhk6KS+vmg= MIME-Version: 1.0 Received: by 10.115.103.40 with SMTP id f40mr1014095wam.38.1269284877017; Mon, 22 Mar 2010 12:07:57 -0700 (PDT) In-Reply-To: <20100322.124505.787670930858384500.imp@bsdimp.com> References: <1269134585.00231959.1269122405@10.7.7.3> <4BA6279E.3010201@FreeBSD.org> <20100322.124505.787670930858384500.imp@bsdimp.com> Date: Mon, 22 Mar 2010 15:07:56 -0400 Message-ID: <3c0b01821003221207p4e4eecabqb4f448813bf5a8a8@mail.gmail.com> From: Alexander Sack To: "M. Warner Losh" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org, scottl@samsco.org, freebsd-current@freebsd.org, mav@freebsd.org, ivoras@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 19:07:58 -0000 On Mon, Mar 22, 2010 at 2:45 PM, M. Warner Losh wrote: > In message: > =A0 =A0 =A0 =A0 =A0 =A0Scott Long writes: > : I'd like to go in the opposite direction. =A0The queue-dispatch-queue > : model of GEOM is elegant and easy to extend, but very wasteful for > : the simple case, where the simple case is one or two simple > : partition transforms (mbr, bsdlabel) and/or a simple stripe/mirror > : transform. =A0None of these need a dedicated dispatch context in order > : to operate. =A0What I'd like to explore is compiling the GEOM stack at > : creation time into a linear array of operations that happen without > : a g_down/g_up context switch. =A0As providers and consumers taste each > : other and build a stack, that stack gets compiled into a graph, and > : that graph gets executed directly from the calling context, both > : from the dev_strategy() side on the top and the bio_done() on the > : bottom. =A0GEOM classes that need a detached context can mark > : themselves as such, doing so will prevent a graph from being > : created, and the current dispatch model will be retained. > > I have a few things to say on this. > > First, I've done similar things at past companies for systems that are > similar to geom's queueing environment. =A0It is possible to convert the > queueing nodes in the graph to filtering nodes in the graph. =A0Another > way to look at this is to say you're implementing direct dispatch into > geom's stack. =A0This can be both good and bad, but should reduce > latency a lot. > > One problem that I see is that you are calling into the driver from a > different set of contexts. =A0The queueing stuff was there to protect > the driver from LoRs due to its routines being called from many > different contexts, sometimes with other locks held (fact of life > often in the kernel). > > So this certainly is something worth exploring, especially if we have > optimized paths for up/down for certain geom classes while still > allowing the current robust, but slow, paths for the more complicated > nodes in the tree. =A0It remains to be see if there's going to be issues > around locking order, but we've hit that with both geom and ifnet in > the past, so caution (eg, running with WITNESS turned on early and > often) is advised. Am I going crazy or does this sound a lot like Sun/SVR's stream based network stack? (design and problems, stream stack locking was notoriously tricky for the exact issue mentioned above, different running contexts with different locking granularity/requirements). -aps