From owner-freebsd-current@FreeBSD.ORG  Tue Dec  8 01:05:11 2009
Return-Path: <owner-freebsd-current@FreeBSD.ORG>
Delivered-To: freebsd-current@FreeBSD.org
Received: from [127.0.0.1] (freefall.freebsd.org [IPv6:2001:4f8:fff6::28])
	by hub.freebsd.org (Postfix) with ESMTP id BF626106566C;
	Tue,  8 Dec 2009 01:05:10 +0000 (UTC)
	(envelope-from jkim@FreeBSD.org)
From: Jung-uk Kim <jkim@FreeBSD.org>
To: Scott Long <scottl@samsco.org>
Date: Mon, 7 Dec 2009 20:05:00 -0500
User-Agent: KMail/1.6.2
References: <3c0b01820912071342u1c722b2clf9c8413e40097279@mail.gmail.com>
	<200912071931.46002.jkim@FreeBSD.org>
	<D7DDDA30-44B2-4E84-9F52-42DD2C43DC62@samsco.org>
In-Reply-To: <D7DDDA30-44B2-4E84-9F52-42DD2C43DC62@samsco.org>
MIME-Version: 1.0
Content-Disposition: inline
Content-Type: text/plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Message-Id: <200912072005.02662.jkim@FreeBSD.org>
Cc: Alexander Sack <pisymbol@gmail.com>, scottl@FreeBSD.org,
	freebsd-current@FreeBSD.org, emaste@FreeBSD.org
Subject: Re: aac(4) resource FIB starvation on BUS scan revisited
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
	<freebsd-current.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
	<mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>,
	<mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 08 Dec 2009 01:05:11 -0000

On Monday 07 December 2009 07:47 pm, Scott Long wrote:
> On Dec 7, 2009, at 5:31 PM, Jung-uk Kim wrote:
> > On Monday 07 December 2009 05:30 pm, Alexander Sack wrote:
> >> On Mon, Dec 7, 2009 at 4:42 PM, Alexander Sack
> >> <pisymbol@gmail.com>
> >
> > wrote:
> >>> Folks:
> >>>
> >>> I posted a similar thread on freebsd-scsi only to realize that
> >>> scottl had fixed my first issue during some MP CAM cleanup with
> >>> respect to a race during resource allocation issues on a later
> >>> version of the driver we are using (I believe we did the same
> >>> thing to resolve a lock issue on bootup).
> >>>
> >>> However on my RELENG_8 box with (2) Adaptec 5085s connected to
> >>> some JBODs (9TB each) I still have a FIB starvation issue
> >>> during the LUN scan:
> >>>
> >>> The number of FIBs allocated to this card is 512 (older cards
> >>> are 256).  The max_target per bus is 287.  On a six channel
> >>> controller with a BUS scan done in parallel I see a lot of
> >>> this:
> >>>
> >>> ...
> >>> (probe501:aacp1:0:214:0): Request Requeued
> >>> (probe501:aacp1:0:214:0): Retrying Command
> >>> (probe520:aacp1:0:233:0): Request Requeued
> >>> (probe520:aacp1:0:233:0): Retrying Command
> >>> (probe528:aacp1:0:241:0): Request Requeued
> >>> (probe528:aacp1:0:241:0): Retrying Command
> >>> (probe540:aacp1:0:253:0): Request Requeued
> >>> (probe540:aacp1:0:253:0): Retrying Command
> >>> (probe541:aacp1:0:254:0): Request Requeued
> >>> (probe541:aacp1:0:254:0): Retrying Command
> >>> ....
> >>>
> >>> I think the driver is much happier with the following attached
> >>> patch (with dmesg).
> >>
> >> Patch again but this time not base-64 encoded:
> >
> > [SNIP!]
> >
> > I want it to be little conservative here, i.e., pre-allocating
> > half of max_fibs.  Will the attached patch work for you?
>
> The FIB allocation scheme was written when it was common for
> machines to only have 64MB of RAM and proportionally less KVA, so
> 256KB or 512KB was a lot of RAM to wire down.  Those days have
> probably passed.

So, what would do if you were hypothetically rewriting it today? :-)

Jung-uk Kim