From owner-freebsd-arch@FreeBSD.ORG Sat Mar 20 19:26:53 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 462CD106566C; Sat, 20 Mar 2010 19:26:53 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f222.google.com (mail-fx0-f222.google.com [209.85.220.222]) by mx1.freebsd.org (Postfix) with ESMTP id 9B9AC8FC0A; Sat, 20 Mar 2010 19:26:52 +0000 (UTC) Received: by fxm22 with SMTP id 22so3930198fxm.14 for ; Sat, 20 Mar 2010 12:26:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=5o8zI4Z2F7aw1yG7OevShLO0BWzWkprMueYsuWfhHao=; b=pN/SPcPgEejIBP+5U2N5ZRH/MZ9vONhwU4R7Iuh1PJszurDVnEqp1fo6++Emokyw2b zamFMKRMLKHUaSlkKtTRj2Ns1Ad2UNXQTXsQoDf+JTb5y2C5wYHz9Nx3a/YFBMjwUci+ xkYv5sRQRRE9QpBsF7Jc1rolk/N/ELLNZwWVM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=UiWzDCsbVt38yvogHTTIARYir2WpEOKGZW83LRduE0rOLGBPjtJH24iCmr6sxR8oye 9CgF1r8zC4Q6LKvVb1mpP1S/+vVGADoFuUmx6YvlBzYaBLz8LVVy+zA09AcO7ZyFdoXR 0gNM7fHkNKBlqjAJqaP13UszNS2kxWMhEMqF0= Received: by 10.223.94.200 with SMTP id a8mr12097540fan.86.1269113211478; Sat, 20 Mar 2010 12:26:51 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id 13sm2177639fxm.14.2010.03.20.12.26.50 (version=SSLv3 cipher=RC4-MD5); Sat, 20 Mar 2010 12:26:51 -0700 (PDT) Sender: Alexander Motin Message-ID: <4BA52179.9030903@FreeBSD.org> Date: Sat, 20 Mar 2010 21:26:49 +0200 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20091212) MIME-Version: 1.0 To: Scott Long References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> In-Reply-To: <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2010 19:26:53 -0000 Scott Long wrote: > On Mar 20, 2010, at 11:53 AM, Matthew Dillon wrote: >> Diminishing returns get hit pretty quickly with larger MAXPHYS values. >> As long as the I/O can be pipelined the reduced transaction rate >> becomes less interesting when the transaction rate is less than a >> certain level. Off the cuff I'd say 2000 tps is a good basis for >> considering whether it is an issue or not. 256K is actually quite >> a reasonable value. Even 128K is reasonable. > > I agree completely. I did quite a bit of testing on this in 2008 and 2009. > I even added some hooks into CAM to support this, and I thought that I had > discussed this extensively with Alexander at the time. Guess it was yet another > wasted conversation with him =-( I'll repeat it here for the record. AFAIR at that time you've agreed that 256K gives improvements, and 64K of DFLTPHYS limiting most SCSI SIMs is too small. That's why you've implemented that hooks in CAM. I have not forgot that conversation (pity that it quietly died for SCSI SIMs). I agree that too high value could be just a waste of resources. As you may see I haven't blindly committed it, but asked public opinion. If you think 256K is OK - let it be 256K. If you think that 256K needed only for media servers - OK, but lets make it usable there. > Besides the nswbuf sizing problem, there is a real problem that a lot of drivers > have incorrectly assumed over the years that MAXPHYS and DFLTPHYS are > particular values, and they've sized their data structures accordingly. Before > these values are changed, an audit needs to be done OF EVERY SINGLE > STORAGE DRIVER. No exceptions. This isn't a case of changing MAXHYS > in the ata driver, testing that your machine boots, and then committing the change > to source control. Some drivers will have non-obvious restrictions based on > the number of SG elements allowed in a particular command format. MPT > comes to mind (its multi message SG code seems to be broken when I tried > testing large MAXPHYS on it), but I bet that there are others. As you should remember, we have made it in such way, that all unchecked drivers keep using DFLTPHYS, which is not going to be changed ever. So there is no problem. I would more worry about non-CAM storages and above stuff, like some rare GEOM classes. > I'm fine with raising MAXPHYS in production once the problems are > addressed. That's why in my post I've asked people about any known problems. I've addressed several related issues in last months, and I am looking for more. To address problems, it would be nice to know about them first. -- Alexander Motin