From owner-svn-src-all@FreeBSD.ORG Wed Jan 6 22:31:20 2010 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7FEC81065676; Wed, 6 Jan 2010 22:31:20 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bw0-f213.google.com (mail-bw0-f213.google.com [209.85.218.213]) by mx1.freebsd.org (Postfix) with ESMTP id 80B948FC13; Wed, 6 Jan 2010 22:31:19 +0000 (UTC) Received: by bwz5 with SMTP id 5so11322380bwz.3 for ; Wed, 06 Jan 2010 14:31:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=GGgRMC5o7xd9M0yC7ks/1n9JbSTpmIR/cs7HKiOQ194=; b=L6u1+YP2ZvutzQYsTkMw3HREJHuVFdADx/cqcNzfFjGA06KbX735EcZwyxfSTxoz6a DLIQXfd9UHpzeGpx+V95NnLz9UwfCGcm79H0xryYv+1j9//J7OR0CLQfqDtdc/yoOD/K Eq97zvWosCScHFgQwrzQrhwEFPnh4VmxKeZ5o= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=t0zgP4GZq4lYENG5gpcY5kP3vqKn5cj42iNKaKUjEtCl9xgpVU/ZQgR31Sf4paxRjy SxUgXWGsHKGYxkQrnMGFinhOqgoMbRlAt/0OYRJtkbc1sSf5gtQUSwuwNvyFDqFg+OGJ xw76ku/PaNRaDXdDdlc96C76D9QGoi7BNQG68= Received: by 10.204.153.202 with SMTP id l10mr1942566bkw.92.1262817073809; Wed, 06 Jan 2010 14:31:13 -0800 (PST) Received: from mavbook.mavhome.dp.ua ([72.14.240.162]) by mx.google.com with ESMTPS id 13sm6853096bwz.10.2010.01.06.14.31.12 (version=SSLv3 cipher=RC4-MD5); Wed, 06 Jan 2010 14:31:13 -0800 (PST) Sender: Alexander Motin Message-ID: <4B450F30.20705@FreeBSD.org> Date: Thu, 07 Jan 2010 00:31:12 +0200 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20091212) MIME-Version: 1.0 To: Ivan Voras References: <201001061712.o06HCICF087127@svn.freebsd.org> <9bbcef731001060938k2b0014a2m15eef911b9922b2c@mail.gmail.com> <4B44D8FA.2000608@FreeBSD.org> <9bbcef731001061103u33fd289q727179454b21ce18@mail.gmail.com> In-Reply-To: <9bbcef731001061103u33fd289q727179454b21ce18@mail.gmail.com> X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org Subject: Re: svn commit: r201658 - head/sbin/geom/class/stripe X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2010 22:31:20 -0000 Ivan Voras wrote: > 2010/1/6 Alexander Motin : >> Ivan Voras wrote: > >>> I think there was one more reason - though I'm not sure if it is still >>> valid because of your current and future work - the MAXPHYS >>> limitation. If MAXPHYS is 128k, with 64k stripes data was only to be >>> read from maximum of 2 drives. With 4k stripes it would have been read >>> from 128/4=32 drives, though I agree 4k is too low in any case >>> nowadays. I usually choose 16k or 32k for my setups. >> While you are right about MAXPHYS influence, and I hope we can rise it >> not so far, IMHO it is file system business to manage deep enough >> read-ahead/write-back to make all drives busy, independently from >> MAXPHYS value. With small MAXPHYS value FS should just generate more >> requests in advance. Except some RAID3/5/6 cases, where short writes >> ineffective, MAXPHYS value should only affect processing overhead. > > Yes, my experience which lead to the post was mostly on UFS which, > while AFAIK it does read-ahead, it still does it serially (I think > this is implied by your experiments with NCQ and ZFS vs UFS) - so in > any case only 2 drives are hit with 64k stripe size at any moment in > time. I do not think it is true. On system with default MAXPHYS I've made gstripe with 64K block of 4 equal drives with 108MB/s of maximal read speed. Reads with dd from large pre-written file on UFS shown: vfs.read_max=8 (default) - 235090074 bytes/sec vfs.read_max=16 - 378385148 bytes/sec vfs.read_max=32 - 386620109 bytes/sec I've put some printfs into the clustering read code and found enough read-ahead there. So it works. One thing IMHO would be nice to see there is the alignment of the read-ahead requests to the array stripe size/offset. Dirty hack I've tried there, reduced number of requests to the array components by 30%. -- Alexander Motin