From owner-freebsd-stable@FreeBSD.ORG Tue May 29 21:24:28 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 86CBB106564A for ; Tue, 29 May 2012 21:24:28 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-qa0-f47.google.com (mail-qa0-f47.google.com [209.85.216.47]) by mx1.freebsd.org (Postfix) with ESMTP id 3E1D58FC08 for ; Tue, 29 May 2012 21:24:28 +0000 (UTC) Received: by qabg1 with SMTP id g1so1790810qab.13 for ; Tue, 29 May 2012 14:24:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=wDjqDD5S6T7z/ZopcS6KOX1BkvvLLrF8Zk4BWANogek=; b=P8KKRfFwp+804nb/U5yttwRLy840OChyWkkFU0b1wocqTFxXQoEma8d7uMzdoOv09x X7Hq4P2Wj1JR9dcrjRDAtjeZpD47FT9MqpEVBDLAGRmAQaZq6zYoh6jXHcg8ivP29Hks bwH1Hr73KWHGwCnleuZr+MMF7HK/3QfYeu7cga73vPT34rXG9RWAVvaMwA8TeqeeBiOU V7Tn3QmJon0mI7txAw23Au1+ZSiqh1zcmF9vT8WeTRv5neUNCU45MU3snYNoZHHoVel4 hSjOlWWtx1xXJgzurjbNphC3dFSXzlx7q3ZbqiN6JHHVHK8YRaTPAb/DyLt7l8f2rHC8 g/Gg== MIME-Version: 1.0 Received: by 10.224.189.19 with SMTP id dc19mr13361712qab.76.1338326667482; Tue, 29 May 2012 14:24:27 -0700 (PDT) Received: by 10.229.20.148 with HTTP; Tue, 29 May 2012 14:24:27 -0700 (PDT) In-Reply-To: <17320979-2ED2-4DF2-97E9-09035F4DD3BB@gmail.com> References: <17320979-2ED2-4DF2-97E9-09035F4DD3BB@gmail.com> Date: Tue, 29 May 2012 14:24:27 -0700 Message-ID: From: Freddie Cash To: Kees Jan Koster Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: FreeBSD 9.0 hangs on heavy I/O X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2012 21:24:28 -0000 On Tue, May 29, 2012 at 2:12 PM, Kees Jan Koster wrote= : >>> You may want to play around with gshed, the GEOM Scheduler. >>> >>> Matt Dillon did a bunch of tests comparing FreeBSD+UFS to >>> DragonflyBSD+HAMMER and found that FreeBSD starves read threads in >>> order to satisfy write threads (or the other way around?). =C2=A0But, >>> adding gsched into the mix helped things immensely, allowing mixed >>> reads/writes to better shares disk I/O resources. >>> >>> I'll see if I can dig up a link to his testing e-mail messages. >> >> Here's the post, part of a thread on benchmarking RAID controllers: >> >> http://leaf.dragonflybsd.org/mailarchive/kernel/2011-07/msg00034.html > > I looked at "sysctl kern.geom.confdot" (another ridiculously useful featu= re) to see where the scheduler should be placed. > > The way I was thinking, I should place a scheduler in such a way that wri= tes to one physical device (ada3 in my case) do not cause reads on another = device to stall (e.g. ada2, where the database lives). However, it looks li= ke the GEOM tree is actually a GEOM bush, with a separate tree for each dev= ice. > > Am I missing something? Is there a way to schedule across devices? Is the= bush a tree after all, maybe? There are others much better versed in the ways of GEOM than I, and hopefully they will jump in to simplify/clarify things. :) The way I understand things is that GEOM is a per-device stack of GEOM classes, with the physical device at the bottom, and the VM/block/I/O (?) system at the top. Thus, unless you use one of the multi-device GEOM classes (graid, gmirror, gstripe, gvinum), then each stack is independent of the others. Meaning gsched only works for a single stack (ie, a single device). Granted, I haven't played with gsched yet (most of our high-I/O systems are ZFS), so there may be a way to use it across-GEOMs. --=20 Freddie Cash fjwcash@gmail.com