From owner-freebsd-stable@FreeBSD.ORG Tue May 29 21:39:14 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 82262106564A for ; Tue, 29 May 2012 21:39:14 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-qa0-f49.google.com (mail-qa0-f49.google.com [209.85.216.49]) by mx1.freebsd.org (Postfix) with ESMTP id 388008FC08 for ; Tue, 29 May 2012 21:39:14 +0000 (UTC) Received: by qabj40 with SMTP id j40so2033757qab.15 for ; Tue, 29 May 2012 14:39:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=L2Ubf4SqCBVAB/YW01VQF4J/Lmrk1UjEUeaGCJD0D+8=; b=vs4ck8r1XFcgkkEU26odz3uTJwX1HxAZBI3t4tuOy1iE6sExgUkF2QyV1KkaZtTkWi uGsxv7qq5coWAI4VOQJsoHik44NWyu3Y9asK1eBd3tK7MZrhKoOJ1wfJa+ZxZj1WaslA v82AB43XIYSOW+SgenLF5PWjUQ7TE+NytNHWToPQf518kWcPcMHni0Q7QDMm0kMkE2dg N2J12l53tm+Kq2Wh7pi+QIXmxL3DDNknEvyoVXUxYaTNX22+fqQI7tq3lrGpCUWvKNLb KYgPSBfRetsJWqbaH2VDvomIp2o771fY2lEfQi8ZTq5XdWUkCNyGcI5CS8phL9/eL3d0 vmcw== MIME-Version: 1.0 Received: by 10.229.105.143 with SMTP id t15mr4254079qco.46.1338327553608; Tue, 29 May 2012 14:39:13 -0700 (PDT) Received: by 10.229.20.148 with HTTP; Tue, 29 May 2012 14:39:13 -0700 (PDT) In-Reply-To: <66F196D1-A6D0-437B-886B-2E1A445A69F2@gmail.com> References: <17320979-2ED2-4DF2-97E9-09035F4DD3BB@gmail.com> <66F196D1-A6D0-437B-886B-2E1A445A69F2@gmail.com> Date: Tue, 29 May 2012 14:39:13 -0700 Message-ID: From: Freddie Cash To: Kees Jan Koster Content-Type: text/plain; charset=UTF-8 Cc: freebsd-stable@freebsd.org Subject: Re: FreeBSD 9.0 hangs on heavy I/O X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2012 21:39:14 -0000 On Tue, May 29, 2012 at 2:30 PM, Kees Jan Koster wrote: > Dear Freddie, > >> Granted, I haven't played with gsched yet (most of our high-I/O >> systems are ZFS), so there may be a way to use it across-GEOMs. > > From my previous experiments ZFS suffers the same fate when there is heavy write activity. Reads just don't get served in time. > > How do you deal with that? We're currently only using FreeBSD (and ZFS) on our backups servers. The two main servers do rsync backups for ~150 remote Linux servers and FreeBSD firewalls (1 server does the elementary and secondary schools; the other server does the admin sites). Then they do zfs sends to a third system off-site. Thus, our workloads tend to be fairly one-sided (all reads on the zfs send side; all writes on the zfs recv side; mostly reads on the rsync side side with some writes). And, most of our working set fits into ARC/L2ARC. Cache devices really help, as most reads come from the L2ARC, while most writes go straight through to the pool. We're still a year or so away from our ultimate goal of using FreeBSD+ZFS+NFS to create a separate/proper SAN/NAS tier for our virtual servers. At that point, we'll look a little deeper into things, and experiment with different L2ARC/ZIL setups to optimise read and write paths. -- Freddie Cash fjwcash@gmail.com