From owner-svn-src-head@freebsd.org Thu Apr 14 22:04:28 2016 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D1D5ADA037 for ; Thu, 14 Apr 2016 22:04:28 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22d.google.com (mail-io0-x22d.google.com [IPv6:2607:f8b0:4001:c06::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C24116AA for ; Thu, 14 Apr 2016 22:04:28 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22d.google.com with SMTP id o126so118098971iod.0 for ; Thu, 14 Apr 2016 15:04:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc; bh=zS1NI1kb1gK726rIRewBbpkGLLMfwC3GP5p0lTKIVJM=; b=v6n7AZqig9kmee8RGp01E586Fa6HF0sgTudh3GgIdqIHjU0IjSQt7dVjya62xajej8 IVqA+D8AckvbX0lZLdT7nQito3Kc9P6gZ4sSqQZZ3ES/YnGSUAILyf9vgRB5iMJup2f/ kuk1FZeLm16LJvxNUJ+xIwMO7xbew2NDb5tGTpPWFkYjP6bSzLM2tBQeeK+hFRNkMsit gCYL2n5O0mOpXGGwFpfrtXFLVOihPAvwSNaRm0VJ8n9YKB1QOx35cuzQ+Jw1Y++eAnGF ietNVZq+R4Lxvvbx6jJOqJcMMHMyglOugkwRiWnKZyjXN9PFrSj29BOVBfP6fOC9FJka d09w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc; bh=zS1NI1kb1gK726rIRewBbpkGLLMfwC3GP5p0lTKIVJM=; b=ELqQ06kCdn88kEjfVvx83OEHUT+dKbPk7k72CkhJFpLVZuff/qGAuzbXYUBOAkKXEt dxUPFuCiiO1hJxx3m9LDK+pWp3CYANwkkHbzTyHFDXEGQ+IgpXV+M/690k1sKzUhZOMk CURzPekhAktawL5IMFbLOKEo95hIfF0cS9DaeC5k9358OBPqqBQ+w1hgwp1Mrpg6uHLH 0tgfQeT+Mdob0ERS76/LWxEf2PcGJonj+rj3f02PYFebunPEjYYiFVxUHbgJHIJIkyQE dCcUZ98rAPWaBA29mTZ6XrXfgyMfKS/ROy59piYgv2Y3JoXATUz1GBSbbxXxaC0dwYsq IBRA== X-Gm-Message-State: AOPr4FUJZW9YdIK/Q+1eZR/n+pkUXL2Ig9dIYoQDIMOqkaGLv7SPrO9N4OUn+7N/4ZXGkdY8ldShqL7UdXhJ4w== MIME-Version: 1.0 X-Received: by 10.107.14.209 with SMTP id 200mr18397734ioo.73.1460671467638; Thu, 14 Apr 2016 15:04:27 -0700 (PDT) Sender: wlosh@bsdimp.com Received: by 10.36.194.3 with HTTP; Thu, 14 Apr 2016 15:04:27 -0700 (PDT) X-Originating-IP: [50.253.99.174] In-Reply-To: References: <201604142147.u3ELlwYo052010@repo.freebsd.org> Date: Thu, 14 Apr 2016 16:04:27 -0600 X-Google-Sender-Auth: ejUv0SPYSK3QSNkJJVLLzHqn1EI Message-ID: Subject: Re: svn commit: r298002 - in head/sys: cam cam/ata cam/scsi conf dev/ahci From: Warner Losh To: Dmitry Morozovsky Cc: Warner Losh , src-committers , "svn-src-all@freebsd.org" , "svn-src-head@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Apr 2016 22:04:28 -0000 On Thu, Apr 14, 2016 at 3:54 PM, Dmitry Morozovsky wrote: > Warner, > > On Thu, 14 Apr 2016, Warner Losh wrote: > > > Author: imp > > Date: Thu Apr 14 21:47:58 2016 > > New Revision: 298002 > > URL: https://svnweb.freebsd.org/changeset/base/298002 > > > > Log: > > New CAM I/O scheduler for FreeBSD. The default I/O scheduler is the > same > > [snip] > > First, thanks so much for this quite a non-trivial work! > What are the ways to enable this instead of deafult, and what ar the > benefits > and drawbacks? You add CAM_NETFLIX_IOSCHED to your kernel config to enable it. Hmmm, looking at the diff, perhaps I should add that to LINT. In production, we use it for three things. First, our scheduler keeps a lot more statistics than the default one. These statistics are useful for us knowing when a system is saturated and needs to shed load. Second, we favor reads over writes because our workload, as you might imagine, is a read mostly work load. Finally, in some systems, we throttle the write throughput to the SSDs. The SSDs we buy can do 300MB/s write while serving 400MB/s read, but only for short periods of time (long enough to do 10-20GB of traffic). After that, write performance drops, and read performance goes out the window. Experiments have shown that if we limit the write speed to no more than 30MB/s or so, then the garbage collection the drive is doing won't adversely affect the read latency / performance. There's likely a few rough edges still, since we run primarily UFS in production and I've only tested it with ZFS on my home server running plex. I wrote a paper about it, though it likely needs to be updated a bit. It's near enough to what I just committed, though, to be useful: https://people.freebsd.org/~imp/bsdcan2015/iosched-v3.pdf > This code has run in production at Netflix for over a year now. > > Looking at this: could it be possibly targeted for MFCing? > While this has been running in 10.x stable for the past year on our servers, I have no plans to MFC it at this time. Warner