From owner-svn-src-all@freebsd.org Thu Apr 14 22:04:28 2016 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8BF2CADA038 for ; Thu, 14 Apr 2016 22:04:28 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22c.google.com (mail-io0-x22c.google.com [IPv6:2607:f8b0:4001:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 543EA16AB for ; Thu, 14 Apr 2016 22:04:28 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22c.google.com with SMTP id u185so117846709iod.3 for ; Thu, 14 Apr 2016 15:04:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc; bh=zS1NI1kb1gK726rIRewBbpkGLLMfwC3GP5p0lTKIVJM=; b=v6n7AZqig9kmee8RGp01E586Fa6HF0sgTudh3GgIdqIHjU0IjSQt7dVjya62xajej8 IVqA+D8AckvbX0lZLdT7nQito3Kc9P6gZ4sSqQZZ3ES/YnGSUAILyf9vgRB5iMJup2f/ kuk1FZeLm16LJvxNUJ+xIwMO7xbew2NDb5tGTpPWFkYjP6bSzLM2tBQeeK+hFRNkMsit gCYL2n5O0mOpXGGwFpfrtXFLVOihPAvwSNaRm0VJ8n9YKB1QOx35cuzQ+Jw1Y++eAnGF ietNVZq+R4Lxvvbx6jJOqJcMMHMyglOugkwRiWnKZyjXN9PFrSj29BOVBfP6fOC9FJka d09w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc; bh=zS1NI1kb1gK726rIRewBbpkGLLMfwC3GP5p0lTKIVJM=; b=NI8DcCUkl27llGKMHSwcRD79tqcBGQ71pNPvma5n/nMT3g4MqqBjJHrr2EH352nLOh c6/guP1l+JW2uRPenMx472AhtO3ImDSLAmnll9kCY6sRRUiOxS29XaJnJam5R9CkF3Kt llIigFU6UaFD+S9K6vY5Hvbo4eu4dqLqOgDfLITC2y+hRKNs1qAVfs0zloGT+7tBpP2D N82ydscT1HcvADpR6h4n1KX1gIgKpyKmbX8idfwny1BqVKP/w60xdsygVCgodcuJ5B4b h3y1RlEuG0VrQcgdqH1bQFm59VKb82RvPfR5r1i/HXRm6koraJaqE++c55VWzGIEQ33k 2Ocg== X-Gm-Message-State: AOPr4FWb2F+Jaai1cSKh4lHFKDqH86K4PxaziaMQAjaY5dZtgnCx+xCRjNnsVrcf1y8vxN3j2k5JbtqYXpF1Bg== MIME-Version: 1.0 X-Received: by 10.107.14.209 with SMTP id 200mr18397734ioo.73.1460671467638; Thu, 14 Apr 2016 15:04:27 -0700 (PDT) Sender: wlosh@bsdimp.com Received: by 10.36.194.3 with HTTP; Thu, 14 Apr 2016 15:04:27 -0700 (PDT) X-Originating-IP: [50.253.99.174] In-Reply-To: References: <201604142147.u3ELlwYo052010@repo.freebsd.org> Date: Thu, 14 Apr 2016 16:04:27 -0600 X-Google-Sender-Auth: ejUv0SPYSK3QSNkJJVLLzHqn1EI Message-ID: Subject: Re: svn commit: r298002 - in head/sys: cam cam/ata cam/scsi conf dev/ahci From: Warner Losh To: Dmitry Morozovsky Cc: Warner Losh , src-committers , "svn-src-all@freebsd.org" , "svn-src-head@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Apr 2016 22:04:28 -0000 On Thu, Apr 14, 2016 at 3:54 PM, Dmitry Morozovsky wrote: > Warner, > > On Thu, 14 Apr 2016, Warner Losh wrote: > > > Author: imp > > Date: Thu Apr 14 21:47:58 2016 > > New Revision: 298002 > > URL: https://svnweb.freebsd.org/changeset/base/298002 > > > > Log: > > New CAM I/O scheduler for FreeBSD. The default I/O scheduler is the > same > > [snip] > > First, thanks so much for this quite a non-trivial work! > What are the ways to enable this instead of deafult, and what ar the > benefits > and drawbacks? You add CAM_NETFLIX_IOSCHED to your kernel config to enable it. Hmmm, looking at the diff, perhaps I should add that to LINT. In production, we use it for three things. First, our scheduler keeps a lot more statistics than the default one. These statistics are useful for us knowing when a system is saturated and needs to shed load. Second, we favor reads over writes because our workload, as you might imagine, is a read mostly work load. Finally, in some systems, we throttle the write throughput to the SSDs. The SSDs we buy can do 300MB/s write while serving 400MB/s read, but only for short periods of time (long enough to do 10-20GB of traffic). After that, write performance drops, and read performance goes out the window. Experiments have shown that if we limit the write speed to no more than 30MB/s or so, then the garbage collection the drive is doing won't adversely affect the read latency / performance. There's likely a few rough edges still, since we run primarily UFS in production and I've only tested it with ZFS on my home server running plex. I wrote a paper about it, though it likely needs to be updated a bit. It's near enough to what I just committed, though, to be useful: https://people.freebsd.org/~imp/bsdcan2015/iosched-v3.pdf > This code has run in production at Netflix for over a year now. > > Looking at this: could it be possibly targeted for MFCing? > While this has been running in 10.x stable for the past year on our servers, I have no plans to MFC it at this time. Warner