From owner-freebsd-fs@freebsd.org Thu May 5 14:39:33 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B815AB2E9B2 for ; Thu, 5 May 2016 14:39:33 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 94B051F23 for ; Thu, 5 May 2016 14:39:33 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: by mailman.ysv.freebsd.org (Postfix) id 9418EB2E9B0; Thu, 5 May 2016 14:39:33 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 93C2BB2E9AF for ; Thu, 5 May 2016 14:39:33 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-io0-x229.google.com (mail-io0-x229.google.com [IPv6:2607:f8b0:4001:c06::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 65ED81F21 for ; Thu, 5 May 2016 14:39:33 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: by mail-io0-x229.google.com with SMTP id u185so99348539iod.3 for ; Thu, 05 May 2016 07:39:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=sender:from:subject:date:message-id:to:mime-version; bh=Xp5PyWKsyWbs+3YmbYHOu6dCW6OcxOaLKueHkDQDoPY=; b=y2Zw0IqkNVLbsKNVl07k78HnGItlYiZ8rMD8/HP9m9KAad4YF612CmjVrDOhUVHLmM uE9e9byDrAeR05YeNIajOH8jxJgYme/DhYVgtWaZQtc/zufrfU9QoIAv+IZ+6EYCqqwx LNixQvqY8LcOC1L4jb3Q8pq5P/z6FnqGPTXRWsNuGsKIdIkgPn8Cb+zDKa2cBYTfI2K2 4T5a8ZLdhr8l3k1TcuBoZ3jqmfNvMAXXvDTl2guO4jXyImYwZPdqYpV5+zcVxphBs65N Y9bXMjaF4r7bNgbn8a1odjhU37W6yhYmpzFsaqVIZJ8WN27btkEbsDrhTSEqY4yyc8pg 74IQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:subject:date:message-id:to :mime-version; bh=Xp5PyWKsyWbs+3YmbYHOu6dCW6OcxOaLKueHkDQDoPY=; b=iyzuX6touETmV8qh9RimsjLVj7EqrlD0h0+IE/4fXgAuOS4sTNT/1MpUyct92biV5S 5qU2SdABfUPJ4rgBxoaItZd+gbVCb6DT35aEOMYE+L2uKni6Ira6hrGGehb5QCc2fmSb OwQWCzIpVGoaOJo9bxCiWyfsMPoMML+4d388uqXvosLRWTIKTOGNePm0qCL4Aomfg87n By6pa7w8i4Q9PhAJSQXy3bMpzJLq+OybOwEDLZngweAalczYXH90rd5Y1MWL0SODHvG0 x+PP7V1bzmLEgDFB/J1orj9+sDYG8mqwKxTvWHkNtBT3cPHvmiJO8cSgL7gOadgkDzWd kxrw== X-Gm-Message-State: AOPr4FVVJtbuceepd49ylXht0v4jp/LoD3QaMLwoWfwgCHgnZ/6bYRHXPid8U5cIQhDn2A== X-Received: by 10.107.11.18 with SMTP id v18mr18985513ioi.184.1462459172669; Thu, 05 May 2016 07:39:32 -0700 (PDT) Received: from mac-wired.nflx.bsdimp.com ([50.253.99.174]) by smtp.gmail.com with ESMTPSA id r81sm4144273ioe.36.2016.05.05.07.39.31 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 05 May 2016 07:39:31 -0700 (PDT) Sender: Warner Losh From: Warner Losh X-Pgp-Agent: GPGMail 2.5.2 Content-Type: multipart/signed; boundary="Apple-Mail=_07680045-06FB-46B3-8F58-52B1F98FEAA7"; protocol="application/pgp-signature"; micalg=pgp-sha512 Subject: ZFS and SSD, trim caused stalling Date: Thu, 5 May 2016 08:39:30 -0600 Message-Id: <5E710EA5-C9B0-4521-85F1-3FE87555B0AF@bsdimp.com> To: fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 May 2016 14:39:33 -0000 --Apple-Mail=_07680045-06FB-46B3-8F58-52B1F98FEAA7 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > What do you think? In some cases it=E2=80=99s clear that TRIM can do = more harm than good. I think it=E2=80=99s best we not overreact. This particular case is cause by the nvd driver, not the Intel P3500 = NVME drive. You need a solution (3): Fix the driver. Specifically, ZFS is pushing down a boatload of BIO_DELETE requests. In = ata/da land, these requests are queued up, then collapsed together as much as makes sense = (or is possible). This vastly helps performance (even with the extra sorting that I forced = to be in there that I need to fix before 11). The nvd driver needs to do the same thing. I=E2=80=99ve implemented, but can=E2=80=99t find in my hg queues, code = to do the deferral. After it gets the first trim, it starts a 100ms timer and pushes all the trims together to = the drive. This vastly unclogs the drive and makes things snappier. I never committed this code = upstream because I never committed it to the Netflix repo. We found better ways in our = application to avoid the thing that was generating a boatload of TRIMs. I also never made them = robust. I=E2=80=99d be extremely hesitant to tossing away TRIMs. They are = actually quite important for the FTL in the drive=E2=80=99s firmware to proper manage the NAND wear. = More free space always reduces write amplification. It tends to go as 1 / freespace, so simply = dropping them on the floor should be done with great reluctance. Warner --Apple-Mail=_07680045-06FB-46B3-8F58-52B1F98FEAA7 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJXK1siAAoJEGwc0Sh9sBEAwBIQAOboHkv2tGcS02W6H3py4h7u oNIcVqR/dnf1WN35TKLQbYJFsfFXqGxFVkVl6yP7bjDj10XlyCj3i0dHiA9wi83u F+Y49TxmKUoYtSpmrB/qFLCRZg/RTBPbMAtMkbs/GhuSiGLT8yZ72ZZpDjufuBwl QcCiekZGIPaaH3CRtnysmWuGM7DS54uFdeUGBiiYxirAI3JXDYFZwN0yIPIZI6EK uiH3sO7JmocLg9FJooHwTQ1uIWLH6ujNQibzX4zzv1S/x3qsTVs1WiniFFCuN1MV PJy1cG8cH+WZSFggoDf7qDp4/XJHKPHuUaCDDDYmzpf569ID+emcLvKz0wEkjHyK +psl53DMXBAHPXg9witTqoe/tcxfc4hw3SOsVtVs3wF10+GhIfDxyBc1ynCvfTs5 XPJApJYXYM07XCvk68Vh+2j8SJRgojIsycRNSFK51Qv/tm0m0ept5k+FL/GE+/sc L4+T8W84sdgM0dwuK3fPua2G2gNHjEoknmlMbdsrXJquCXACx64aGpNDnwvF0ZZZ T3iPH2sZfrzK3p+Gx6vHJI34NBAUv6wqDn/Mc6z/UZJyYCHrBKKrAregxxI7KXJa /s7FBeKS3Ysz4SKvmemV03reqDI0ep+nXC1y/4cZoiCakL7TNSk9rjOKmupvQG+9 bNSabmhoVEFeK0Cej4nF =50Cy -----END PGP SIGNATURE----- --Apple-Mail=_07680045-06FB-46B3-8F58-52B1F98FEAA7--