From owner-freebsd-fs@freebsd.org Tue Jun 20 18:50:32 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A766FD9F4E4 for ; Tue, 20 Jun 2017 18:50:32 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-bl2nam02on0069.outbound.protection.outlook.com [104.47.38.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 041F96E86B for ; Tue, 20 Jun 2017 18:50:31 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weatherford.onmicrosoft.com; s=selector1-ca-weatherford-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=sT6O56LXHDB+Ckj0yAzcA7LZwcbIZQ1nZP+5HPkD5mo=; b=t2Pi6+GkvyK/HJtGbDz4Lr2MVvjq8valLiDXJGdJjLe+rh9oaeQM/MJk50070rh9FWISq9hR5lpxVeltZoED+S3MIqNLyKEkIlxZ108HjwurYToYBONOLj/w9A4Y5aWgDY+4DhntsMLP9ibTGo66DEkNlUfMv7t05ONqdBppUVU= Received: from MWHPR03CA0018.namprd03.prod.outlook.com (10.175.133.156) by MWHPR03MB3166.namprd03.prod.outlook.com (10.174.174.35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1178.14; Tue, 20 Jun 2017 18:50:29 +0000 Received: from BN1AFFO11FD009.protection.gbl (2a01:111:f400:7c10::176) by MWHPR03CA0018.outlook.office365.com (2603:10b6:300:117::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1178.14 via Frontend Transport; Tue, 20 Jun 2017 18:50:29 +0000 Authentication-Results: spf=pass (sender IP is 23.103.226.20) smtp.mailfrom=ca.weatherford.com; denninger.net; dkim=none (message not signed) header.d=none;denninger.net; dmarc=bestguesspass action=none header.from=ca.weatherford.com; Received-SPF: Pass (protection.outlook.com: domain of ca.weatherford.com designates 23.103.226.20 as permitted sender) receiver=protection.outlook.com; client-ip=23.103.226.20; helo=032-smtp-out.weatherford.com; Received: from 032-smtp-out.weatherford.com (23.103.226.20) by BN1AFFO11FD009.mail.protection.outlook.com (10.58.52.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1178.14 via Frontend Transport; Tue, 20 Jun 2017 18:50:28 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net (141.251.110.81) by DM2PR58MB014.032d.mgd.msft.net (141.251.110.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1178.14; Tue, 20 Jun 2017 18:50:27 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) by DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) with mapi id 15.01.1178.018; Tue, 20 Jun 2017 18:50:27 +0000 From: "Caza, Aaron" To: Karl Denninger , "freebsd-fs@freebsd.org" Subject: RE: FreeBSD 11.1 Beta 2 ZFS performance degradation on SSDs Thread-Topic: RE: FreeBSD 11.1 Beta 2 ZFS performance degradation on SSDs Thread-Index: AdLp8ugzXKKDkz6vS/yx+crAeAsw7A== Date: Tue, 20 Jun 2017 18:50:27 +0000 Message-ID: Accept-Language: en-CA, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [141.251.205.196] x-ms-publictraffictype: Email X-MS-TrafficTypeDiagnostic: DM2PR58MB014:|MWHPR03MB3166: X-MS-Office365-Filtering-Correlation-Id: 7e2e0221-15fa-44f0-2543-08d4b80d3881 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OrganizationHeadersPreserved: DM2PR58MB014.032d.mgd.msft.net X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:23.103.226.20; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(39410400002)(39850400002)(39840400002)(39450400003)(39400400002)(39860400002)(2980300002)(438002)(24454002)(13464003)(42174003)(51914003)(189002)(199003)(377454003)(9170700003)(38730400002)(8936002)(6306002)(55016002)(9686003)(81166006)(8676002)(5660300001)(97756001)(54356999)(22756006)(50466002)(33646002)(50986999)(66066001)(229853002)(6246003)(2906002)(53936002)(305945005)(8746002)(47776003)(42882006)(53546009)(106466001)(478600001)(72206003)(6116002)(3846002)(189998001)(102836003)(356003)(7736002)(108616004)(46406003)(2900100001)(86146001)(2501003)(7696004)(23726003)(86362001)(24736003)(5890100001); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR03MB3166; H:032-smtp-out.weatherford.com; FPR:; SPF:Pass; MLV:sfv; MX:1; A:0; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BN1AFFO11FD009; 1:T8ENabrBw0ApL6knpcg8ax/ErTlSDYO+X+mcCd1+ki2Dffd+6RRXtHZNZK1p9Dgz94OEVSpGCQGthyfhlc8f1Z3zt+ZSeGUoiBhVrPYWs9WMCcjL2QqFy7pSW8tjoh8pvPv2Rkn8afT+Pjnu6LOqhCe8MHhf7KOK2P8GfsnKjL+P0qGTMDPLfBhSn8EN8aOZAzmK49Qpq+P9X1pBPCXqPsAZZa23GRmttNiHR3fbaRPBpWQvc27DMxekWwAkDQuiEjNwE+tpsyfIE2IEWYeuSOqaiS0Fo+HFwfH2nhCkRug7b1t1PTs+KtUfhzv9NzFfVd1Jl45lTpingflRr/piphmAaWZYKO/jCXtQsNe3ft7cp7dkSEomFdj4eRgco2OlR7Q1I8s0lfmmwss/AFE5SlP6xdjt2q4m1VC4N1/J4LmNwTDNpcThNEUEhhTwYxp0Jc9TxfYzxVXe5fogRamIE0AYANrHulkAXEOoU0rkMuF1UT5HXZWdaBHSiJN40ujkZe+R2OoQ3sDDw2t/k3AQVQ== X-CrossPremisesHeadersPromoted: BN1AFFO11FD009.protection.gbl X-CrossPremisesHeadersFiltered: BN1AFFO11FD009.protection.gbl X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500055)(300135000095)(300000501055)(300135300095)(300000502055)(300135100095)(22001)(8251501002)(2017030254075)(300000503055)(300135400095)(201703131423075)(201703031133081)(300000504055)(300135200095)(300000505055)(300135600095)(300000506048)(300135500095); SRVR:MWHPR03MB3166; X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB3166; 3:2W0P9Z/G4E2qeNSjwDmlCBl6RH1/a3MhBciIYL0rilefQ/OPdqau5gQCMdqt0t48sVkdWjiHgofeuuyq6VrcsgdnbjfUpFGFD9a1a3xs62UHUCzpdMjKyq9doTB2uDRubR1EnnTZvB0lQmCagU2dSqCz4fkRDrXHHla5rX0sFW2bpCZzkxePmg9kCPs4WWwnTYj3bsJx9rBiJRKbiL/eJQie+buJt0VUv8c/CXZuHPLMX0Mi9n8FflRXy9bGrX8UhXARlC/Mw+KPtXz5QE4PceIUT6MZXhpP1JrB0tQhi1xy2jATHDnFnOsB4+wFNvOVNysYxAxHbSHzYGYxSwMQV/Hxd90tuyAfLnJvPf6JHU78lHlI/MkSqq228nCSYfajNwCo7agkgrPAzy2O/kgBx/SQvMNc3506i+WFYS77PeE6Il9rTXLcZLOcoQBrPdPuk1YQ2cXiL0iPMI8sli6saRQA43MZkKYmLjD3LTGYSb0FrGbCEfCetTgmJEOxdK85QMayui7xDSQeMI+TOSIs/PNAIzUVAbeYD6nJM0In7lp2rAv2/lOrnIqZR9nT7IFjK++47KHlnZgCOH9z4ydOTgwU5MdbdQIcRgSrMtm5SDFWm1k7ZpRJx8h+3i0TMCHtgmNNUHt086QB62O2TRBvVhyPtbu4AcC5uTEebS+/hAOmh79PhUl5pXVOSl8mUYJgHZX4p9GzeHVpm/VHsP7gpN8DNSdn8Ar8oGj9jaDiK9kfr9dpZ5iSwh5kDSLwJSJEG+IdF0718nhHHlhu3Br0oFzbnf2n/2XxNYklPw5crs1BY8G1C37j6DvNMSybwdVmEtLlUmrWJTzv5Rrwu0N2AykuWq+R67oCi4JVDJVGzHuJr8MVZOfuoLZUmvIVwfxDw+X1NXw2QVEL1Xg/YRpmzQ== X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB3166; 25:MFvI+kFXLtk9l7v8I8Ix/zuKLpWj3ELMdpNZPyGmfbba1n7cTgX4qiyc96XKabVvRPq0PQbkjvbCyTmbrAj5q8Av8JDrrPjj/q6ffRtFto9nfutlQv1aYosKQRTpv15mg13XZ4Co0osC6l8et0NgOZBg2hzJpkukEItevT8yShQYcTm+cD/4vsJappQJ14TXq0Y4+0u4CeVMoR6gx+ETiYc+vMCFFRxSU6pN/Tuu6/jxr+0EEiQ9ThsO6LFWKOQkm4ZW5GHyePeR0GvpIxQcrBA/x1qZeNKsdW2kJ3tQZjsfuOnCLuiPNXHgsMdiwftK9cD/ju8MpnIcEaZmCT4++s9U1t7g3ZQ8AgH6PUu6kgQamE2yjzy9no8Q9Yj+jTRaJ0XQ5ebAnWtUicOQT8CHs/DzQo9pKX6BgAIuE9GwpnyyO9QgjXWLS/mubNEiyjGKc93y3VYTa48ED8EzkDDDs+Hi/wm4wEx7zq1EyVd9GC2vs/m3C8h9m1LZFMbZtn398F1eEWGir5v/Mk6cMWyFbKF48mL+iSboLxRmfJYjwT8to0825V4s5Ihbj6aKBReiNDuugSIDHm7pCN2tzNIRJMPtz+5TZT1jHaYL6pUiAELJ7FXWTZdLMg0nxu9O3Uv6y8DwADaAYdRtMUHd4eY4u8HryhTR30N4wBmmWUcya4d+YQKdNql7sw6Scg8TtwSk66i9gxM6WTiyduRHqwkJhB7w7UcEivMVmG63krQGRdfknpwPbc/Tuz2qMClGVxowkEWIheyd3mAXA7NZPV3tnwk2X1zmGft6gB6nP2Os4xwcp2slCGu3KtrCpAKQYYRlSj1Fv4WJuAp0AjrfiULv4WgG3PQuJsH6uIPoocfI59NQnV8pBbrKJOY2Xy6iJ5L4vLv2RCCXAsAZGXLP6nXARvAsjNlx/SN6xM0yjiwYwao= X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB3166; 31:sEOayVR9HMPlWyuky19TNqDQi54ODJ/GoaZRUg5CF7/RxPtXQeisXKqiX4MQrbjh0rvzsNbQxQIqB1PA+rK5BOpb9yvcrJ686Finjx0/d1HwUmoX0sJtcINnwD4UA8BGI4zmzW/UCs5b7/MfX3si/AmxwVUboggrRZ9HUAMaVkUxgwl9iCTMktXKQ2g1/JXkVXgBiNtCo6JfIgestkI3Ed8hmjh9VBdjdEHfVNav+eNGe2D8fN361RJQ0aCw7EuIaHwywMs8IVgMrg+mtCb5VZA1Neqy43jzQn6uVasBzlOV3IvG80oqUqaF8LwpowzOQx5IE3uPTgy/jUx0tZ6KSHAJj9PoFHdTVLyefVpWGjfDLHhF6NYq+B26ekn0MFxpCbmUAcvHZU4qDOh4Fi6nXEy8U2dOnhi/9rNtDeT5U5UXf4v0iIPuE3FVAHB4n0fWH/xKT+1pGtsuxxrGkkPOKYrihLyU7KQMz1KiNQ8d1VE1asflMCEZv7dSTgd1Jhl2gFyjIxldmRbymL1HzpBtC6Xai9d0a8orJaOQIx0Ut+rRDFpd5N95gEx/xScFAsxYK2TF1REgbnJa2NojH3Pb+BsoYzgZb93tpiAsVoP3TeD4Ze8O9K8dw6WWrI9Vjqj9xbW/f6kV7/fBIMYNkBsTJB2xUpm4FjHJSAg0eMJCg46twyGfZavdTzIVZCzhgl3R7mgjx9ityLrQhS4qMlEbsQ== X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB3166; 20:dnDKvTSucW7JPXpm5n7QLm5ShCps69fuZT4e/OVynfzbAIB06b9fnToja4GtQQSMU8PhoDL6Z4yKGpSHrAKwAY5xkXpntx9d1t01OnBpozuutsj6363V5IfBvm3R3p26sgydxfpYmDgp4oqhUWV/ENQgProNtASIN5kaQOVY7yAnxTzD5MZwEqNeZB/4dFix+LcAGcbgsgHsqQoYQqHR/MDUvRRyg/v2+6nTTFp+hsYeuXpNdn/bvz0rB0zaajLvw0uaD3FPvI6X4aB9khLEpPuvUhZg9GOTTlmT4VMNUs6ZKHHmSAbwVQuV2wyzOwk7QAdTTWWBGQ4BXD+1G8cNo8Fc6mYgmHHvqF8z/dyTaOIg1NbrQiTfoigVubHnFu1xp+Yg1aiy2Vg6iBTFGn378xfyxFARc5mXFOkOCx1ziSOsBJQ7f3h3CE0bzEd+CZmm2S4p3gaCAi10DrKP6zGUya84BqnSrTJjT0pA05wTyZARKXvRCA+X0sfxtI+q+IAS X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(158342451672863)(192374486261705); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(11241501159)(6040450)(601004)(2401047)(8121501046)(5005006)(13016025)(13018025)(93006095)(93004095)(3002001)(10201501046)(100000703101)(100105400095)(6055026)(6041248)(20161123560025)(20161123555025)(20161123558100)(20161123562025)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:MWHPR03MB3166; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:MWHPR03MB3166; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; MWHPR03MB3166; 4:etbsJ+XWPKt6AxfCgMN3z8Xvoz3m/t8X/M9wG0OFa1?= =?us-ascii?Q?4Ypt/V4a79P81a9WadFkz7TSgwIqF9us2ED8reNU8bAYrsRcyEJX1wXW7sae?= =?us-ascii?Q?zOWRGCrRzZQ9S0S+uvJQjurWrF5nNMzhYI2iTg3RKzoO3EnpVrW13jfM1pJM?= =?us-ascii?Q?4WuAmqVOS9iCI21mFZ0IHrG0Ast6Kp97mkobzI3jni27IMDTmP0cj+P+HW1l?= =?us-ascii?Q?WWfRjxR5UnAHkE+e834k5lX2vF82q5s22Sxzln9wBAS1pG6X/hEZgi1bllxB?= =?us-ascii?Q?j6LijK1d14FYr9fo3slKAgf8IW8n8woscyHoiYM6TyvplGi+b3JffbpHcQ/X?= =?us-ascii?Q?flsMsmDcF0nnnFohcQCUnjT1mxE9M1KkULjFYdPN2U/47nCdZh+aWP/qv5x3?= =?us-ascii?Q?Q4gc+zuhc1zMWKZqeK79F6RVOwUh9IpDKT+S9zldblkY5mvxuGFTMqLlvFxA?= =?us-ascii?Q?EsFOKEuvU7aH6gX/EOcj98r80ZlZbJGzEgjkAibh71yFCbxiVoca1aBxMIZj?= =?us-ascii?Q?kRI9FJgCKoImblYLBNVEJu2+KCu9e1+dnRKE+IhFO0AGHxa2RxGodYGPAmbq?= =?us-ascii?Q?Mar1+dXJ6LuXm0+xjKkaLymKXoXESNaq9SwgAKnpJmtqrW9XrinigAHFFZcn?= =?us-ascii?Q?WLYwQNEBNSUYJF1ANJm54xg/gCSmuSruxA9eeUCExxNpJv+Qqifh45HLv1WC?= =?us-ascii?Q?IfkUM0AyX1BOowmw2QRLrM2UEfTG79XoJKsELrFx1/M/jwfI+ZE7oeI1RI3p?= =?us-ascii?Q?3XCPez4CLnNV+wFsEDed0M9q+EqmfPHFsGGOGTBE9Ql9hUAcOVMWuCyc3ZZ9?= =?us-ascii?Q?7olVtHnpM1B1ta8RqUTod9E4DmSZXnb6dxh3+WeCcJNTka8WFALVDnHeRmX7?= =?us-ascii?Q?0X+UUfVQt0rVytsZXZ3JGvYHfCnx1/tiAK/h6x43zwkNd8Dxy75vpdlDhX5T?= =?us-ascii?Q?zSLV1w0cd/GMIOuDBgvKh3kZ8iZAi6n8HqQJEhCYRbGtLnqrAaftDTC4zt6K?= =?us-ascii?Q?OFawYU7zw72pbaMIu3FlR9Tk6MbJR3KRD0snJV6t4ynMNQs3bDCoHzZgFYBV?= =?us-ascii?Q?fGP5CznX+Pr45S63odusX7rNAB2p8Z7r6TCjhnOqc/VUN6CXqXtq9i4011ZY?= =?us-ascii?Q?GiJ2b9C6N1ypAtyCI9jDkzlUrAyxdelWe84tTJN56Zmf1D3zMzF5UnsZNuxY?= =?us-ascii?Q?0A6BdQI78QZpmtyB8OdOHUyYCSeIBmhXEDlAMu74QAvARqrL4hkoCJJga4fj?= =?us-ascii?Q?vWtZfuAYsRjYZ8Nd6KwjK0f9BqB785X/mY7oFvPOwZdcIQHyIacOjrXwzVy6?= =?us-ascii?Q?mMKhaOrMXqUOnTGH3Lt9w=3D?= X-Forefront-PRVS: 03449D5DD1 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; MWHPR03MB3166; 23:qfwOJKhxOaQBAwU75qmfLU+31QsV+NiAxA8ol5uYx?= =?us-ascii?Q?qK+cgNyELQYw9j3OT1WfzuuomXk/d3R6Sl9G5e2HOaKoFwvJT4SOQoNr0aal?= =?us-ascii?Q?bXrZbrImLoJIKOVXWLfdIdX8NDvJbk6KNN8ZJ4oZVbn6jSf9Iqa8lgyQu9Uf?= =?us-ascii?Q?R6DskBqGDfu/pTyBB+kGu2xP+mlL1Y1cxn28KhX4Ed2mI11SOlnBUnUwFrNS?= =?us-ascii?Q?peTHE2AqcVUMq90AXgi5UfXEVuUabnu7eTGer03/yRm0gH83kH0N1kPHE71F?= =?us-ascii?Q?V79pvo9rShdQrhffHnuqJmNF6AZqpZBDsk7BZa21qP3XZ+6P6N0MyL8i8rEv?= =?us-ascii?Q?oBwt2qU26t0pfaSDAezczRvUFZqalrUrUz+UdYEhzY0Qw2gyEN0x/73wha1q?= =?us-ascii?Q?FgDYoXRGqmwNeWp38XZYu5Vn5+DkLt6hztGW0VKNb496HR6QPOMc4sZsNj8L?= =?us-ascii?Q?Us7O1k2H9HlUbZBI1gcyIIZCBkyhwFake7LaTUDCN6oCAY9QmQ6b7Ll7Gy0x?= =?us-ascii?Q?HTP9FFBkjQePLV6D08RMrdDKmZ90NS001NgRQ4hPiO3GRHqaUYTir2uXrAU1?= =?us-ascii?Q?G2NMOXq4tHSKVvqQT9Pga3/mW0WO6R/AnS0UJZS0pS5Xi2WXaPZaogzzgFj9?= =?us-ascii?Q?cT2ra2sYdlkS6WVQc64G7Z6POnvbynTL58+sDrl8zHH0bmN7Uiool6MikLVS?= =?us-ascii?Q?uTHgb0c9ODF+EA5r5b3MW1IQ1rN5KK0plBTp+HM8eB6ymKzueAtjzvivftVs?= =?us-ascii?Q?Y+e6EBfNqRktrdfDHs0KwqpM8U3hjdTKu2lvcBWpiv4kpO7pyw9mv/lKG9Wp?= =?us-ascii?Q?vW0AZOxpR6BtZ3W1l4Jn+EKAveGxSaXS+ZJ7TnGd3Z/0H+JIGCVq+2mhning?= =?us-ascii?Q?v060Mog5mWGuRRpFfAlz3KcUiUi2pXKkurzEHijlh/2taeNil8htu7L/xciM?= =?us-ascii?Q?SB3cvEN3rUo3tl50ARFmSfe9mMiYpKIMbKMR7IIfP3DGX1R1QqdAoOVrR50x?= =?us-ascii?Q?2SXsigvjcKrFS5VWSlHCFvRaJO2CrIa/wzmz4tlAgVNZfeNsrJkh1xLrlbr5?= =?us-ascii?Q?bKbH2NtKmNiJgbS87rGEW0g0cFtVA67kY5JMkJHKcPEXIhHrHxHRBMRVu5en?= =?us-ascii?Q?SaAspy/bv55SN6wTddMIYshXXUJ33lEcgGF6RonHPREHi46osOQDk0BYnOzX?= =?us-ascii?Q?mHuI1KxOBM3L7f+T7csgDZhuZKw7EB5rcCCRukgCdOx//QNIrDtfNWVhXwMw?= =?us-ascii?Q?Kbwb/X7kwXfs2bwO+cJqPmqTEUycSzgI2pKPHEdoCi4nM1dMpHL5e6UYpDbB?= =?us-ascii?Q?KUI6t08rnslBOfWUqVe45pw18ZXPlAZNQDEZjCP9iB05FONYtQf3yAE4/I57?= =?us-ascii?Q?NkySe/9tnzyhLCEC9CLFwsD0tJ01a1JiimiHRvP+fheAT4g9piwaIc8BKV08?= =?us-ascii?Q?RjFXTwtUI5zpuSvk+GHiRLsY5XWrMI=3D?= X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; MWHPR03MB3166; 6:8vw75vBXynj0VBV89ysRnM7muU/tCtto79j/WzpXsL?= =?us-ascii?Q?zZudVAfyGjubAigxsBDu4bQNcW5P1jIPK1JnMKO3izVhtDaa7US/avgUFTQT?= =?us-ascii?Q?L4TfIz7f3q4xj51RpUTbsXeJZjEH37Lf6fF47EOHVv3K4sUiBykmrDYHBjgS?= =?us-ascii?Q?2Dkgpb8Tz9QtGRKrhXk5l90eTT8flcPhRHzIdCupn5ZqI8yfK3y2F/VszMOM?= =?us-ascii?Q?1K7GxY1te8W+yAzetB5tkbZDzwJGzHmuJEYGokmGttvjkqxzHr/vtYSKwMip?= =?us-ascii?Q?72xMa6NpIHdadbrKtkHQsXqEnpQ6ORBYSu7ID4a6b1vPMlqv3OSSS0nBq56O?= =?us-ascii?Q?G+OgQPPetTkx96TTV+vtXlrUsS341xc78bWj5kd0ieciMBHnAgwbqwzSJpdN?= =?us-ascii?Q?ZNE6GOn2YAMwAS+kvYbbLASfb5LTXZLsSPKZQmV6ewWk4c46nmf74wY0Q3gw?= =?us-ascii?Q?Q6s9O4wydxEQ/SaXZGjxPLdIuJhafY4hoBvyp0huZB7xONfHMyhyhoUwBD7W?= =?us-ascii?Q?O8os75YHeGJHAqD+QVyROQtLRK7V26v3YOYanQG5uId4a0a3iAhhsB99F9Va?= =?us-ascii?Q?OQJnmLo2ADxfkysqTE87fwZO+YcwdqAGbGCLPv47k5vcVJ1SiOz/x8zzGZg+?= =?us-ascii?Q?XEwoONX1OJraS+gnWpaZW8+6bfEFXXzYG8end/2TZyRTHxiENZHN+/j7BRLm?= =?us-ascii?Q?qSPl2+tk3gpFb+FP8j5gc6EiIK5a87PNkamfUFoPhGAwqV4V9pgp3H1/a4vn?= =?us-ascii?Q?YftNfaFGAoEVZ8W4KoMlZkwF1TIZMkHXvCxdCA2k9OKY3DwZSidx7A8k4W9x?= =?us-ascii?Q?TXZjuaNIztwWhlyXo/1DkY+rhHRfHfhwb6wuQp25vFt6rhyyR5YA961JjSYy?= =?us-ascii?Q?g38gNDI+7XyjpghpDUNsCNVCD5BZazVPOqYZcP8or+CY0u+95PCKiptiFVkZ?= =?us-ascii?Q?rXu8HgUnpHUk2jf+9mefvNWE4TMrkvL2H+8/T3Ygd5aXrAOiWy61YTtaGdKV?= =?us-ascii?Q?g5Sds/eZPZv1Vb/uvkLEyx?= X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB3166; 5:q9n7tAfvYw2eIz6ZwuvaNBvCWevIvRjxJJ88eh6kn5DL79rQogNvhpsJZYH7tAfATKrpkagyRsRrCGvNqEN5TlMH32ZTrz4SpmajFJbTyDJH1QbLRZ+X6fszm9Bmjr/HUbWBPw1lVc4hrXFQIZ+hatmBNC9xjpoYmwSqiElThR8AWWtH95Wgqx6JhF/+C/JQFOC4NOse8iepVlU55VS5h/ZhyY0WBXZZWJ+hWYcHHRfc4taqNw9YNXGJRQowc61sDGRJgiDHxjcheCL+2d3QbDEZv6s8S/0GWMXsE4S0X7lGhKbHNWfHASH7hE+pZrVtGwMCvzKEy3ANFdVii64L3LBUqupPAmv8d1Yu7UmkUnqdjeDCm7OCFvyDafL4O2r2UJI3nYYT0GbKhXdCOojF45cALVecdmo0tBn8sD5Tnqji9IupIZCC+bp3RZI+sj/4dF+SrsYqFTXTNMoJpcxGckm5ZQRgvaSwm7GdMBz2HwddwNuj/TSSkk1nyHB4QMzm; 24:9iz5HcdOy/oF9APTxRzUHsDjYT5gvCFHVnncTwgv7wmLCnYIyipIj1Oerlb8XvW58dSJSTJn+EIzrtan2dBYY/APIfSs0HgNRpLK0lY9gVA= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB3166; 7:THgvPC+HRTeKOnl43WnTkCH1mbxllQD+aA3dQYeNyFdP4r/0KwFZCO9MVivyBVlUSuhAUEzEoehTPsLiJMmtrCh7eMNJAA2jmPfVZBr4UmkUUx6Au1Lm/lDV1ic4A1VjNt+/d9WhlvpCtJNTLEpfgMxQRSqUB47YNNKDvSwD3eDyfvu91jfcSvaIbvW2RDv0dYjPzDxsvodgSpMMm2ExyLGAI06g3YDoLoVfksitmirFcXrO1QUFM9v6LF62iR9MAO+pHLrCLuInIq7ZaMi+mRIjY+2sh/wwY5YC4ml2HiXsyhM3QAWZddSpce1NbpE1vEbNDTnmvwLKx2FhDmGkEwqQUTRhKQBQr1EeUu9SxRcNX2oJSM5jjIGkc5kHHAdvA63CxOq1XlWSwxcO6wvTJHOXbhWj2p0qx/JkQa9NDwNE+JWQaWIBOECBgpBz3eo5kWupSB/jNB9nK2Cq8MYXYtf7m0QiNhWWMgGnhhIoO/ZZCG/Y4RepO8ZmaMUIiwO2Q8VyVqjvnZV4sJywnweVCytNElfvexmwwRtJO3esSCKQMW+o7/h4wAYpyS0VBKt65rK/JazVCKSiLfgPeqFChfFa7ab3hbh3WIGoAfA3hk1jkqjO5To1cHf7yEZJZoCPvMgPhysOyxHhoPoBXjHVvhr9q7X/CDtMlw1/t6QuE9Hu/I3LMXcO8Y094DimhyjPdhrFwakC5r6rtGMaouxzcaeb79jODVnVzc0vsFTsxErHjPdarXKlSXax0RfEMp9gS9JuVDEVu6tglnKLaacimLsfMXTHDaWxY5zPtvP0Mew= X-OriginatorOrg: ca.weatherford.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jun 2017 18:50:28.7375 (UTC) X-MS-Exchange-CrossTenant-Id: dd63fb60-07f6-4d96-8d40-ebeca61a524e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=dd63fb60-07f6-4d96-8d40-ebeca61a524e; Ip=[23.103.226.20]; Helo=[032-smtp-out.weatherford.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR03MB3166 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2017 18:50:32 -0000 > -----Original Message----- > From: Karl Denninger [mailto:karl@denninger.net] > Sent: Tuesday, June 20, 2017 11:58 AM > To: freebsd-fs@freebsd.org > Subject: Re: FreeBSD 11.1 Beta 2 ZFS performance degradation on SSDs > > On 6/20/2017 12:29, Caza, Aaron wrote: > >> -----Original Message----- > >> From: Karl Denninger [mailto:karl@denninger.net] > >> Sent: Monday, June 19, 2017 7:28 PM > >> To: freebsd-fs@freebsd.org > >> Subject: Re: FreeBSD 11.1 Beta 2 ZFS performance degradation on SSDs > >> > >> Just one note below... > >> > >> On 6/19/2017 19:57, Caza, Aaron wrote: > >>> Note that file /testdb/test is 16GB, twice the size of ram available = =3Dn this system. The /testdb directory is a ZFS file system with recordsi= =3De=3D8k, chosen as ultimately it's intended to host a PostgreSQL database= =3Dwhich uses an 8k page size. > >> Do not make this assumption blindly. Yes, I know the docs say to set > >> recordsize=3D8k but this is something you need to benchmark against > >> yo=3Dr actual working data set. > >> > >> MANY Postgres workloads are MUCH faster (2x or more!) if you use a > >> default page size and lz4 compression -- including one I have in > >> production and have extensively benchmarked. The difference is NOT sm= =3Dll.. > >> .... > >> > >> zroot/ticker compressratio 1.53x - > >> zroot/ticker mounted yes - > >> zroot/ticker quota none defa= =3Dlt > >> zroot/ticker reservation none defa= =3Dlt > >> zroot/ticker recordsize 128K defa= =3Dlt > >> zroot/ticker mountpoint /usr/local/pgsql/data-ticker loca= =3D > >> zroot/ticker sharenfs off defa= =3Dlt > >> zroot/ticker checksum fletcher4 > >> inherited from zroot > >> zroot/ticker compression lz4 > >> inherited from zroot > >> zroot/ticker atime off > >> inherited from zroot > >> > >> You may also want to consider setting logbias=3Dthroughput. In some > >> c=3Dses the improvement there can be quite material as well -- > >> depending on th=3D insert/update traffic to the database in question. > >> > >> -- > >> Karl Denninger > >> karl@denninger.net /The Market Ticker/ > >> /[S/MIME encrypted email preferred]/ > > Thanks for the suggestions Karl. I'll investigate further after I reso= =3Dve this performance degradation issue I'm experiencing. I recently read= =3Danother FreeBSD+ZFS+PostgreSQL user's Scale15x presentation, PostgreZFS,= =3DSean Chittenden if I recall correctly, who also advised lz4 compression = =3D 16K page size rather than 8K with PostgreZFS. > > > > With regards to my performance woes, I was originally using PostgreSQL = =3Dn my posts to freebsd-hackers@freebsd.org but started using 'dd' to remo= =3De it as a point of contention. In attempting to resolve this issue, I t= =3Died using your patch to PR 187594 (https://bugs.freebsd.org/bugzilla/sho= =3D_bug.cgi?id=3D187594). Took a bit of effort to > > find a revision of = FreeB=3DD 10 Stable to which your FreeBSD10 patch would both apply and comp= ile c=3Deanly; however, it didn't resolve the issue I'm experiencing. > I would not have expected my PR to impact this issue. > > I suspicious of a drive firmware interaction with your I/O pattern; SSDs > are somewhat-notorious for having that come up under certain workloads > that involve a lot of writes. > I've observed this performance degradation on 6 different hardware systems = using 4 differents SSDS (2x Intel 510 120GB, 2x Intel 520 120GB, 2x Intel 5= 40 120GB, 2x Samsung 850 Pro SSDs) on FreeBSD10.3 RELEASE, FreeBSD 10.3 REL= EASEp6, FreeBSD 10.3RELEASEp19, FreeBSD 10-Stable, FreeBSD11.0 RELEASE, Fre= eBSD 11-Stable and now FreeBSD11.1 Beta 2. This latest testing I'm not doi= ng much in the way of writing - only logging the output of the 'dd' command= along with 'zfs-stats -a' and 'uptime' to go along with it once an hour. = Ran for ~20hrs before performance drop kicked in though why it happens is = inexplicable as this server isn't doing anything other than running this te= st hourly. I have a FreeBSD9.0 system using 2x Intel 520 120GB SSDs that doesn't exhib= it this performance degradation, maintaining ~400MB/s speeds even after man= y days of uptime. This is using the GEOM ELI layer to provide 4k sector em= ulation for the mirrored zpool as I previously described. Interestingly, using the GEOM ELI layering, I was seeing the following - FreeBSD 10.3 RELEASE : performance ~750MB/s when dd'ing 16GB file - FreeBSD 10 Stable : performance ~850MB/s when dd'ing 16GB file - FreeBSD 11 Stable : performance ~950MB/s when dd'ing 16GB file During the above testing, which was all done after reboot, gstat would show= %busy of 90-95%. When performance degradation hits, %busy drops to ~15%. Switching to FreeBSD 11.1 Beta 2 with Auto(ZFS) ashift-based 4k emulation o= f ZFS mirrored pool: - FreeBSD 11.1 Beta 2 : performance ~450MB/s when dd'ing 16GB file wit= h gstat %busy of ~60%. When performance degradation hits, %busy drops to ~= 15%. Now, I expected that removing the GEOM ELI layer and just using vfs.zfs.min= _auto_ashift=3D12 to do the 4k sector emulation would provide even better p= erformance. It's seems strange to me that it doesn't. > -- > Karl Denninger > karl@denninger.net > /The Market Ticker -- Aaron This message may contain confidential and privileged information. If it has= been sent to you in error, please reply to advise the sender of the error = and then immediately delete it. If you are not the intended recipient, do n= ot read, copy, disclose or otherwise use this message. The sender disclaims= any liability for such unauthorized use. PLEASE NOTE that all incoming e-m= ails sent to Weatherford e-mail accounts will be archived and may be scanne= d by us and/or by external service providers to detect and prevent threats = to our systems, investigate illegal or inappropriate behavior, and/or elimi= nate unsolicited promotional e-mails (spam). This process could result in d= eletion of a legitimate e-mail before it is read by its intended recipient = at our organization. Moreover, based on the scanning results, the full text= of e-mails and attachments may be made available to Weatherford security a= nd other personnel for review and appropriate action. If you have any conce= rns about this process, please contact us at dataprivacy@weatherford.com.