From owner-freebsd-stable@freebsd.org Mon Nov 28 18:07:02 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F296AC5AD0B for ; Mon, 28 Nov 2016 18:07:02 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A938210DE for ; Mon, 28 Nov 2016 18:07:02 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x235.google.com with SMTP id a197so213628846wmd.0 for ; Mon, 28 Nov 2016 10:07:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=W1G2RZlFL9QUGHWGZDL7uS7lYHAvj3SEcP5BA/wWxDM=; b=YylqPNwnZDOSpFfOz+pd+JI86MjFgaE3FCGzZhZr/sagAvK4vOsFYry9yg/YHoFHiw 3hFvz4OZUoswBL/z9TDiLPwQdE/EjNxUNnFcW0tzqaEMu9S1p1q0Ut30rwsc8UqqoJ7+ MVRXEfVrPSLpsz1MtLRSCSjWoH3TZYdhOEaB3XkwDNjoTtndUXNlLbaYbm+SvQOQZ0kG SBz6jW12QA1jATbsXb3ye32cXjRk+fpKAOn2ZLaHpSY9megqhVnHjPBENZmZRLHytlio CAR0RnW+1e7Mhn9gQaKuEc0V1aPAoKpLblWaeh4G/1H+2RUtE2aXhOFtcnAGVy0AXT2m +zsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=W1G2RZlFL9QUGHWGZDL7uS7lYHAvj3SEcP5BA/wWxDM=; b=H/bhgJusWvtts3CL9/O3fVoAojehq2rLJqTsE17Trdo4aBMjHtonOPEM0/ejiHlbH/ 27m3ekVKGDmobFwIDIzn9/SL7E3YHcgwD3WoQF/SlyX1J49vWQXed9zaCuMXflYh2ooz rlV/xVdJKjcda6OO/j5fTXwopgasSD8iR/nezRG05KnChuTtbe3ri5xSD/Jw0vIiHNyU O5fx4hNRCW9MctkGJyn/wqEvnVgP3N+CA/aaH2xjK/KIfQkouX5cKrDh6Mpk0lNqlErc FKrKoJkqQwvEac20nvefzwMVFp2wYPAQ9ZTHpwGNQGTofk/2563DllszTQXjWwhqCWWX l1nA== X-Gm-Message-State: AKaTC01IIZiuT7QDw/BXJMere9jq6ft/mIKVsi2chX/B9LYz6LLdsji80dFlHYsWwCAXOmKd X-Received: by 10.28.93.202 with SMTP id r193mr21888947wmb.30.1480356419645; Mon, 28 Nov 2016 10:06:59 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id di9sm63678640wjc.37.2016.11.28.10.06.58 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Nov 2016 10:06:58 -0800 (PST) Subject: Re: Sandisk CloudSpeed Gen. II Eco Channel SSD vs ZFS = we're in hell To: freebsd-stable@freebsd.org References: <345ed411-1430-50f7-6887-5ac00d259fd9@norma.perm.ru> From: Steven Hartland Message-ID: <553eff7a-0cd5-eea9-4acd-4bc770fa1baf@multiplay.co.uk> Date: Mon, 28 Nov 2016 18:07:26 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.5.0 MIME-Version: 1.0 In-Reply-To: <345ed411-1430-50f7-6887-5ac00d259fd9@norma.perm.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Nov 2016 18:07:03 -0000 Check your gstat with -dp so you also see deletes, it may be that your drives have a very slow TRIM. On 28/11/2016 17:54, Eugene M. Zheganin wrote: > Hi, > > recently we bough a bunch of "Sandisk CloudSpeed Gen. II Eco Channel" > disks (the model name by itself should already made me suspicious) for > using with zfs SAN on FreeBSD, we're plugged them into the LSI SAS3008 > and now we are experiencing the performance that I would call > "literally awful". I'm using already some of the zfs SANs on FreeBSD > with Intel/Samsung SSD drives, including the LSI SAS3008 controller, > but never saw anything like this (and yes, these are all SSDs): > > dT: 1.004s w: 1.000s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name > 75 472 78 367 104.4 12 1530 94.8 113.4| da0 > 75 475 81 482 79.2 12 1530 94.5 113.1| da1 > 69 490 96 626 106.9 12 1530 124.9 149.4| da2 > 75 400 72 382 51.5 10 1275 93.7 93.4| da3 > 0 0 0 0 0.0 0 0 0.0 0.0| da4 > 75 400 72 382 55.0 10 1275 93.9 93.7| da5 > 2 3975 3975 24020 0.3 0 0 0.0 21.0| da6 > 0 3967 3967 24144 0.3 0 0 0.0 21.4| da7 > 1 3929 3929 24259 0.3 0 0 0.0 21.6| da8 > 0 3998 3998 23933 0.3 0 0 0.0 21.2| da9 > 0 0 0 0 0.0 0 0 0.0 0.0| da10 > 0 4037 4037 23710 0.2 0 0 0.0 21.3| da11 > 0 0 0 0 0.0 0 0 0.0 0.0| da12 > 0 0 0 0 0.0 0 0 0.0 0.0| da13 > 0 0 0 0 0.0 0 0 0.0 0.0| da14 > 0 0 0 0 0.0 0 0 0.0 0.0| da15 > 0 0 0 0 0.0 0 0 0.0 0.0| da16 > > Disks are ogranized in the raidz1 pools (which is slower than the > raid1 or 10, but, considering the performance of SSDs, we got no > problems with Intel or Samsung drives), the controller is flashed with > last firmware available (identical controller with Samsung drives > performs just fine). Disks are 512e/4K drives, and "diskinfo > -v"/"camcontrol identify" both report that they have 4K > stripersize/physical sector. Pools are organized using dedicated > disks, so, considering all of the above, I don't see any possiblity to > explain this with the alignment errors. No errors are seen in the > dmesg. So, right at this time, I'm out of ideas. Everything point that > these Sandisk drives are the roort of the problem, but I don't see how > this is possible- according to the various benchmarks (taken, however, > with regular drives, not "Channel" ones, and so far I haven't figured > out what is the difference between "Channel" and non-"Channel" ones, > but they run different firmware branches) they have to be okay (or > seem so), just the ordinary SSD. > > If someone has the explanation of this awful performance, please let > me know. > > Thanks. > > Eugene. > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"