From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 10 16:07:52 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9435FFBE; Fri, 10 Apr 2015 16:07:52 +0000 (UTC) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1CE15DCE; Fri, 10 Apr 2015 16:07:52 +0000 (UTC) Received: by wiun10 with SMTP id n10so2218781wiu.1; Fri, 10 Apr 2015 09:07:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=xHxDm2NVV49XqVwEfpdGF8RFpyyEpHxV9KvangNlvfs=; b=xutWrkTnCELg8VaChEM9vFVV9G/bcRABCSSIy6OptBlcBZ9AkBaLLwaxYwEah1eKmr 7/+LjJDXgZSZYgmEG18Ex2VsWrX/M2XvYp6jI0HhYOH9xM9yrq4GV6kJqFyGC6Q/vFgE KAlFYTwYM10YYDigSqychosMVGhxi3Q/pbpujBVkmE/b4mrmK/VY4MX0uh8pddRPeXpK x+UEaETCrx7cm1Ptlgepk+O3YT2dlEKCJOGxfJmDxlTUm6Ob2SXMnWW/cX+n61kxGPVG kyaryZsaZw1zzUG78URRFSQPqd4WxheFdDhznsABGR8cvuDJiZvAl4GHsR1ZocTjm6TA V4qg== X-Received: by 10.180.102.130 with SMTP id fo2mr16445201wib.30.1428682070542; Fri, 10 Apr 2015 09:07:50 -0700 (PDT) Received: from [192.168.43.73] ([89.15.236.65]) by mx.google.com with ESMTPSA id w8sm3621220wja.4.2015.04.10.09.07.49 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 10 Apr 2015 09:07:49 -0700 (PDT) Message-ID: <5527F554.2030806@gmail.com> Date: Fri, 10 Apr 2015 18:07:48 +0200 From: Tobias Oberstein User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: NVMe performance 4x slower than expected References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> <5526EA33.6090004@gmail.com> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Konstantin Belousov , "freebsd-hackers@freebsd.org" , Michael Fuckner , Jim Harris , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 16:07:52 -0000 Hi Adrian, > Dell has graciously loaned me a bunch of hardware to continue doing FWIW, Dell has a roughly comparable system: Dell R920. But they don't have Intel NVMe's on their menu, only Samsung (and FusionIO, but that's not NVMe). > NUMA development on, but I have no NVMe hardware. I'm hoping people at The 8 NVMe PCIe SSDs in the box we're deploying are a key feature of this system (will be a data-warehouse). A single NVMe probably won't have triggered (all) issues we experienced. We are using the largest model (2TB), and this amounts to 50k bucks for all eight. The smallest model (400GB) is 1.5k, so 12k in total. > Intel can continue kicking along any desires for NUMA that they > require. (Which they have, fwiw.) It's already awesome that Intel has senior engineers working on FreeBSD driver code! And it would underline Intel's Open-source commitment and tech leadership if they donated a couple of these beefy NVMes. The specs are incredible, but extracing all the performance is non-trivial .. Cheers, /Tobias