From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 10 16:58:03 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8E682B31; Fri, 10 Apr 2015 16:58:03 +0000 (UTC) Received: from mail-qk0-x229.google.com (mail-qk0-x229.google.com [IPv6:2607:f8b0:400d:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3E332392; Fri, 10 Apr 2015 16:58:03 +0000 (UTC) Received: by qkhg7 with SMTP id g7so37269846qkh.2; Fri, 10 Apr 2015 09:58:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=h08uLZVGKITrUQcopce8baambKS6hqiwFVVfetzxL8c=; b=IGNghlS7J2isTnR3xzcIBprZ1/fJi1YZgbJEgGS2tYh4RDBJAfJqj57onmskvx/3ll CNPgxfVjGHETLkDOhDXk2n5ZVnEE9hJkmp1JA7Kh+7Kbsdj9uegHxmLF0UzpUbooaKEb cm1hzMIsZSXJ4s8AMHusK6qtz3Dj8laZDFbll6Q6/KXtA/TIksbvPwdRV79lxZKxw9pD vBCX8XSxOgQ6b3WT2rdUlWVa8yKowrnESvMazgZzXTO3INHHj4UrGqYOSc6a83MyHr/t 5i0Zbi5ciovymU5CYwFq7NBmnQPC32TTjKAomNHiw62GdWsTdgV7MFX1T/3n+/xQNXCo PCzg== MIME-Version: 1.0 X-Received: by 10.55.53.137 with SMTP id c131mr4772699qka.102.1428685082463; Fri, 10 Apr 2015 09:58:02 -0700 (PDT) Received: by 10.140.38.73 with HTTP; Fri, 10 Apr 2015 09:58:02 -0700 (PDT) In-Reply-To: <5527F554.2030806@gmail.com> References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> <5526EA33.6090004@gmail.com> <5527F554.2030806@gmail.com> Date: Fri, 10 Apr 2015 09:58:02 -0700 Message-ID: Subject: Re: NVMe performance 4x slower than expected From: Jim Harris To: Tobias Oberstein Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Konstantin Belousov , "freebsd-hackers@freebsd.org" , Adrian Chadd , Michael Fuckner , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 16:58:03 -0000 On Fri, Apr 10, 2015 at 9:07 AM, Tobias Oberstein < tobias.oberstein@gmail.com> wrote: > Hi Adrian, > > > Dell has graciously loaned me a bunch of hardware to continue doing > > FWIW, Dell has a roughly comparable system: Dell R920. But they don't have > Intel NVMe's on their menu, only Samsung (and FusionIO, but that's not > NVMe). > > NUMA development on, but I have no NVMe hardware. I'm hoping people at >> > > The 8 NVMe PCIe SSDs in the box we're deploying are a key feature of this > system (will be a data-warehouse). A single NVMe probably won't have > triggered (all) issues we experienced. > > We are using the largest model (2TB), and this amounts to 50k bucks for > all eight. The smallest model (400GB) is 1.5k, so 12k in total. > > Intel can continue kicking along any desires for NUMA that they >> require. (Which they have, fwiw.) >> > > It's already awesome that Intel has senior engineers working on FreeBSD > driver code! And it would underline Intel's Open-source commitment and tech > leadership if they donated a couple of these beefy NVMes. > Intel has agreed to send DC P3700 samples to the FreeBSD Foundation to put in the cluster for this kind of work - we are working on getting these through the internal sample distribution process at the moment. -Jim