From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 7 12:39:42 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0DABBB7B; Tue, 7 Apr 2015 12:39:42 +0000 (UTC) Received: from mail-wi0-x232.google.com (mail-wi0-x232.google.com [IPv6:2a00:1450:400c:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8ACD2A5C; Tue, 7 Apr 2015 12:39:41 +0000 (UTC) Received: by wiun10 with SMTP id n10so16901990wiu.1; Tue, 07 Apr 2015 05:39:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=/pqHC5InaA4z972wjSMbvHB/vSBtG23bAJI54qCO3U8=; b=JdkspgvvlMxxDv1vUBH36izWMX2UQE52x6DRpxJEtcs+Acsr3P7kzdqce59AlzrDV3 csTYgLGAgCrZi72Ao6IlUd9PQAGlVbbgZlBgbX3DTjKuVdzVxOAwGj2cmMuo6rnXs2yj QxOqntT3/h3nS4JVS+rk2j+AthKTPJYGYXtrSkhTgyZmmb6UXIo8CurhSaAqSD3J6X0/ Fd78Iwq64a5cK4+dl3dKxBZlPhdOnOdDc63NGeZwXGZprLEij+mfNZOKipjRRuALkZ8E +iyCe6GMjPBmEFnooldDYq9CdiiLNHdCeO4gCvdWJcgjituJolwD4cfz4B/5JKw8Gmcw HgcQ== X-Received: by 10.194.158.234 with SMTP id wx10mr40301245wjb.23.1428410380143; Tue, 07 Apr 2015 05:39:40 -0700 (PDT) Received: from [192.168.1.130] (ppp-88-217-5-177.dynamic.mnet-online.de. [88.217.5.177]) by mx.google.com with ESMTPSA id it5sm10877949wid.3.2015.04.07.05.39.38 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Apr 2015 05:39:38 -0700 (PDT) Message-ID: <5523D009.7070302@gmail.com> Date: Tue, 07 Apr 2015 14:39:37 +0200 From: Tobias Oberstein User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: NVMe performance 4x slower than expected References: <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> <551C6B62.7080205@gmail.com> <551D4E5F.9090400@gmail.com> <20150402221029.GE2379@kib.kiev.ua> <551FC7B3.80203@gmail.com> <20150404234839.GM2379@kib.kiev.ua> In-Reply-To: <20150404234839.GM2379@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" , Michael Fuckner , Jim Harris , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Apr 2015 12:39:42 -0000 >>>> Something is severely wrong. It seems, there might be multiple issues >>>> (not only NVMe). And this is after already running with 3 patches to >>>> make it even boot. >>> The speed of dd already looks wrong. >> >> Yes. > Try some simple checks for the performance anomalies. E.g., verify the > raw memory bandwidth, check that it is in the expected range. Quick search > popped up the following tool, e.g.: > https://zsmith.co/bandwidth.html Here are results for those 2 machines (single socket 4 core E3, quad socket 48 core E7): https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_memperf.md https://raw.githubusercontent.com/oberstet/scratchbox/master/freebsd/cruncher/results/bw_4core.bmp https://raw.githubusercontent.com/oberstet/scratchbox/master/freebsd/cruncher/results/bw_48core.bmp