From owner-freebsd-stable@freebsd.org Tue Oct 6 19:37:38 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CABFE9B638C for ; Tue, 6 Oct 2015 19:37:38 +0000 (UTC) (envelope-from jim.harris@gmail.com) Received: from mail-ob0-x235.google.com (mail-ob0-x235.google.com [IPv6:2607:f8b0:4003:c01::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9856013A4 for ; Tue, 6 Oct 2015 19:37:38 +0000 (UTC) (envelope-from jim.harris@gmail.com) Received: by obbda8 with SMTP id da8so162840643obb.1 for ; Tue, 06 Oct 2015 12:37:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MC1HmZ4wVjzT27XnXkvLICs2kTU9OYjhTV8Zi/5I0aE=; b=x+MzI9abiSTgS9AQhpOXCcn4Gjop7pgkq7nq2zEQLQjA32b4Z8dnnwS4rUaeEuKhF0 saysV8vqmLIdPYQTjiF1Sdx62YqTd6Jxzlql4hw9Z9Oyy6WX8JhhxaY9x2EMBsKFkmZz J2OvM1lmXJ+Shh2dH+h52GbD2z1k473Ws6K0NTJlyIZpKvUrf/pSVE6/KoKpnCD4fW3H vqXIThyNd+IP2lOfcbw5PfHphuEvWXobwQo6wNcbYgY6vnkyOD5itdl6VGaJ3z1BLCRc 1zc0auJZzRByKr6yIsEewWwVRGLfkTJ80INGGanjvoHtVg0qS7C0m5UOlP17Q411jTu5 yb1w== MIME-Version: 1.0 X-Received: by 10.60.41.9 with SMTP id b9mr21673715oel.37.1444160258011; Tue, 06 Oct 2015 12:37:38 -0700 (PDT) Received: by 10.202.212.201 with HTTP; Tue, 6 Oct 2015 12:37:37 -0700 (PDT) In-Reply-To: <561416FB.1050802@multiplay.co.uk> References: <5613FA02.2080205@multiplay.co.uk> <561416FB.1050802@multiplay.co.uk> Date: Tue, 6 Oct 2015 12:37:37 -0700 Message-ID: Subject: Re: Dell NVMe issues From: Jim Harris To: Steven Hartland Cc: Sean Kelly , FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Oct 2015 19:37:39 -0000 On Tue, Oct 6, 2015 at 11:46 AM, Steven Hartland wrote: > On 06/10/2015 19:03, Jim Harris wrote: > > > > On Tue, Oct 6, 2015 at 9:42 AM, Steven Hartland < > killing@multiplay.co.uk> wrote: > >> Also looks like nvme exposes a timeout_period sysctl you could try >> increasing that as it could be too small for a full disk TRIM. >> > >> Under CAM SCSI da support we have a delete_max which limits the max >> single request size for a delete it may be we need something similar for >> nvme as well to prevent this as it should still be chunking the deletes to >> ensure this sort of thing doesn't happen. > > > See attached. Sean - can you try this patch with TRIM re-enabled in ZFS? > > I would be curious if TRIM passes without this patch if you increase the > timeout_period as suggested. > > -Jim > > > Interesting does the nvme spec not provide information from the device as > to what its optimal / max deallocate request size should be like the ATA > spec exposes? > Correct - there is no way for devices to specify a max/optimal deallocate size in NVMe. There is an implicit limit from the 32-bit LBA length in the DSM Range data structure defined by the spec. So this patch is needed anyways to make sure we don't overflow the 32-bit LBA length. Sean's drives are 1.6TB which fits in an unsigned 32-bit value on a 512-byte sector formatted controller, so I don't think that is the problem in Sean's case. -Jim > Regards > Steve >