From owner-freebsd-stable@freebsd.org Wed Nov 18 00:28:52 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E937BA32684 for ; Wed, 18 Nov 2015 00:28:51 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F0711F7C for ; Wed, 18 Nov 2015 00:28:51 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by wmvv187 with SMTP id v187so254166211wmv.1 for ; Tue, 17 Nov 2015 16:28:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay_co_uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:cc:message-id:date:user-agent :mime-version:in-reply-to:content-type:content-transfer-encoding; bh=WrfcV9A5j8hoTxH7Ej4tM+CK8mNQw48+UwsY+frvIIk=; b=AhgLLBViXd11l9UFeIG80QfjfJhrhOaWNlckWz9nR+ZVCu1ddcyO8x7ZmwbZadSSSQ fCmPyeb215zj4wVOpl4TRBdtWdSMbLLuKivT/4wcrnYvnTIoHkjk+mb6lfxoTRWMtW3M 5fIcyM9Tj+5kEOkZS2ZCPv+hZ58YVNE3/F2LGLX/kkvA14VzNuPEm83UCvxZcv0CcxkW ejYlFvqv0UJe6LeqXcs44QFGOPa+A9q03L1FjyvWgvrD4iOQC9W2JAa6jEww988EOHT3 s8QvBXQMJqqFY1xNOlRzrBSLGuMObKk76WzDblaic/apA7nt4Epil1cGsI53mCl4DeZh HVyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:cc:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=WrfcV9A5j8hoTxH7Ej4tM+CK8mNQw48+UwsY+frvIIk=; b=TEFQen1hRd4CPD6GG0TJTuyFKTiSBBCmaxDX97W4fxaVV+bTwVawAr7DAzN5GPziWE nF/vw4Ww38YcpNEwlHNrVJ970kNrETIIvcBIsHNy/6ONKZn0QzUwMCLlK8/8J6UNF59M x26osJ2LeoZIwST8fVfEbvhyjZC1i/HwseSMsMIg9rcc8nrgL+Odbd0c5X/dPQ9XzU88 INjy3tEb6pxUmvBMy9FWss9UfBKDmjDtZ+dNEwnvoIO5KCz8L2H3XAjBsOYx+yUaCAeb mcHDMTJjnKuJMWFMI7rt1V5l2LnVNuY44bycpugzw9t3DiuzI74RJeGlUbe4X64SaoVt KIeQ== X-Gm-Message-State: ALoCoQmCvjDYnMOpt5r60y1LWz5FZeEW8ugRz/owJ/RYUqtCIpPQg7Rw9rXjbkCo8eUZIL8FX68C X-Received: by 10.194.189.133 with SMTP id gi5mr46563691wjc.13.1447806528683; Tue, 17 Nov 2015 16:28:48 -0800 (PST) Received: from [10.10.1.58] (liv3d.labs.multiplay.co.uk. [82.69.141.171]) by smtp.gmail.com with ESMTPSA id v192sm26418214wmv.5.2015.11.17.16.28.47 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 17 Nov 2015 16:28:47 -0800 (PST) Subject: Re: Bug 204641 - 10.2 UNMAP/TRIM not available on a zfs zpool that uses iSCSI disks, backed on a zpool file target To: freebsd-stable@freebsd.org References: From: Steven Hartland Cc: Alexander Motin Message-ID: <564BC646.5070708@multiplay.co.uk> Date: Wed, 18 Nov 2015 00:28:54 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 00:28:52 -0000 On 17/11/2015 22:08, Christopher Forgeron wrote: > I just submitted this as a bug: > > ( https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204641 ) > > ..but I thought I should bring it to the list's attention for more exposure > - If that's a no-no, let me know, as I have a few others that are related > to this that I'd like to discuss. > > - - - - > > > Consider this scenario: > > Virtual FreeBSD Machine, with a zpool created out of iSCSI disks. > Physical FreeBSD Machine, with a zpool holding a sparse file that is the > target for the iSCSI disk. > > This setup works in an environment with all 10.1 machines, doesn't with all > 10.2 machines. > > - The 10.2 Machines are 10.2-p7 RELEASE, updated via freebsd-update, no > custom. > - The 10.1 Machine are 10.1-p24 RELEASE, updated via freebsd-update, no > custom. > - iSCSI is all CAM iSCSI, not the old istgt platform. > - The iSCSI Target is a sparse file, stored on a zpool (not a vdev Target) > > The target machine is the same physical machine, with the same zpools - I > either boot 10.1 or 10.2 for testing, and use the same zpool/disks > > to ensure nothing is changing. > > If I have a 10.2 iSCSI Initiator (client) connected to a 10.2 iSCSI Target, > TRIM doesn't work (shows as NONE below). > If I have a 10.2 iSCSI Initiator (client) connected to a 10.1 iSCSI Target, > TRIM does work. > > (There is another bug with that last scenario as well, but I will open it > separately) > > ...for clarity, a 10.1 iSCSI Initiator connected to a 10.1 iSCSI Target > also works perfectly. I have ~20 of these in the field. > > On the 10.1 / 10.2 Targets, the ctl.conf file is identical. Zpools are > identical, because they are shared between reboots of the same iSCSI > > target machine. > > > > On the 10.2 initiator machine, connected to a 10.2 Target machine: > > # sysctl -a | grep cam.da > > kern.cam.da.2.minimum_cmd_size: 6 > kern.cam.da.2.delete_max: 131072 > kern.cam.da.2.delete_method: NONE > kern.cam.da.1.error_inject: 0 > kern.cam.da.1.sort_io_queue: 0 > kern.cam.da.1.minimum_cmd_size: 6 > kern.cam.da.1.delete_max: 131072 > kern.cam.da.1.delete_method: NONE > kern.cam.da.0.error_inject: 0 > kern.cam.da.0.sort_io_queue: -1 > kern.cam.da.0.minimum_cmd_size: 6 > kern.cam.da.0.delete_max: 131072 > kern.cam.da.0.delete_method: NONE > > Note the delete_method is NONE > > > # sysctl -a | grep trim > vfs.zfs.trim.max_interval: 1 > vfs.zfs.trim.timeout: 30 > vfs.zfs.trim.txg_delay: 32 > vfs.zfs.trim.enabled: 1 > vfs.zfs.vdev.trim_max_pending: 10000 > vfs.zfs.vdev.trim_max_active: 64 > vfs.zfs.vdev.trim_min_active: 1 > vfs.zfs.vdev.trim_on_init: 1 > kstat.zfs.misc.zio_trim.failed: 0 > kstat.zfs.misc.zio_trim.unsupported: 181 > kstat.zfs.misc.zio_trim.success: 0 > kstat.zfs.misc.zio_trim.bytes: 0 > > Note no trimmed bytes. > > > On the target machine, 10.1 and 10.2 share the same config file: > /etc/ctl.conf > > portal-group pg0 { > discovery-auth-group no-authentication > listen 0.0.0.0 > listen [::] > } > > lun 0 { > path /pool92/iscsi/iscsi.zvol > blocksize 4K > size 5T > option unmap "on" > option scsiname "pool92" > option vendor "pool92" > option insecure_tpc "on" > } > } > > > target iqn.iscsi1.zvol { > auth-group no-authentication > portal-group pg0 > > lun 0 { > path /pool92_1/iscsi/iscsi.zvol > blocksize 4K > size 5T > option unmap "on" > option scsiname "pool92_1" > option vendor "pool92_1" > option insecure_tpc "on" > } > } > > > When I boot a 10.1 Target server, the 10.2 initiator connects, and we do > see proper UNMAP ability: > > > kern.cam.da.2.minimum_cmd_size: 6 > kern.cam.da.2.delete_max: 5497558138880 > kern.cam.da.2.delete_method: UNMAP > kern.cam.da.1.error_inject: 0 > kern.cam.da.1.sort_io_queue: 0 > kern.cam.da.1.minimum_cmd_size: 6 > kern.cam.da.1.delete_max: 5497558138880 > kern.cam.da.1.delete_method: UNMAP > kern.cam.da.0.error_inject: 0 > kern.cam.da.0.sort_io_queue: -1 > kern.cam.da.0.minimum_cmd_size: 6 > kern.cam.da.0.delete_max: 131072 > kern.cam.da.0.delete_method: NONE > > > Please let me know what you'd like to know next. > Having a quick flick through the code it looks like umap is now only supported on dev backed and not file backed. I believe the following commit is the cause: https://svnweb.freebsd.org/base?view=revision&revision=279005 This was an MFC of: https://svnweb.freebsd.org/base?view=revision&revision=278672 I'm guessing this was an unintentional side effect mav? Regards Steve