From owner-freebsd-stable@freebsd.org Tue Nov 17 22:08:35 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 039DFA31768; Tue, 17 Nov 2015 22:08:35 +0000 (UTC) (envelope-from csforgeron@gmail.com) Received: from mail-io0-x236.google.com (mail-io0-x236.google.com [IPv6:2607:f8b0:4001:c06::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BC5341313; Tue, 17 Nov 2015 22:08:34 +0000 (UTC) (envelope-from csforgeron@gmail.com) Received: by ioir85 with SMTP id r85so34205812ioi.1; Tue, 17 Nov 2015 14:08:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=c6bzUkfOgqEN3rAxugbiP+LAQGCwAaGiN+dfbLw7TV4=; b=U3KyJnatjOltFVD5CjmNYqasV2uRZCO5Bix1uDHgfNMHbtD+3dQeKrSqK3gQ3vEMxn d+D9+QSEytQthLriWZ9f9yq2usuDk8pXKNnRtmw8fmja1s6oExSnnbj9DENKKsRMMfdz 46moPD8ZpYOGIqd6y++HPsYfC8+M9fgsyu0diLW3eRBtmTYhiKCiufoZ9tW6pJv9ePYz ph93c+WTufmmkiAW+Oogk2zYtvpdFvFztX7CqsOeiY2c4P2aB5QIn/xhr9Ea1DWO7SIR bdtMFnK8ZXg/hCnPLdrNEJvQEqkFZNi6LD61iEaTZrxt8Q8iPnivYEOaxzVpMqjpgJC3 a6fQ== MIME-Version: 1.0 X-Received: by 10.107.155.149 with SMTP id d143mr41223533ioe.145.1447798114108; Tue, 17 Nov 2015 14:08:34 -0800 (PST) Received: by 10.36.159.67 with HTTP; Tue, 17 Nov 2015 14:08:34 -0800 (PST) Date: Tue, 17 Nov 2015 18:08:34 -0400 Message-ID: Subject: Bug 204641 - 10.2 UNMAP/TRIM not available on a zfs zpool that uses iSCSI disks, backed on a zpool file target From: Christopher Forgeron To: FreeBSD stable , FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 22:08:35 -0000 I just submitted this as a bug: ( https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204641 ) ..but I thought I should bring it to the list's attention for more exposure - If that's a no-no, let me know, as I have a few others that are related to this that I'd like to discuss. - - - - Consider this scenario: Virtual FreeBSD Machine, with a zpool created out of iSCSI disks. Physical FreeBSD Machine, with a zpool holding a sparse file that is the target for the iSCSI disk. This setup works in an environment with all 10.1 machines, doesn't with all 10.2 machines. - The 10.2 Machines are 10.2-p7 RELEASE, updated via freebsd-update, no custom. - The 10.1 Machine are 10.1-p24 RELEASE, updated via freebsd-update, no custom. - iSCSI is all CAM iSCSI, not the old istgt platform. - The iSCSI Target is a sparse file, stored on a zpool (not a vdev Target) The target machine is the same physical machine, with the same zpools - I either boot 10.1 or 10.2 for testing, and use the same zpool/disks to ensure nothing is changing. If I have a 10.2 iSCSI Initiator (client) connected to a 10.2 iSCSI Target, TRIM doesn't work (shows as NONE below). If I have a 10.2 iSCSI Initiator (client) connected to a 10.1 iSCSI Target, TRIM does work. (There is another bug with that last scenario as well, but I will open it separately) ...for clarity, a 10.1 iSCSI Initiator connected to a 10.1 iSCSI Target also works perfectly. I have ~20 of these in the field. On the 10.1 / 10.2 Targets, the ctl.conf file is identical. Zpools are identical, because they are shared between reboots of the same iSCSI target machine. On the 10.2 initiator machine, connected to a 10.2 Target machine: # sysctl -a | grep cam.da kern.cam.da.2.minimum_cmd_size: 6 kern.cam.da.2.delete_max: 131072 kern.cam.da.2.delete_method: NONE kern.cam.da.1.error_inject: 0 kern.cam.da.1.sort_io_queue: 0 kern.cam.da.1.minimum_cmd_size: 6 kern.cam.da.1.delete_max: 131072 kern.cam.da.1.delete_method: NONE kern.cam.da.0.error_inject: 0 kern.cam.da.0.sort_io_queue: -1 kern.cam.da.0.minimum_cmd_size: 6 kern.cam.da.0.delete_max: 131072 kern.cam.da.0.delete_method: NONE Note the delete_method is NONE # sysctl -a | grep trim vfs.zfs.trim.max_interval: 1 vfs.zfs.trim.timeout: 30 vfs.zfs.trim.txg_delay: 32 vfs.zfs.trim.enabled: 1 vfs.zfs.vdev.trim_max_pending: 10000 vfs.zfs.vdev.trim_max_active: 64 vfs.zfs.vdev.trim_min_active: 1 vfs.zfs.vdev.trim_on_init: 1 kstat.zfs.misc.zio_trim.failed: 0 kstat.zfs.misc.zio_trim.unsupported: 181 kstat.zfs.misc.zio_trim.success: 0 kstat.zfs.misc.zio_trim.bytes: 0 Note no trimmed bytes. On the target machine, 10.1 and 10.2 share the same config file: /etc/ctl.conf portal-group pg0 { discovery-auth-group no-authentication listen 0.0.0.0 listen [::] } lun 0 { path /pool92/iscsi/iscsi.zvol blocksize 4K size 5T option unmap "on" option scsiname "pool92" option vendor "pool92" option insecure_tpc "on" } } target iqn.iscsi1.zvol { auth-group no-authentication portal-group pg0 lun 0 { path /pool92_1/iscsi/iscsi.zvol blocksize 4K size 5T option unmap "on" option scsiname "pool92_1" option vendor "pool92_1" option insecure_tpc "on" } } When I boot a 10.1 Target server, the 10.2 initiator connects, and we do see proper UNMAP ability: kern.cam.da.2.minimum_cmd_size: 6 kern.cam.da.2.delete_max: 5497558138880 kern.cam.da.2.delete_method: UNMAP kern.cam.da.1.error_inject: 0 kern.cam.da.1.sort_io_queue: 0 kern.cam.da.1.minimum_cmd_size: 6 kern.cam.da.1.delete_max: 5497558138880 kern.cam.da.1.delete_method: UNMAP kern.cam.da.0.error_inject: 0 kern.cam.da.0.sort_io_queue: -1 kern.cam.da.0.minimum_cmd_size: 6 kern.cam.da.0.delete_max: 131072 kern.cam.da.0.delete_method: NONE Please let me know what you'd like to know next. Thanks.