From owner-freebsd-geom@FreeBSD.ORG Thu Mar 12 18:25:33 2015 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 52988D1D for ; Thu, 12 Mar 2015 18:25:33 +0000 (UTC) Received: from thyme.infocus-llc.com (thyme.infocus-llc.com [199.15.120.10]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1BDAC3BA for ; Thu, 12 Mar 2015 18:25:32 +0000 (UTC) Received: from draco.over-yonder.net (c-75-65-60-66.hsd1.ms.comcast.net [75.65.60.66]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by thyme.infocus-llc.com (Postfix) with ESMTPSA id 3902337B66E for ; Thu, 12 Mar 2015 13:25:31 -0500 (CDT) Received: by draco.over-yonder.net (Postfix, from userid 100) id 3l2z9Q40X0z1fK; Thu, 12 Mar 2015 13:25:30 -0500 (CDT) Date: Thu, 12 Mar 2015 13:25:30 -0500 From: "Matthew D. Fuller" To: freebsd-geom@freebsd.org Subject: Re: RFC: Pass TRIM through GELI Message-ID: <20150312182530.GF24274@over-yonder.net> References: <20150308000131.GP1742@over-yonder.net> <7e0bacf0.308742dc@fabiankeil.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <7e0bacf0.308742dc@fabiankeil.de> X-Editor: vi X-OS: FreeBSD User-Agent: Mutt/1.5.23-fullermd.4 (2014-03-12) X-Virus-Scanned: clamav-milter 0.98.6 at thyme.infocus-llc.com X-Virus-Status: Clean X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Mar 2015 18:25:33 -0000 On Thu, Mar 12, 2015 at 12:02:40PM +0100 I heard the voice of Fabian Keil, and lo! it spake thus: > > I tested it with ZFS below geli and the result was newfs hanging > "unkillable" after sending the first delete request (according to > the patched gnop): That is interesting. gstat seems to show an outstanding request down to the zvol, but not beyond: L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 0 0 0 0.0 0 0 0.0 0.0| ada0s1 1 0 0 0 0.0 0 0 0.0 0.0| zvol/t/tvol 1 0 0 0 0.0 0 0 0.0 0.0| zvol/t/tvol.eli Pool 't' created on ada0s1, zvol 'tvol' created, 'onetime -e NULL -t' geli on top of that, newfs -E'd. In this case, ada0 wouldn't support TRIM anyway, but I get identical results doing it on a md(4) that would. Doesn't seem to happen the other way around (zvol on zpool on .eli). Or, in a handful of tests, on anything I can make not a zvol... Little investigation attempt: if I set kern.geom.debugflags=2 and do a newfs on the zvol/t/tvol (if I'm reading this right), the BIO_DELETE goes in and comes right out: Mar 12 12:32:49 netboot-current kernel: bio_request(0xfffff80004d42ba0) from 0xfffff80004cff400(zvol/t/tvol) to 0xfffff800041fc300(zvol/t/tvol) cmd 4 Mar 12 12:32:49 netboot-current kernel: g_io_deliver(0xfffff80004d42ba0) from 0xfffff80004cff400(zvol/t/tvol) to 0xfffff800041fc300(zvol/t/tvol) cmd 4 error 0 off 65536 len 104792064 (then there's a read, and a buncha writes) On the zvol/t/tvol.eli, though, the BIO_DELETE goes through the tvol.eli to the tvol, and then... does nothing: Mar 12 12:33:24 netboot-current kernel: bio_request(0xfffff800041991f0) from 0xfffff800491a7e80(zvol/t/tvol.eli) to 0xfffff800041fb600(zvol/t/tvol.eli) cmd 4 Mar 12 12:33:24 netboot-current kernel: bio_request(0xfffff80004d42ba0) from 0xfffff800491a7f00(zvol/t/tvol.eli) to 0xfffff800041fc300(zvol/t/tvol) cmd 4 (nothing happens for 5 seconds, and then there're some writes between zfs::vdev and ada0s1, which I presume are the flushes of the writes that happen before the _DELETE is attempted. No further 'cmd 4's show up in the log, at least for the couple minutes 'till I rebooted.) What if we build a .nop on the tvol? Mar 12 12:47:30 netboot-current kernel: bio_request(0xfffff800041c0c98) from 0xfffff80002ab0780(zvol/t/tvol.nop) to 0xfffff8004910fe00(zvol/t/tvol.nop) cmd 4 Mar 12 12:47:30 netboot-current kernel: bio_request(0xfffff800041b36c8) from 0xfffff80002ab0800(zvol/t/tvol.nop) to 0xfffff80002862100(zvol/t/tvol) cmd 4 [ ... couple lines of other requests elided ... ] Mar 12 12:47:31 netboot-current kernel: g_io_deliver(0xfffff800041b36c8) from 0xfffff80002ab0800(zvol/t/tvol.nop) to 0xfffff80002862100(zvol/t/tvol) cmd 4 error 0 off 65536 len 104792064 Mar 12 12:47:31 netboot-current kernel: g_io_deliver(0xfffff800041c0c98) from 0xfffff80002ab0780(zvol/t/tvol.nop) to 0xfffff8004910fe00(zvol/t/tvol.nop) cmd 4 error 0 off 65536 len 104792064 Well, so that starts out just the same as the .eli case, but it gets a response back. .eli on a md? Clean too. Mar 12 13:05:28 netboot-current kernel: bio_request(0xfffff800041b35d0) from 0xfffff800672fce80(md2s1.eli) to 0xfffff8004910f000(md2s1.eli) cmd 4 Mar 12 13:05:28 netboot-current kernel: bio_request(0xfffff800041c17c0) from 0xfffff800672fce00(md2s1.eli) to 0xfffff8004910fe00(md2s1) cmd 4 Mar 12 13:05:28 netboot-current kernel: g_part_start: cmd=4, provider=md2s1 Mar 12 13:05:28 netboot-current kernel: bio_request(0xfffff80004c8eba0) from 0xfffff80002886e80(md2) to 0xfffff800044e1600(md2) cmd 4 Mar 12 13:05:31 netboot-current kernel: g_io_deliver(0xfffff80004c8eba0) from 0xfffff80002886e80(md2) to 0xfffff800044e1600(md2) cmd 4 error 0 off 90112 len 104747008 Mar 12 13:05:31 netboot-current kernel: g_io_deliver(0xfffff800041c17c0) from 0xfffff800672fce00(md2s1.eli) to 0xfffff8004910fe00(md2s1) cmd 4 error 0 off 65536 len 104747008 Mar 12 13:05:31 netboot-current kernel: g_io_deliver(0xfffff800041b35d0) from 0xfffff800672fce80(md2s1.eli) to 0xfffff8004910f000(md2s1.eli) cmd 4 error 0 off 65536 len 104747008 Not telling me much... -- Matthew Fuller (MF4839) | fullermd@over-yonder.net Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream.