From owner-freebsd-hackers@freebsd.org Thu Jun 30 14:00:52 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 659E6B86432 for ; Thu, 30 Jun 2016 14:00:52 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay05.ispgateway.de (smtprelay05.ispgateway.de [80.67.31.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2D1202116 for ; Thu, 30 Jun 2016 14:00:51 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from [78.35.169.69] (helo=fabiankeil.de) by smtprelay05.ispgateway.de with esmtpsa (TLSv1.2:AES128-GCM-SHA256:128) (Exim 4.84) (envelope-from ) id 1bIcWH-0006aD-H1 for freebsd-hackers@freebsd.org; Thu, 30 Jun 2016 16:00:41 +0200 Date: Thu, 30 Jun 2016 16:00:40 +0200 From: Fabian Keil To: freebsd-hackers@freebsd.org Subject: Re: ggatel(8) extension for binding multiple files Message-ID: <20160630160040.279fc306@fabiankeil.de> In-Reply-To: References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; boundary="Sig_/LuxbYHEM76xLGs6Q6PdZ1HH"; protocol="application/pgp-signature" X-Df-Sender: Nzc1MDY3 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 14:00:52 -0000 --Sig_/LuxbYHEM76xLGs6Q6PdZ1HH Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Julian Hsiao wrote: > I've been working on extending ggatel(8) to support binding multiple files > to one device, similar to how sparse bundles work on OS X. You could > simulate the functionality with ggatel(8) / md(4) + gconcat(8), but this > scales poorly when you have 10^5 files. I've got a working prototype that > passes some simple tests I wrote. I also tried using it as backing store > for UFS and ZFS, copied /usr/{src,obj} over, and ran make build world (wi= th > ZFS followed by zpool scrub). >=20 > To use it, instead of passing a file to ggatel(8) as the last argument, > pass a directory with files numerically named in hex (i.e. 0, 1, 2 ..., 9, > a, b, ..., e, f, 10, 11, ...). >=20 > So, I think it's a good time to gauge the community's interest in this > feature and whether it'd be possible to merge it to trunk. In what situation do you think this is preferable to simply using a single sparse file or zvol as backing store? Even using gvirstor sounds like less maintenance hassle to me. > Known issues: [...] > - Both ggatel(8) and md(4) implement BIO_DELETE by zeroing the requested > range. However, this interacts poorly with ZFS since it BIO_DELETEs > the entire device at pool creation. I know there is a ZFS sysctl to > disable the behavior, but I think the device should just not advertise > support if it had to fake it anyway, so I didn't implement it. In ElectroBSD, ggated writes zeros for BIO_DELETEs[0] and as far as I'm concerned this interacts rather well with ZFS as ZFS will convert the zeros back to BIO_DELETEs (if any compression is enabled) and thus the intended effect is achieved. I suspect that the reason why it may perform poorly with ggatel is that IIRC ggatel does not use a request queue and worker threads and thus performs poorly in general (compared to ggatec/ggated). Fabian [0] Patch 042 in: https://www.fabiankeil.de/sourcecode/electrobsd/ElectroBSD-r295083-0c7f4d6.= diff (The patch set is out of date, but the ggate[cd]-related patches should sti= ll apply.) --Sig_/LuxbYHEM76xLGs6Q6PdZ1HH Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAld1JggACgkQBYqIVf93VJ2AsQCcDsQbrwAiLzaYrbVfjCOQ/Ulq gH0An2Y2+JaCmMLfkPXivVtfI0wo6bb1 =HZon -----END PGP SIGNATURE----- --Sig_/LuxbYHEM76xLGs6Q6PdZ1HH--