From owner-freebsd-arm@FreeBSD.ORG Sat May 31 17:59:29 2014 Return-Path: Delivered-To: freebsd-arm@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0B800B02 for ; Sat, 31 May 2014 17:59:29 +0000 (UTC) Received: from mail-pd0-f170.google.com (mail-pd0-f170.google.com [209.85.192.170]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CC5322DDB for ; Sat, 31 May 2014 17:59:28 +0000 (UTC) Received: by mail-pd0-f170.google.com with SMTP id g10so2043292pdj.1 for ; Sat, 31 May 2014 10:59:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=zdMEBP1Cb4HkYCWM0bSDVWOPfdPIkGYjeoJ+L8uPNKQ=; b=XpZ2NWaRDCRLutnkXjHoH3DQdGgU30WagQ0BYGf7eppanwmji8a4xwxkRFP6VTX5WS Gql+jj+lXWk1fKwoB6epgqzghuhbKiTefs1zy4lJJB4ZEHXm3mViHM04wjlSV2AFts2V RliZ//FKHcO504SNmL0DGAUHOffhxey3Z7Br/0klSjbNYtr0m3hmm5n8oy1kOLk28tCq ctoXSlJSwX5WxLC+1ImjgwsVbAXBnYfshkdlhmRNbvlVCxwXbFfzXBg8Wr4rrLQUP77c IWKpEM8idTJHlb+Lw79g40A5AbUiGSxStgG8/Ra1XaaRWIc/cTWuoWEyRGhg9VyCY6fO bSmw== X-Gm-Message-State: ALoCoQn+2dpU2hVCoaXz7PsFFDy9GQlUL1/im/fYG1S3UnxFxe0y85g3BqpFP1vomc6l4MG207fD X-Received: by 10.68.194.134 with SMTP id hw6mr28304640pbc.49.1401559161882; Sat, 31 May 2014 10:59:21 -0700 (PDT) Received: from lgmac-rtangirala.corp.netflix.com (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id ox3sm12262610pbb.88.2014.05.31.10.59.20 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 31 May 2014 10:59:21 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_3D186CE6-EEDF-4711-ABBB-18836371C086"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: TRIM on SD cards From: Warner Losh In-Reply-To: <1401558499.20883.44.camel@revolution.hippie.lan> Date: Sat, 31 May 2014 11:59:22 -0600 Message-Id: <5B622E08-6BAE-4A39-B04B-B80F40D1C692@bsdimp.com> References: <20140531004306.GI26883@cicely7.cicely.de> <1401505209.20883.34.camel@revolution.hippie.lan> <20140531102305.GK26883@cicely7.cicely.de> <05005B04-1BDA-4242-946B-28D0DA069A42@bsdimp.com> <1401558499.20883.44.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1878.2) Cc: freebsd-arm@FreeBSD.org, Bernd Walter , ticso@cicely.de X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 May 2014 17:59:29 -0000 --Apple-Mail=_3D186CE6-EEDF-4711-ABBB-18836371C086 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1251 On May 31, 2014, at 11:48 AM, Ian Lepore wrote: > On Sat, 2014-05-31 at 10:45 -0600, Warner Losh wrote: >> On May 31, 2014, at 4:23 AM, Bernd Walter = wrote: >>=20 >>> On Fri, May 30, 2014 at 09:00:09PM -0600, Ian Lepore wrote: >>>> On Sat, 2014-05-31 at 02:43 +0200, Bernd Walter wrote: >>>>> It seems SD cards support a delete command, which FreeBSD supports >>>>> with the mmcsd driver. >>>>> newfs and tunefs support TRIM in that new filesystems are trim'ed >>>>> and the filesystems automatically trim free'ed blocks. >>>>> So far so good. >>>>> On the practical side with SD based ARM you don't write = filesystems >>>>> directly via mmcsd. >>>>> We either create an image, which id dd'ed onto SD or in some cases >>>>> we use an USB SD drive. >>>>> With dd the unused blocks are written as well, which effectively >>>>> hurts by writing data. >>>>> Is there some kind of dd, which actually don't write zero blocks, >>>>> or even better does a trim call for them? >>>>=20 >>>> I don't think dd can safely do that. If it finds a block of zeroes = on >>>> the input side, how does it know it's okay to do a DELETE for those >>>> (which sets the block to all-bits-on on most flash media). Maybe = it's >>>> important for that data to really be zero; dd doesn't know. >>>=20 >>> That 1 bit thing is true for raw flash media. >>> A amanged NAND flash device simply unmaps a logical block from = physical >>> storage. >>> This is similar to having holes in an ufs file, which per definition >>> returns zero when reading a hole range. >>> However from reading the thread it is not save to assume managed = flash >>> devices return zero blocks when reading TRIM'ed space. >>=20 >> One of the things that I did for images years ago was compressed tar = files. There was so much variation between CF makers and at the time CF = geometry was important to the BIOS, so we made our images as tar balls. = We then had a makefile target that would create a partition on the card = that was actually there, put boot blocks on it then extract the tarball=85= I never have liked DD for creating images, even when LBAs ruled the = day because you=92d always have to grow/shrink the FS afterwards. The = only advantage it had was it was easy=85 Perhaps it is time to go back = to that model? The alternative that wouldn=92t suck too bad would be to = create variable sized images based on how much data was actually present = and ensure there are no holes (or minimal holes) in the filesystem. >>=20 >> Hmmm, if we know WHAT filesystem we=92re dealing with, then we could = perhaps enhance fsck and/or growfs to BIO_DELETE all the blocks that it = knows are free, which would be a useful, data-driven approach that could = ensure we start out with a nicely trimmed FS. Given the vagaries of the = different kinds of TRIMs and the various translation layers we have, = that might be the most robust. >>=20 > I wouldn't at all mind an fsck or growfs option or a standalone tool > that would issue deletes for unused blocks on an existing filesystem. = I > have several big SSDs on this system and I had no TRIM support for a > long time, so there are lots of blocks on the drives that bogusly look > used to the onboard block management code. This is precisely why I=92d like this=85 There are also times one might = want to turn TRIM off if you are going to do a huge delete, followed by = adding lots of data. It might be faster to just trim what you didn=92t = refill=85 Or you might want to have a best-effort time strategy if you = know your FS can cope. Then if you get too many, you can discard them = and cleanup after the fact on the next boot. The number of use cases = here is actually rather large in addition to the one that tisco wants :) Warner --Apple-Mail=_3D186CE6-EEDF-4711-ABBB-18836371C086 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTihh6AAoJEGwc0Sh9sBEAiEMQALurYPzFHXuVPt1ATik0kzoi TO3gm9za9a8cqAGKk4rQVF0oo37wOcH5zuh5JmJaK0oHO/RyF7S97rrNANiFZp49 xjKWOW3HqzlQpGkLD2e2AYdeCIXHXyxRJuSluXT55BJyU5CEeq+Sh6B/wo5TnNTo 5YYtPZKnb31TUms5zAoDYjBZniYgBvWv17hb0EZ653U3sB8m5gjUuqMg3/s67B6J TEetOMMB1UaAnGPzKU5KTFVikh2ioPI39iwXlPw+2wertyZXoDy+UNAMK+YwBdLy VLB3UHbXeIyfUqfh2WYaADRd2RQs2wlgBI3rq/wFb8miVEa5Y+6W1LH8EWI33oKj +Eonleo7iNj5WKSVUshNw1QJivTJVFRkjdUpqDjjThZi/AwEABdDWqwqDQNef3MO Q+7Zf/qEbpZ1Zk5oZeqrToqJYQsHv2KSXiRwVQTkSaLkDIy1fx/AqhRgwIgZUdPt j9VRDNkbdriZHEbetroOmchd3nEITNNzkcS2TbmOOEJZbSa1x2IhKyKDkASFCscr vB91a7+kRGgcvPM/APs321J7Lo6FR4yiaVFaQ3ZLX5/zuE7c15WX1AKH9XWn5CZO S4mtpIcVKAr+x+xfYPpbCTU4PSpoJJd43Rv7ItCVA5S5wdiwkKDoJ0CxpRwITyOu chQFScl4lAkcrtlHvX8a =88FI -----END PGP SIGNATURE----- --Apple-Mail=_3D186CE6-EEDF-4711-ABBB-18836371C086--