From nobody Sat Jan 15 03:54:01 2022 X-Original-To: freebsd-questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6553F19552C3 for ; Sat, 15 Jan 2022 03:54:20 +0000 (UTC) (envelope-from tomek@cedro.info) Received: from mail-oo1-xc32.google.com (mail-oo1-xc32.google.com [IPv6:2607:f8b0:4864:20::c32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JbPRv5NLbz3DnR for ; Sat, 15 Jan 2022 03:54:19 +0000 (UTC) (envelope-from tomek@cedro.info) Received: by mail-oo1-xc32.google.com with SMTP id w15-20020a4a9d0f000000b002c5cfa80e84so3154715ooj.5 for ; Fri, 14 Jan 2022 19:54:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cedro.info; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ne2w3EPpih/8rvmYgdMwjuFxtp/+pU1pSpp/tWTaNrA=; b=JHOhmuQ7WazpnVkG5IkHrBCpyqtJUjJB+aPUR87ku9kScqQGN6ttYePXsEIy9hln4H aQ3MGmMOZ9PXEdTDX7izYm37JqArRwcFABF3vauWKcZQ41QLUXq5O6cGcQKM5GwqPyRv BsUcAvdSGS20tIFHZ5oXmFx36CAP61iaQHgBDuEflL6k38ogbnPrVmd1CLlMEublfkQt 4lptG7C64e8C1z/6qr/ubtXTm9DhkKIR9sOQs8ERSPYsx0mqTTEhmo559W9Bjy2r06ZD H04rJnMdGfOHDWJfWYaDOFCRrxe2HmxuT11DZc4fJfbQCcWn7GQEM9YDP0bs3XkDM7VI 45vQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ne2w3EPpih/8rvmYgdMwjuFxtp/+pU1pSpp/tWTaNrA=; b=ep6kfHTLm/P4a7hhjlbcJ4T4ccOWFuaynb7u6WNhcsM/tPcyRA5MRJbmsHEzE4mUse lHkJF7p36qACUr5wYxGP+tbyPQBkJamUNvu/rNhS7bk60kjHGWDStp2WInHSpZzAZ494 5F49OVG/TUEamw5Hki0Ey9W2TdRZVwlOg+akK4PalmxBexwBHhR1sFrzvTMGIT3zKqED 8EvYrkMKy30TQ7Hyz65UnTNVWQw7RTyS2COaBfAMrFmlQvtf+Ya1HXBPRqdnd2h5+tEy F9ANb8sb5CmcjY7UQsoX+x8c/OyI/bKzecJOJgNWXPB1l9PTH8N/yYgI2t+oilZOEwZl RY0A== X-Gm-Message-State: AOAM532VP21yesuPayxt7b+TbtQFsZ8QProsm/hqIn2KxJxvelfCBk12 i0qRoAaNi1ZVZ9h3tqsT3mfvnSgxVDRZuGl37NO1RvfZfh36jg== X-Google-Smtp-Source: ABdhPJzMX+PDxZrhnOsPJ/ZzMIuqJ0W9Diowu6zmXQjitQf0IiXJsecyrPrCKJ3n3wY4HtnBEZ3ZGhkv0Ob7dx1fxQ4= X-Received: by 2002:a4a:9483:: with SMTP id k3mr5785969ooi.14.1642218853513; Fri, 14 Jan 2022 19:54:13 -0800 (PST) List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 References: <77680665-7ddb-23c5-e866-05d112339b60@holgerdanske.com> <20220114023002.GP61872@eureka.lemis.com> <20220114045558.GQ61872@eureka.lemis.com> In-Reply-To: From: Tomasz CEDRO Date: Sat, 15 Jan 2022 04:54:01 +0100 Message-ID: Subject: Re: zero filling a storage device (was: dd and mbr) To: "Kevin P. Neal" Cc: "Greg 'groggy' Lehey" , David Christensen , FreeBSD Questions Mailing List Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4JbPRv5NLbz3DnR X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=cedro.info header.s=google header.b=JHOhmuQ7; dmarc=none; spf=none (mx1.freebsd.org: domain of tomek@cedro.info has no SPF policy when checking 2607:f8b0:4864:20::c32) smtp.mailfrom=tomek@cedro.info X-Spamd-Result: default: False [-3.20 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.90)[-0.902]; R_DKIM_ALLOW(-0.20)[cedro.info:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; DMARC_NA(0.00)[cedro.info]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[cedro.info:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::c32:from]; MLMMJ_DEST(0.00)[freebsd-questions]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On Sat, Jan 15, 2022 at 4:13 AM Kevin P. Neal wrote: > > On Fri, Jan 14, 2022 at 03:55:58PM +1100, Greg 'groggy' Lehey wrote: > > On Thursday, 13 January 2022 at 22:19:36 -0500, Kevin P. Neal wrote: > > > I'm not 100% that an SSD will always keep a logical block assigned > > > to a physical block. And I'm not 100% certain that an SSD won't > > > notice that all zeros are being written to a block and just optimize > > > out the write. > > > > If I understand what you mean here, you're suggesting that SSDs may > > keep a list of zeroed-out blocks and just optimize them away? It's > > possible, though I haven't heard of it. I don't know how often blocks > > are completely zeroed out, but I suspect that it wouldn't be worth > > it. If they did, it would be a good advertisement, because it would > > indirectly increase the storage capacity of the SSD. > > Yes, I believe I'm making sense to you at least. :) > > No, it would decrease wear on the drive. It would increase the life > expectancy of the drive. See below. > > > Until proof of the contrary, I'd say "no, this doesn't happen". > > Doesn't ZFS notice all-zero blocks in some cases? I have a hazy memory > of this being done when compression is turned off. > > Anyway, an SSD stores multiple blocks per page. They can only erase whole > pages, and if multiple blocks are stored on that page then those blocks > must be rewritten after the page is cleared. Then another block on the > page can be written. > > This is why SSDs are overprovisioned. They try hard to keep the number of > blocks written per-page down to avoid rewriting them when writing an > otherwise unrelated other block. > > A mapping of logical (or wire addressable if you prefer) blocks is then > needed. It would make sense to have that mapping indicate that a block is > all zeros to avoid wasting space on pages. If you ask the drive for the > contents of the block it will tell you it is all zeros. But no space in > the drive need be used for the zeros. > > This doesn't increase the usable space on the drive. Rather, it decreases > the wear on the drive. > > But I'm not an SSD firmware guy so I can't swear that all or even any > drives actually optimize this way. "It makes sense to me." :) > -- > Kevin P. Neal http://www.pobox.com/~kpn/ Using ZFS compression LZ4 may keep the repeating patterns out of the disk so only compressed stuff lands to memory.. but this happens at ZFS level not the disk level.. it also can save some disk usage I once did a dd image of a 1TB drive that was partially empty and the image only consumed around 1/3 of that space (the rest was marked as free). LZ4 is amazingly fast and efficient :-) Another thing is that there is no guarantee what happens on the plate because disk firmware can do all sorts of the trick on-the-fly between plate and the interface.. I guess this is not very different between HDD and SSD.. except when you buy a "RED" class of disks that are supposed to only perform raw read/write nothing in the middle. I recently noticed that RED disks are 1/3 faster than GREEN disks, see numbers below :-) Here is a work in progress article on how to use M2.NVM drive with PCI-E 4.0 controller on older motherboard with PCI-E 2.0 that only provides onboard SATA :-) The funny thing is that BIOS does not see neither controller nor the disk so it is not available for direct boot, but running boot and kernel from SATA HDD allows it to use NVM SSD as ZFS / with no additional drivers, NVM/UEFI firmwares, etc :-) https://www.tomek.cedro.info/m2-nvm-disk-pci-e-controller-on-freebsd/ -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info