Date: Fri, 3 Sep 2021 13:30:38 -0700 From: David Christensen <dpchrist@holgerdanske.com> To: freebsd-questions@freebsd.org Subject: Re: FreeBSD 12.2-RELEASE-p9 trim: open failed: /dev/ada0: Operation not permitted Message-ID: <64099455-5526-beb5-8ead-6ce8a9d073e6@holgerdanske.com> In-Reply-To: <47d0f250-a47c-bbe0-2875-4f42f4464fcc@chezmarcotte.ca> References: <dda0a570-53bc-7676-9f38-67bcd7a6f8bb@holgerdanske.com> <47d0f250-a47c-bbe0-2875-4f42f4464fcc@chezmarcotte.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
On 9/3/21 10:09 AM, Derek (freebsd lists) wrote:
> On 2021-08-29 10:26 p.m., David Christensen wrote:
>> I would like to trim the SSD, but trim(8) is not happy:
>>
>> 2021-08-29 19:22:36 toor@f1 ~
>> # trim -f /dev/ada0
>> trim /dev/ada0 offset 0 length 60022480896
>> trim: open failed: /dev/ada0: Operation not permitted
>>
>
> I don't know if it's still relevant to the thread, but I feel like what
> you are encountering here relates to what you are trying to trim.
>
> It feels like if you trim the "raw device", the system wouldn't know
> what is in use, and what isn't.
>
> I feel like you should be trying to trim a filesystem on the device,
> rather than the device itself, as the utility would be able to determine
> what is currently allocated with the filesystem data.
>
> It might be that a raw device could be trimmed, but perhaps there's some
> safety there to stop it from wiping a device that is backing a mounted
> filesystem.
>
> Something to look at?
>
> Derek
That is a good point -- the filesystem knows what blocks are in use,
what blocks are not in use, and when blocks are removed from use.
Somehow, trim(8) gets this information (from the kernel filesystem stack?).
RTFM trim(8):
SYNOPSIS
trim [-Nfqv] [-[lo] offset[K|k|M|m|G|g|T|t]] [-r rfile] device ...
<snip>
-l offset[K|k|M|m|G|g|T|t]
-o offset[K|k|M|m|G|g|T|t]
Specify the length -l of the region to trim or its offset
-o from
the beginning of the device. The whole device is erased by
default unless one or both of these options are presented.
<snip>
Note that actual success of the operation depends of underlying device
driver such as ada(4), da(4) and others. Refer to corresponding
manual
pages for detail on possible caveats in low level support for ATA
TRIM or
SCSI UNMAP commands.
Note:
1. The command line argument is *device*.
2. The descriptions for the -l and -o options state trim(8) erases the
whole *device*.
Here is another console session:
2021-09-03 12:07:27 toor@f2 ~
# freebsd-version ; uname -a
12.2-RELEASE-p10
FreeBSD f2.tracy.holgerdanske.com 12.2-RELEASE-p7 FreeBSD
12.2-RELEASE-p7 GENERIC amd64
2021-09-03 12:10:06 toor@f2 ~
# kldstat | egrep 'Id|zfs'
Id Refs Address Size Name
2 1 0xffffffff8247c000 3bad38 zfs.ko
2021-09-03 12:11:54 toor@f2 ~
# find /boot -iname '*zfs*'
2021-09-03 12:12:18 toor@f2 ~
# geom disk list ada0
Geom name: ada0
Providers:
1. Name: ada0
Mediasize: 60022480896 (56G)
Sectorsize: 512
Mode: r3w3e9
descr: INTEL SSDSC2CW060A3
lunid: ****************
ident: ******************
rotationrate: 0
fwsectors: 63
fwheads: 16
2021-09-03 12:13:14 toor@f2 ~
# geom part show -p
=> 63 117231345 ada0 MBR (56G)
63 1985 - free - (993K)
2048 29360128 ada0s1 freebsd [active] (14G)
29362176 87867392 ada0s2 freebsd (42G)
117229568 1840 - free - (920K)
=> 0 29360128 ada0s1 BSD (14G)
0 4194304 ada0s1a freebsd-zfs (2.0G)
4194304 4194304 ada0s1b freebsd-swap (2.0G)
8388608 20971520 ada0s1d freebsd-zfs (10G)
2021-09-03 12:15:28 toor@f2 ~
# geli status
Name Status Components
ada0s1d.eli ACTIVE ada0s1d
mirror/swap.eli ACTIVE mirror/swap
2021-09-03 12:16:41 toor@f2 ~
# zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP
HEALTH ALTROOT
bootpool 1.88G 475M 1.41G - - 3% 24% 1.08x
ONLINE -
f2_zroot 9.50G 5.88G 3.62G - - 23% 61% 1.21x
ONLINE -
2021-09-03 12:17:34 toor@f2 ~
# zpool status
pool: bootpool
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: scrub repaired 0 in 0 days 00:00:02 with 0 errors on Sat May 29
16:18:50 2021
config:
NAME STATE READ WRITE CKSUM
bootpool ONLINE 0 0 0
ada0s1a ONLINE 0 0 0
errors: No known data errors
pool: f2_zroot
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: scrub repaired 0 in 0 days 00:00:23 with 0 errors on Sat May 29
16:19:12 2021
config:
NAME STATE READ WRITE CKSUM
f2_zroot ONLINE 0 0 0
ada0s1d.eli ONLINE 0 0 0
errors: No known data errors
2021-09-03 12:18:46 toor@f2 ~
# zfs get mountpoint bootpool f2_zroot
NAME PROPERTY VALUE SOURCE
bootpool mountpoint /bootpool local
f2_zroot mountpoint /f2_zroot local
2021-09-03 12:19:06 toor@f2 ~
# trim -f -v /bootpool
trim /bootpool offset 0 length 3
trim: open failed: /bootpool: Is a directory
2021-09-03 12:20:00 toor@f2 ~
# trim -f -v /f2_zroot
trim /f2_zroot offset 0 length 2
trim: open failed: /f2_zroot: Is a directory
2021-09-03 12:21:49 toor@f2 ~
# trim -f -v ada0
trim ada0 offset 0 length 60022480896
trim: open failed: ada0: Operation not permitted
2021-09-03 12:22:52 toor@f2 ~
# trim -f -v ada0s1
trim ada0s1 offset 0 length 15032385536
trim: open failed: ada0s1: Operation not permitted
2021-09-03 12:23:55 toor@f2 ~
# trim -f -v ada0s1a
trim ada0s1a offset 0 length 2147483648
trim: open failed: ada0s1a: Operation not permitted
2021-09-03 12:24:57 toor@f2 ~
# trim -f -v ada0s1b
trim ada0s1b offset 0 length 2147483648
trim: open failed: ada0s1b: Operation not permitted
2021-09-03 12:25:58 toor@f2 ~
# trim -f -v ada0s1d
trim ada0s1d offset 0 length 10737418240
trim: open failed: ada0s1d: Operation not permitted
2021-09-03 12:26:57 toor@f2 ~
# man 4 ada | grep -i trim
2021-09-03 12:27:35 toor@f2 ~
# man 4 ada | grep -i unmap
2021-09-03 13:24:51 toor@f2 ~
# man 8 zpool | grep -i trim
2021-09-03 13:24:54 toor@f2 ~
# man 8 zpool | grep -i unmap
2021-09-03 13:24:55 toor@f2 ~
# man 8 zfs | grep -i trim
2021-09-03 13:25:07 toor@f2 ~
# man 8 zfs | grep -i unmap
My WAG:
1. trim(8) is not integrated with ZFS.
2. ZFS does not provide a TRIM configuration settings or userland tools.
3. Given the disk was zeroed before installation and given the
monotonically increasing sizes of the 'gzip -1' compressed raw images I
take monthly, TRIM is not implemented by ZFS.
David
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?64099455-5526-beb5-8ead-6ce8a9d073e6>