From nobody Mon Feb 7 12:15:57 2022 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id DF4B819A5EFE; Mon, 7 Feb 2022 12:16:00 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JslV860HCz3qVn; Mon, 7 Feb 2022 12:16:00 +0000 (UTC) (envelope-from avg@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1644236160; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=cd9ui52/RStpcE3HJrID3rJG8TtS+b6Yhyc6AkbB79Y=; b=JMPPBA43oEZcUPH3LrxOsVhNPj7StLTS0PYIpFrsC9Q0FPbNVg3SJIVoTLN663xASj08mu vICWbjr5SfEV2u2MZ9oDM41s9OddeuYa67iiB8qyBZah4cyiXbjnVQs8jqyxcwGpPze5F4 hmLR/FJ+a1cWvtTVS1xYANMwpEvkv6EjF/DLGgP7ditFfxYB5SRvRmLAdxCsfzYaVhTi5s 51ezkYN/+tLvQHZ4o3wi8Q4upUdyaxpY4x4M0Mftwl159zJgNzAFneAujgJPHElkE6CoKD ZkXSS6emmoMiEN6LV2ybdR5LY7xzG3ECAYsuXUZb8ANnWyBD7PyRQ7wo070hFA== Received: from [192.168.0.88] (unknown [195.64.148.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: avg/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 36FE42C328; Mon, 7 Feb 2022 12:16:00 +0000 (UTC) (envelope-from avg@FreeBSD.org) Message-ID: <346d021f-b737-f41a-883f-e821389c4431@FreeBSD.org> Date: Mon, 7 Feb 2022 14:15:57 +0200 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:91.0) Gecko/20100101 Firefox/91.0 Thunderbird/91.5.1 Content-Language: en-US To: freebsd-stable List , freebsd-fs , "freebsd-geom@FreeBSD.org" From: Andriy Gapon Subject: fsck -C -p: NO WRITE ACCESS Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1644236160; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=cd9ui52/RStpcE3HJrID3rJG8TtS+b6Yhyc6AkbB79Y=; b=cIenr7KV+C6How86wGdecdaxCrUuZ7kzaUv4Pay22mPHsXBrdo02G8JdvNQEDa+21VymPU sxtT9oJPiLwCfl6eGrSgcrKQJ2GIK+spMKdd4LrMoZinYjty7uwNfdVWgD+oM6ah+/HClJ 8ySCY7Z6zSmij1YAAAkWSY5qL20v85P75Ltz0/9oRQrK18XLJGZBVPt6TSW1YVKWYI5Afe jo+zT5cRo2JyNbvRpb22y0Kk1LTxQP8H/Um80Nc3+pQ6ByV6G1y8GLovmusg3FIpfQkf0O 1a4gQ+aQHRU5Nd1dLV7g7jtYQu5U3430ToX9R1WjxlvMZCOr8oHOQtyxfOd9Xw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1644236160; a=rsa-sha256; cv=none; b=vX2X8LVjOh4KTAAWkDa59YYg9DDKTchzIhsoHQgfuBbynxmjR2ZvNWcURVeH3OxxhlW++E 95cGcaZr11cUjGpMjk8KUxpaAoopbaCwkRUbTBB4VlypSbQXOlgTMtfR+K6fP5hGatrNhl 9gqH9aKoDNzn1SFixGfkPG9nqrkT/s6UyTBIIgyR6+XGBrDweXoIZnca7dTmXaYVa24cxR 4kxhkKB9qQNCiuZJtmVA3ktGgxhvaEiZCychS4CWwdjfcAi2WABqG3pLvk+l0UkAhXFScw fDkoCc4o8Ze0HtObjXb2jEcUXCRY1tploeN5zaKMK/CT7sZG1/w+nZyzDNX45A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N I've got a problem where fsck behaves differently from my expectations. The problem happens with a filesystem on a GELI encrypted ZVOL. The volume has 4K block size and that's the GELI's sector size as well. FreeBSD is stable/13 from mid January. Let me demonstrate: # fsck -C -p /dev/zvol/.../vault.eli /dev/zvol/.../vault.eli: NO WRITE ACCESS /dev/zvol/.../vault.eli: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY. # fsck /dev/zvol/.../vault.eli ** /dev/zvol/.../vault.eli SAVE DATA TO FIND ALTERNATE SUPERBLOCKS? [yn] n ADD CYLINDER GROUP CHECK-HASH PROTECTION? [yn] n ADD SUPERBLOCK CHECK-HASH PROTECTION? [yn] n ADD INODE CHECK-HASH PROTECTION? [yn] n ** Last Mounted on /usr/home/avg/secret ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 136 files, 371 used, 253491 free (35 frags, 31682 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** # fsck -C -p /dev/zvol/.../vault.eli /dev/zvol/.../vault.eli: NO WRITE ACCESS /dev/zvol/.../vault.eli: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY. # mount /dev/zvol/.../vault.eli ~/secret So, mount can mount the filesystem without problems. Plain fsck suggests to enable some features, it does not find any problem and sees the filesystem as clean. But fsck -C -p fails with a rather cryptic error. I don't think I saw this before. If I try to allow fsck to enable some things that it wants, it actually does fail to write: # fsck /dev/zvol/.../vault.eli ** /dev/zvol/.../vault.eli SAVE DATA TO FIND ALTERNATE SUPERBLOCKS? [yn] y CANNOT WRITE BLK: 15 CONTINUE? [yn] n With ktrace I see: fsck_ufs CALL pwrite(0x4,0x800aa0000,0x1000,0x1e00) fsck_ufs RET pwrite -1 errno 22 Invalid argument So, a 4K write but the offset is not 4K aligned. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ A little bit more data: # dumpfs -m /dev/zvol/.../vault.eli # newfs command for /dev/zvol/.../vault.eli (/dev/zvol/.../vault.eli) newfs -L vault -O 2 -a 4 -b 32768 -d 32768 -e 4096 -f 4096 -g 16384 -h 64 -i 8192 -k 2616 -m 8 -o time -s 2097144 /dev/zvol/.../vault.eli # geli list Geom name: zvol/.../vault.eli State: ACTIVE EncryptionAlgorithm: AES-XTS KeyLength: 128 Crypto: software Version: 7 UsedKey: 0 Flags: NONE KeysAllocated: 1 KeysTotal: 1 Providers: 1. Name: zvol/.../vault.eli Mediasize: 1073737728 (1.0G) Sectorsize: 4096 Mode: r0w0e0 Consumers: 1. Name: zvol/.../vault Mediasize: 1073741824 (1.0G) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r1w1e1 # tunefs -p /dev/zvol/.../vault.eli tunefs: POSIX.1e ACLs: (-a) disabled tunefs: NFSv4 ACLs: (-N) disabled tunefs: MAC multilabel: (-l) disabled tunefs: soft updates: (-n) disabled tunefs: soft update journaling: (-j) disabled tunefs: gjournal: (-J) disabled tunefs: trim: (-t) disabled tunefs: maximum blocks per file in a cylinder group: (-e) 4096 tunefs: average file size: (-f) 16384 tunefs: average number of files in a directory: (-s) 64 tunefs: minimum percentage of free space: (-m) 8% tunefs: space to hold for metadata blocks: (-k) 2616 tunefs: optimization preference: (-o) time tunefs: volume label: (-L) vault # dumpfs /dev/zvol/.../vault.eli magic 19540119 (UFS2) last mounted time Mon Feb 7 13:44:30 2022 last modified time Mon Feb 7 13:51:18 2022 superblock location 65536 id [ 581f28b7 8a03690e ] ncg 4 size 262143 blocks 253862 bsize 32768 shift 15 mask 0xffff8000 fsize 4096 shift 12 mask 0xfffff000 frag 8 shift 3 fsbtodb 3 minfree 8% optim time symlinklen 120 maxbsize 32768 maxbpg 4096 maxcontig 4 contigsumsize 4 nbfree 31682 ndir 27 nifree 130934 nffree 35 bpg 8192 fpg 65536 ipg 32768 unrefs 0 nindir 4096 inopb 128 maxfilesize 2252349704110079 sbsize 4096 cgsize 16384 csaddr 2088 cssize 4096 sblkno 24 cblkno 32 iblkno 40 dblkno 2088 cgrotor 0 fmod 0 ronly 0 clean 1 metaspace 2616 avgfpdir 64 avgfilesize 16384 flags none fsmnt /usr/home/avg/secret volname vault swuid 0 providersize 262143 cs[].cs_(nbfree,ndir,nifree,nffree): (7905,3,32748,1) (7920,17,32691,10) (7928,3,32755,21) (7929,4,32740,3) blocks in last group 8191 cg 0: magic 90255 tell 20000 time Fri Dec 17 00:25:26 2021 cgx 0 ndblk 65536 niblk 32768 initiblk 256 unrefs 0 nbfree 7905 ndir 3 nifree 32748 nffree 1 rotor 4872 irotor 19 frotor 4744 frsum 1 0 0 0 0 0 0 sum of frsum: 1 clusters 1-3: 0 0 0 clusters size 4 and over: 2 clusters free: 264-587, 611-8191 inodes used: 0-19 blks free: 2112-4703, 4751, 4888-65535 cg 1: magic 90255 tell 10020000 time Thu Oct 21 10:39:16 2021 cgx 1 ndblk 65536 niblk 32768 initiblk 256 unrefs 0 nbfree 7920 ndir 17 nifree 32691 nffree 10 rotor 4776 irotor 20 frotor 4776 frsum 5 1 1 0 0 0 0 sum of frsum: 10 clusters 1-3: 0 0 1 clusters size 4 and over: 2 clusters free: 0-2, 265-587, 598-8191 inodes used: 0-19, 21-66, 68, 70, 74-82 blks free: 0-23, 2118, 2120-4703, 4726, 4743, 4745, 4757-4758, 4764, 4781-65535 cg 2: magic 90255 tell 20020000 time Tue Dec 28 18:40:22 2021 cgx 2 ndblk 65536 niblk 32768 initiblk 256 unrefs 0 nbfree 7928 ndir 3 nifree 32755 nffree 21 rotor 4720 irotor 6 frotor 4712 frsum 0 0 0 2 0 1 1 sum of frsum: 21 clusters 1-3: 0 0 1 clusters size 4 and over: 3 clusters free: 0-2, 262-587, 590-641, 645-8191 inodes used: 0-5, 8-9, 12-14, 17, 19 blks free: 0-23, 2090-4703, 4716-5135, 5144-5147, 5153-65535 cg 3: magic 90255 tell 30020000 time Wed Sep 1 12:52:02 2021 cgx 3 ndblk 65535 niblk 32768 initiblk 256 unrefs 0 nbfree 7929 ndir 4 nifree 32740 nffree 3 rotor 4736 irotor 26 frotor 4728 frsum 0 0 1 0 0 0 0 sum of frsum: 3 clusters 1-3: 0 0 1 clusters size 4 and over: 2 clusters free: 0-2, 261-587, 592-8190 inodes used: 0-25, 27-28 blks free: 0-23, 2088-4703, 4731-4733, 4736-65527 -- Andriy Gapon