From owner-freebsd-fs@freebsd.org Wed Oct 14 16:55:16 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EF7E643E2FB for ; Wed, 14 Oct 2020 16:55:16 +0000 (UTC) (envelope-from se@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CBJSN63dkz4Frw; Wed, 14 Oct 2020 16:55:16 +0000 (UTC) (envelope-from se@freebsd.org) Received: from Stefans-MBP-WLAN.fritz.box (p200300cd5f0bbc007cc4f02f434627f0.dip0.t-ipconnect.de [IPv6:2003:cd:5f0b:bc00:7cc4:f02f:4346:27f0]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4D40621E5F; Wed, 14 Oct 2020 16:55:16 +0000 (UTC) (envelope-from se@freebsd.org) To: Nikita Grechikhin References: <1720721602692234@mail.yandex.ru> From: Stefan Esser Cc: "freebsd-fs@freebsd.org" Subject: Re: fsck on big UFS filesystems (>100TB) Message-ID: <1998de69-b381-4258-2654-47fd439c402c@freebsd.org> Date: Wed, 14 Oct 2020 18:55:11 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: <1720721602692234@mail.yandex.ru> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="CgrS5hA1IFED7gHU3rE4aIJ2RrCPFWJNZ" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2020 16:55:17 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --CgrS5hA1IFED7gHU3rE4aIJ2RrCPFWJNZ Content-Type: multipart/mixed; boundary="asO6qngYmXXwN2XkiOWjTKLToCBKpDr54"; protected-headers="v1" From: Stefan Esser To: Nikita Grechikhin Cc: "freebsd-fs@freebsd.org" Message-ID: <1998de69-b381-4258-2654-47fd439c402c@freebsd.org> Subject: Re: fsck on big UFS filesystems (>100TB) References: <1720721602692234@mail.yandex.ru> In-Reply-To: <1720721602692234@mail.yandex.ru> --asO6qngYmXXwN2XkiOWjTKLToCBKpDr54 Content-Type: multipart/mixed; boundary="------------7DBCB3634E8ECB88FE3E57A5" Content-Language: de-DE This is a multi-part message in MIME format. --------------7DBCB3634E8ECB88FE3E57A5 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable Am 14.10.20 um 18:19 schrieb Nikita Grechikhin: > Hi, everyone. > =20 > I have several servers with large UFS partitions (>100TB). > Physically it is RAID-6 on MegaRaid 9361. >=20 > If i try to run fsck on them, i get same error on all servers: > =20 > Code: >=20 > ** Phase 5 - Check Cyl groups > fsck_ffs: inoinfo: inumber 18446744071562087424 out of range This number is near minus 2^31 - without having looked at the code this smells like an integer overflow. > inumber is exact same on all servers. > tested on clean and drity filesystems with same result. >=20 > And without succsessful run of fsck i can't remove dirty flag from file= system > =20 > This is a fresh install of FreeBSD 12.1 and filesystem created in 12.1 = from scratch with default options. The default options assume an average file size of just a few KB, and I severely doubt you are going to fill 100 TB with small files. > This is file storage for videoproduction. So, there is not too much fil= es and they is relatively big. > =20 > Code: >=20 > # find . -type f -print | wc -l > 251485 > =20 > # find . -type d -print | wc -l > 48351 What fraction of the storage is filled? You may need to recreate the file system with an appropriate value if "-i", the number of bytes per inode. If your files are generally in the order of 1 GB, then you should try -i 100000000 (i.e. 100 mio to allow for up to 1 million files to have some slack). > Maybe i doing something wrong? > =20 > Code: >=20 > # uname -a > FreeBSD *** 12.1-RELEASE FreeBSD 12.1-RELEASE r354233 GENERIC amd64 >=20 > # gpart show mfid3 > =3D> 40 250031243184 mfid3 GPT (116T) > 40 250031243184 1 freebsd-ufs (116T) >=20 > # df -i /mnt/data2 > Filesystem Size Used Avail Capacity iused ifree %iused Mount= ed on > /dev/mfid3p1 116T 15T 91T 14% 300k 3.9G 0% /mnt/= data2 Yes, way too many inodes (4G) - you are wasting space and run into the fsck issue due to them ... Regards, STefan --------------7DBCB3634E8ECB88FE3E57A5-- --asO6qngYmXXwN2XkiOWjTKLToCBKpDr54-- --CgrS5hA1IFED7gHU3rE4aIJ2RrCPFWJNZ Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEEo3HqZZwL7MgrcVMTR+u171r99UQFAl+HLW8FAwAAAAAACgkQR+u171r99UQm 8wf+OCfMxzdm0oQo0Z1Z7IKyQPKzs8T6Mf/KKywyjYUKZO9sxDZnpUpMMCTgvcVmV/tST9ETdbNL n7y/3IxSFLjQp0jocO2OnInJnbjFjEaNrYYfktwFiTFkJWacPCUSmo9Uxbt0UDzMObnFAxjgOQm7 YNnuYJdiCncLZk9+mJJ/9cBPbEEUlU44Pzed9IQvdmFtQiqU62eJBod08626gT/KAQpPV6AJtX9/ VFP1LBg71eV+mzEEvEA1OpEotm2c4lMZgN7rkNkfZnNkvTE3daAtUVo6BsjOulGT3QH6/a0zSc2/ TkB5O2zZiy1SYW1JfcvM/m/cjRebqccSJUupDFojdQ== =B0Iq -----END PGP SIGNATURE----- --CgrS5hA1IFED7gHU3rE4aIJ2RrCPFWJNZ--