Date: Wed, 29 Mar 2023 07:50:51 GMT From: =?utf-8?Q?Stefan=20E=C3=9Fer?= <se@FreeBSD.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: git: c33db74b5323 - main - fs/msdosfs: add tracking of free root directory entries Message-ID: <202303290750.32T7opKd045070@gitrepo.freebsd.org>
next in thread | raw e-mail | index | archive | help
The branch main has been updated by se: URL: https://cgit.FreeBSD.org/src/commit/?id=c33db74b5323480fba7adef58e8aa88f6091d134 commit c33db74b5323480fba7adef58e8aa88f6091d134 Author: Stefan Eßer <se@FreeBSD.org> AuthorDate: 2023-03-29 06:46:01 +0000 Commit: Stefan Eßer <se@FreeBSD.org> CommitDate: 2023-03-29 06:46:01 +0000 fs/msdosfs: add tracking of free root directory entries This update implements tallying of free directory entries during create, delete, or rename operations on FAT12 and FAT16 file systems. Prior to this change, the total number of root directory entries was reported as number of inodes, but 0 as the number of free inodes, causing system health monitoring software to warn about a suspected disk full issue. The FAT12 and FAT16 file systems provide a limited number of root directory entries, e.g. 512 on typical hard disk formats. The valid range of values is 1 to 65535, but the msdosfs code will effectively round up "odd" values to the next multiple of 16 (e.g. 513 would allow for 528 root directory entries). This update implements tracking of directory entries during create, delete, or rename operations, with initial values determined by scanning the directory when the file system is mounted. Total and free directory entries are reported in the f_files and f_ffree elements of struct statfs, despite differences in semantics of these values: - There is no limit on the number of files and directories that can be created on a FAT file system. Only the root directory of FAT12 and FAT16 file systems is limited, any number of files can still be created in sub-directories, even when 0 free "inodes" are reported. - A single file can require 1 to 21 directory entries, depending on the character set, structure, and length of the name. The DOS 8.3 style file name takes up 1 entry, and if the name does not comply with the syntax of a DOS 8.3 file name, 1 additional entry is used for each 13 characters of the file name. Since all these entries have to be contiguous, it is possible that a file or directory with a long name can not be created, despite a sufficient total number of free directory entries. - Renaming a file can require more directory entries than currently allocated to store its long name, which may prevent an in-place update of the name if more entries are needed. This may cause a rename operation to fail if no contiguous range of free entries for the new name can be found. - The volume label is stored in a directory entry. An empty FAT file system with a volume label will therefore show 1 used "inode" in df. - The perceentage of free inodes shown in df or monitoring tools does only represent the state of the root directory of a FAT12 or FAT16 file system. Neither does a reported value of 0% free inodes does prevent files from being created in sub-directories, nor does a value of 50% free inodes guarantee that even a single file with a "long" name can be created in the root directory (if every other directory entry is occupied and there are no 2 contiguous entries). The statfs(2) and df(1) man pages have been updated with a notice regarding the possibly different semantics of values reported as total and free inodes for non-Unix file systems. PR: 270053 Reported by: Ben Woods <woodsb02@freebsd.org> Approved by: mckusick MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D38987 --- bin/df/df.1 | 13 ++++- lib/libc/sys/statfs.2 | 18 ++++++- sys/fs/msdosfs/msdosfs_lookup.c | 3 ++ sys/fs/msdosfs/msdosfs_vfsops.c | 105 +++++++++++++++++++++++++++++++++++++++- sys/fs/msdosfs/msdosfsmount.h | 17 +++++++ 5 files changed, 152 insertions(+), 4 deletions(-) diff --git a/bin/df/df.1 b/bin/df/df.1 index 64e471fedc42..aa35318c2c65 100644 --- a/bin/df/df.1 +++ b/bin/df/df.1 @@ -29,7 +29,7 @@ .\" @(#)df.1 8.3 (Berkeley) 5/8/95 .\" $FreeBSD$ .\" -.Dd March 11, 2022 +.Dd March 29, 2023 .Dt DF 1 .Os .Sh NAME @@ -246,6 +246,17 @@ $ df -h /dev/ada1p2 Filesystem Size Used Avail Capacity Mounted on /dev/ada1p2 213G 152G 44G 78% / .Ed +.Sh NOTES +For non-Unix file systems, the reported values of used and free inodes +may have a different meaning than that of used and available files and +directories. +An example is msdosfs, which in the case of FAT12 or FAT16 file systems +reports the number of available and free root directory entries instead +of inodes +.Po +where 1 to 21 such directory entries are required to store +each file or directory name or disk label +.Pc . .Sh SEE ALSO .Xr lsvfs 1 , .Xr quota 1 , diff --git a/lib/libc/sys/statfs.2 b/lib/libc/sys/statfs.2 index a865652022bd..4df0e801e418 100644 --- a/lib/libc/sys/statfs.2 +++ b/lib/libc/sys/statfs.2 @@ -28,7 +28,7 @@ .\" @(#)statfs.2 8.5 (Berkeley) 5/24/95 .\" $FreeBSD$ .\" -.Dd March 30, 2020 +.Dd March 29, 2023 .Dt STATFS 2 .Os .Sh NAME @@ -230,6 +230,22 @@ error occurred while reading from or writing to the file system. .It Bq Er EINTEGRITY Corrupted data was detected while reading from the file system. .El +.Sh NOTES +The fields in the +.Vt statfs +structure have been defined to provide the parameters relevant for +traditional +.Tm UNIX +file systems. +For some other file systems, values that have similar, but not +identical, semantics to those described above may be returned. +An example is msdosfs, which in case of FAT12 or FAT13 file systems +reports the number of available and of free root directory entries +instead of inodes +.Po +where 1 to 21 such directory entries are required to store +each file or directory name or disk label +.Pc . .Sh SEE ALSO .Xr fhstatfs 2 , .Xr getfsstat 2 diff --git a/sys/fs/msdosfs/msdosfs_lookup.c b/sys/fs/msdosfs/msdosfs_lookup.c index d67a96ad11c5..0a6f141903c1 100644 --- a/sys/fs/msdosfs/msdosfs_lookup.c +++ b/sys/fs/msdosfs/msdosfs_lookup.c @@ -684,6 +684,7 @@ createde(struct denode *dep, struct denode *ddep, struct denode **depp, return error; } ndep = bptoep(pmp, bp, ddep->de_fndoffset); + rootde_alloced(ddep); DE_EXTERNALIZE(ndep, dep); @@ -721,6 +722,7 @@ createde(struct denode *dep, struct denode *ddep, struct denode **depp, ndep--; ddep->de_fndoffset -= sizeof(struct direntry); } + rootde_alloced(ddep); if (!unix2winfn(un, unlen, (struct winentry *)ndep, cnt++, chksum, pmp)) break; @@ -1015,6 +1017,7 @@ removede(struct denode *pdep, struct denode *dep) */ offset -= sizeof(struct direntry); ep--->deName[0] = SLOT_DELETED; + rootde_freed(pdep); if ((pmp->pm_flags & MSDOSFSMNT_NOWIN95) || !(offset & pmp->pm_crbomask) || ep->deAttributes != ATTR_WIN95) diff --git a/sys/fs/msdosfs/msdosfs_vfsops.c b/sys/fs/msdosfs/msdosfs_vfsops.c index 37e5c4d4500a..e2b1fd6b91f5 100644 --- a/sys/fs/msdosfs/msdosfs_vfsops.c +++ b/sys/fs/msdosfs/msdosfs_vfsops.c @@ -409,6 +409,97 @@ msdosfs_mount(struct mount *mp) return (0); } +/* + * The FAT12 and FAT16 file systems use a limited size root directory that + * can be created with 1 to 65535 entries for files, directories, or a disk + * label (but DOS or Windows creates at most 512 root directory entries). + * This function calculates the number of free root directory entries by + * counting the non-deleted entries (not starting with 0xE5) and by adding + * the amount of never used entries (with the position indicated by an + * entry that starts with 0x00). + */ +static int +rootdir_free(struct msdosfsmount* pmp) +{ + struct buf *bp; + struct direntry *dep; + u_long readsize; + int dirclu; + int diridx; + int dirmax; + int dirleft; + int ffree; + + dirclu = pmp->pm_rootdirblk; + + /* + * The msdosfs code ignores pm_RootDirEnts and uses pm_rootdirsize + * (measured in DEV_BSIZE) to prevent excess root dir allocations. + */ + dirleft = howmany(pmp->pm_rootdirsize * DEV_BSIZE, + sizeof(struct direntry)); + + /* Read in chunks of default maximum root directory size */ + readsize = 512 * sizeof(struct direntry); + +#ifdef MSDOSFS_DEBUG + printf("rootdir_free: blkpersec=%lu fatblksize=%lu dirsize=%lu " + "firstclu=%lu dirclu=%d entries=%d rootdirsize=%lu " + "bytespersector=%hu bytepercluster=%lu\n", + pmp->pm_BlkPerSec, pmp->pm_fatblocksize, readsize, + pmp->pm_firstcluster, dirclu, dirleft, pmp->pm_rootdirsize, + pmp->pm_BytesPerSec, pmp->pm_bpcluster); +#endif + ffree = dirleft; + while (dirleft > 0 && ffree > 0) { + if (readsize > dirleft * sizeof(struct direntry)) + readsize = dirleft * sizeof(struct direntry); +#ifdef MSDOSFS_DEBUG + printf("rootdir_free: dirclu=%d dirleft=%d readsize=%lu\n", + dirclu, dirleft, readsize); +#endif + if (bread(pmp->pm_devvp, dirclu, readsize, NOCRED, &bp) != 0) { + printf("rootdir_free: read error\n"); + if (bp != NULL) + brelse(bp); + return (-1); + } + dirmax = readsize / sizeof(struct direntry); + for (diridx = 0; diridx < dirmax && dirleft > 0; + diridx++, dirleft--) { + dep = (struct direntry*)bp->b_data + diridx; +#ifdef MSDOSFS_DEBUG + if (dep->deName[0] == SLOT_DELETED) + printf("rootdir_free: idx=%d <deleted>\n", + diridx); + else if (dep->deName[0] == SLOT_EMPTY) + printf("rootdir_free: idx=%d <end marker>\n", + diridx); + else if (dep->deAttributes == ATTR_WIN95) + printf("rootdir_free: idx=%d <LFN part %d>\n", + diridx, (dep->deName[0] & 0x1f) + 1); + else if (dep->deAttributes & ATTR_VOLUME) + printf("rootdir_free: idx=%d label='%11.11s'\n", + diridx, dep->deName); + else if (dep->deAttributes & ATTR_DIRECTORY) + printf("rootdir_free: idx=%d dir='%11.11s'\n", + diridx, dep->deName); + else + printf("rootdir_free: idx=%d file='%11.11s'\n", + diridx, dep->deName); +#endif + if (dep->deName[0] == SLOT_EMPTY) + dirleft = 0; + else if (dep->deName[0] != SLOT_DELETED) + ffree--; + } + brelse(bp); + bp = NULL; + dirclu += readsize / DEV_BSIZE; + } + return (ffree); +} + static int mountmsdosfs(struct vnode *odevvp, struct mount *mp) { @@ -747,6 +838,15 @@ mountmsdosfs(struct vnode *odevvp, struct mount *mp) goto error_exit; pmp->pm_fmod = 1; } + + if (FAT32(pmp)) { + pmp->pm_rootdirfree = 0; + } else { + pmp->pm_rootdirfree = rootdir_free(pmp); + if (pmp->pm_rootdirfree < 0) + goto error_exit; + } + mp->mnt_data = pmp; mp->mnt_stat.f_fsid.val[0] = dev2udev(dev); mp->mnt_stat.f_fsid.val[1] = mp->mnt_vfc->vfc_typenum; @@ -948,8 +1048,9 @@ msdosfs_statfs(struct mount *mp, struct statfs *sbp) sbp->f_blocks = pmp->pm_maxcluster + 1; sbp->f_bfree = pmp->pm_freeclustercount; sbp->f_bavail = pmp->pm_freeclustercount; - sbp->f_files = pmp->pm_RootDirEnts; /* XXX */ - sbp->f_ffree = 0; /* what to put in here? */ + sbp->f_files = howmany(pmp->pm_rootdirsize * DEV_BSIZE, + sizeof(struct direntry)); + sbp->f_ffree = pmp->pm_rootdirfree; return (0); } diff --git a/sys/fs/msdosfs/msdosfsmount.h b/sys/fs/msdosfs/msdosfsmount.h index 5caa40f2b648..5afbfc44dc0a 100644 --- a/sys/fs/msdosfs/msdosfsmount.h +++ b/sys/fs/msdosfs/msdosfsmount.h @@ -108,6 +108,7 @@ struct msdosfsmount { u_int pm_fatmult; /* these 2 values are used in FAT */ u_int pm_fatdiv; /* offset computation */ u_int pm_curfat; /* current FAT for FAT32 (0 otherwise) */ + int pm_rootdirfree; /* number of free slots in FAT12/16 root directory */ u_int *pm_inusemap; /* ptr to bitmap of in-use clusters */ uint64_t pm_flags; /* see below */ void *pm_u2w; /* Local->Unicode iconv handle */ @@ -222,6 +223,22 @@ struct msdosfs_fileno { ? roottobn((pmp), (dirofs)) \ : cntobn((pmp), (dirclu))) +/* + * Increment the number of used entries in a fixed size FAT12/16 root + * directory + */ +#define rootde_alloced(dep) \ + if ((dep)->de_StartCluster == MSDOSFSROOT) \ + (dep)->de_pmp->pm_rootdirfree--; + +/* + * Decrement the number of used entries in a fixed size FAT12/16 root + * directory + */ +#define rootde_freed(dep) \ + if ((dep)->de_StartCluster == MSDOSFSROOT) \ + (dep)->de_pmp->pm_rootdirfree++; + #define MSDOSFS_LOCK_MP(pmp) \ lockmgr(&(pmp)->pm_fatlock, LK_EXCLUSIVE, NULL) #define MSDOSFS_UNLOCK_MP(pmp) \
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202303290750.32T7opKd045070>