From owner-freebsd-fs@freebsd.org Tue May 17 08:20:57 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B0D63B3E992 for ; Tue, 17 May 2016 08:20:57 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 9BF3E1D7C for ; Tue, 17 May 2016 08:20:57 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 9B34EB3E991; Tue, 17 May 2016 08:20:57 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9AD48B3E990 for ; Tue, 17 May 2016 08:20:57 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 33C901D7B for ; Tue, 17 May 2016 08:20:56 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u4H8Kohw013266 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 17 May 2016 11:20:51 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u4H8Kohw013266 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u4H8Ko7G013262; Tue, 17 May 2016 11:20:50 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 17 May 2016 11:20:50 +0300 From: Konstantin Belousov To: Bruce Evans Cc: fs@freebsd.org Subject: Re: quick fix for slow directory shrinking in ffs Message-ID: <20160517082050.GX89104@kib.kiev.ua> References: <20160517072705.F2157@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160517072705.F2157@besplex.bde.org> User-Agent: Mutt/1.6.1 (2016-04-27) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 May 2016 08:20:57 -0000 On Tue, May 17, 2016 at 07:54:27AM +1000, Bruce Evans wrote: > ffs does very slow shrinking of directories after removing some files > leaves unused blocks at the end, by always doing synchronous truncation. > > This often happens in my normal usage: medium size builds expand /tmp > from 512 to 1024 to hold a few more hundred bytes of file names; > expansion is async and fast, but shrinking is sync and slow, and > with a certain size of build the boundary is crossed back and forth > very often. > > My /tmp directory is always on an async-mounted file system, so this > quick fix of always doing an async truncation for async mounts works > for me. Using IO_SYNC when not asked to is a bug for async mounts > in all cases anyway. > > The file system has block size 8192 and frag size 1024, so it is also > wrong to shrink to size DIRBLKSIZE = 512. The shrinkage seems to be > considered at every DIRBLKSIZE boundary, so not only small directories > are affected. > > The patch fixes an unrelated typo in a message. > > X Index: ufs_lookup.c > X =================================================================== > X --- ufs_lookup.c (revision 299263) > X +++ ufs_lookup.c (working copy) > X @@ -1131,9 +1131,9 @@ > X if (tvp != NULL) > X VOP_UNLOCK(tvp, 0); > X error = UFS_TRUNCATE(dvp, (off_t)dp->i_endoff, > X - IO_NORMAL | IO_SYNC, cr); > X + IO_NORMAL | (DOINGASYNC(dvp) ? 0 : IO_SYNC), cr); > X if (error != 0) > X - vprint("ufs_direnter: failted to truncate", dvp); > X + vprint("ufs_direnter: failed to truncate", dvp); > X #ifdef UFS_DIRHASH > X if (error == 0 && dp->i_dirhash != NULL) > X ufsdirhash_dirtrunc(dp, dp->i_endoff); > The IO_SYNC flag, for non-journaled SU and any kind of non-SU mounts, only affects the new blocks allocation mode, and write-out mode for the last fragment. The truncation itself (for -J) is performed in the context of the truncating thread. The cg blocks, after the bits are set to free, are marked for delayed write (with the background write hack). The inode block is written according to the mount mode, ignoring IO_SYNC. That is, for always fully populated directory files, I do not see how anything is changed by the patch. I committed the typo fix.