From owner-freebsd-fs@freebsd.org Thu Aug 4 17:04:09 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C689BAFA5B for ; Thu, 4 Aug 2016 17:04:09 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (chez.mckusick.com [IPv6:2001:5a8:4:7e72:d250:99ff:fe57:4030]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "chez.mckusick.com", Issuer "chez.mckusick.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 01BB31898; Thu, 4 Aug 2016 17:04:08 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (localhost [IPv6:::1]) by chez.mckusick.com (8.15.2/8.15.2) with ESMTP id u74H47hb090342; Thu, 4 Aug 2016 10:04:07 -0700 (PDT) (envelope-from mckusick@mckusick.com) Message-Id: <201608041704.u74H47hb090342@chez.mckusick.com> From: Kirk McKusick To: Maxim Sobolev Subject: Re: Optimizing UFS 1/2 for non-rotating / compressed storage cc: FreeBSD Filesystems In-reply-to: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <90340.1470330247.1@chez.mckusick.com> Date: Thu, 04 Aug 2016 10:04:07 -0700 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Aug 2016 17:04:09 -0000 > From: Maxim Sobolev > Date: Wed, 20 Jul 2016 11:45:03 -0700 > Subject: Optimizing UFS 1/2 for non-rotating / compressed storage > To: Kirk McKusick , > FreeBSD Filesystems > > Hi Kirk et al, > > Do you by any chance have some hints of what parameters we need to set in > newfs to maximally fit the following criteria: > > 1. Minimize free space fragmentation, i.e. we start with huge array of > zeroes, we want to end up with as few number of continuous zero areas as > possible (i.e. minimize free space discontinuity). > > 2. Blocks that belong to the same file should be as continuous as possible > "on disk". > > 3. Each individual file should preferably start at the block offset that is > multiple of certain pre-defined power-of-two size from the start of > partition, e.g. 64k, 128k etc. > > The file system in question is write-mostly. We create image from scratch > every time and them populate with installworld + pkg add. Any free space is > subsequently erased with dd if=/dev/zero of=/myfs/bigfile; rm > /myfs/bigfile, unmounted and image is compressed. We also grossly > over-provision space, i.e. 2GB UFS image is created, less than 1GB is used > at the end. > > Any hints would be appreciated. Thanks! > > -Maxim Just back from spending the month of July in Tasmania (Australia) and trying to get caught up on email... Unfortunately UFS/FFS is not well designed for what you want to do. It splits the filesystem space up into "cylinder groups" and then tries to place the files evenly across the cylinder groups. At least it packs the files into the front of each cylinder group, so you will tend to get a big block of unallocated space at the end of each cylinder group. You could benefit from allocating the fewest number of cylinder groups possible which is what newfs does by default. But you could help this along by creating a filesystem with no fragments (just full-sized blocks) as that keeps the bitmaps small (the bitmap needs one bit per possible fragment). I will note that going without fragments will blow up your disk usage if you have many small files, as a small file will use 8x as much space as it would if you had fragments. Use the `-e maxbpg' parameter to newfs (or tunefs after the fact) to set a huge value for contiguous blocks before being forced to move to a new cylinder group. Note that doing this will penalize your small file read performance, so you may want to leave this alone. To get all files to start on a particular block boundary, set your filesystem block size to the starting offset boundary you desire (e.g., if you want files to start on a 32k offset, use a 32k block size for your filesystem). If you create a filesystem with no fragments, then all files will by definition start on a block boundary. Kirk McKusick