From owner-svn-src-head@FreeBSD.ORG Wed Aug 28 17:38:05 2013 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A9B3C9B4; Wed, 28 Aug 2013 17:38:05 +0000 (UTC) (envelope-from mckusick@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 87BA12A75; Wed, 28 Aug 2013 17:38:05 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7SHc5Vh070711; Wed, 28 Aug 2013 17:38:05 GMT (envelope-from mckusick@svn.freebsd.org) Received: (from mckusick@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id r7SHc5BE070710; Wed, 28 Aug 2013 17:38:05 GMT (envelope-from mckusick@svn.freebsd.org) Message-Id: <201308281738.r7SHc5BE070710@svn.freebsd.org> From: Kirk McKusick Date: Wed, 28 Aug 2013 17:38:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r254995 - head/sys/ufs/ffs X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Aug 2013 17:38:05 -0000 Author: mckusick Date: Wed Aug 28 17:38:05 2013 New Revision: 254995 URL: http://svnweb.freebsd.org/changeset/base/254995 Log: A performance problem was reported in PR kern/181226: I have 25TB Dell PERC 6 RAID5 array. When it becomes almost full (10-20GB free), processes which write data to it start eating 100% CPU and write speed drops below 1MB/sec (normally to gives 400MB/sec). The revision at which it first became apparent was http://svnweb.freebsd.org/changeset/base/249782. The offending change reserved an area in each cylinder group to store metadata. The new algorithm attempts to save this area for metadata and allows its use for non-metadata only after all the data areas have been exhausted. The size of the reserved area defaults to half of minfree, so the filesystem reports full before the data area can completely fill. However, in this report, the filesystem has had minfree reduced to 1% thus forcing the metadata area to be used for data. As the filesystem approached full, it had only metadata areas left to allocate. The result was that every block allocation had to scan summary data for 30,000 cylinder groups before falling back to searching up to 30,000 metadata areas. The fix is to give up on saving the metadata areas once the free space reserve drops below 2%. The effect of this change is to use the old algorithm of just accepting the first available block that we find. Since most filesystems use the default 5% minfree, this will have no effect on their operation. For those that want to push to the limit, they will get their crappy block placements quickly. Submitted by: Dmitry Sivachenko Fix Tested by: Dmitry Sivachenko PR: kern/181226 MFC after: 2 weeks Modified: head/sys/ufs/ffs/ffs_alloc.c Modified: head/sys/ufs/ffs/ffs_alloc.c ============================================================================== --- head/sys/ufs/ffs/ffs_alloc.c Wed Aug 28 16:59:55 2013 (r254994) +++ head/sys/ufs/ffs/ffs_alloc.c Wed Aug 28 17:38:05 2013 (r254995) @@ -516,7 +516,13 @@ ffs_reallocblks_ufs1(ap) ip = VTOI(vp); fs = ip->i_fs; ump = ip->i_ump; - if (fs->fs_contigsumsize <= 0) + /* + * If we are not tracking block clusters or if we have less than 2% + * free blocks left, then do not attempt to cluster. Running with + * less than 5% free block reserve is not recommended and those that + * choose to do so do not expect to have good file layout. + */ + if (fs->fs_contigsumsize <= 0 || freespace(fs, 2) < 0) return (ENOSPC); buflist = ap->a_buflist; len = buflist->bs_nchildren; @@ -737,7 +743,13 @@ ffs_reallocblks_ufs2(ap) ip = VTOI(vp); fs = ip->i_fs; ump = ip->i_ump; - if (fs->fs_contigsumsize <= 0) + /* + * If we are not tracking block clusters or if we have less than 2% + * free blocks left, then do not attempt to cluster. Running with + * less than 5% free block reserve is not recommended and those that + * choose to do so do not expect to have good file layout. + */ + if (fs->fs_contigsumsize <= 0 || freespace(fs, 2) < 0) return (ENOSPC); buflist = ap->a_buflist; len = buflist->bs_nchildren;