From owner-freebsd-current@FreeBSD.ORG Sun Jun 27 04:39:13 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6A34316A4CF for ; Sun, 27 Jun 2004 04:39:13 +0000 (GMT) Received: from gw.catspoiler.org (217-ip-163.nccn.net [209.79.217.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id EF95943D48 for ; Sun, 27 Jun 2004 04:39:12 +0000 (GMT) (envelope-from truckman@FreeBSD.org) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.12.11/8.12.11) with ESMTP id i5R4cxQw064263; Sat, 26 Jun 2004 21:39:04 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Message-Id: <200406270439.i5R4cxQw064263@gw.catspoiler.org> Date: Sat, 26 Jun 2004 21:38:59 -0700 (PDT) From: Don Lewis To: avleeuwen@piwebs.com In-Reply-To: MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii cc: freebsd-current@FreeBSD.org Subject: Re: Giving up on x buffers - losing files X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Jun 2004 04:39:13 -0000 On 26 Jun, Arjan van Leeuwen wrote: > I've had this problem for a long time with 5.x, but it doesn't seem to > happen that often. Today it bit me hard. > > Sometimes, particularly after doing a lot of file writes (i.e. compiling a > lot of ports, building world and mergemastering, etc), I get the 'Giving > up on x buffers' message on shutdown, and my filesystems come up dirty > when I restart. > > This wouldn't be such an enormous problem, if it wouldn't always erase the > files I changed most recently. The files are simply reduced to 0 bytes. My > configuration files for Opera and KDE have been victim to this more than > once (because Opera writes to the file on exit, for example), but today, > it was /etc/master.passwd that was reduced to 0 bytes (because I had just > changed something in it). > > I understand that turning of write caching might improve the situation, > but it also makes my system a lot slower, and I don't like that on my > desktop system. > > So, why does this happen? And how do I prevent it from happening? This > definitely does _not_ sound like something I want my servers to do when > 5.x goes -STABLE. I've mentioned this a couple of times on this list in the last six months or so. The last time was in the last couple of weeks. I can reliably trigger this problem with mergemaster. I'm pretty sure that the problem relates to soft updates and how the file system syncer is shut down, which leaves unresolved dependencies that keep a number of dirty blocks from being flushed to disk at the end of the system shutdown. I have some ideas on how to fix the problem, but I haven't had the time to work on it and nobody else has stepped up with a fix. I am able to reliably work around the problem by running the sync command and waiting a short while after running mergemaster and before shutting down or rebooting the machine.