From owner-freebsd-fs@FreeBSD.ORG Thu Jul 20 02:08:08 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B023416A4DD for ; Thu, 20 Jul 2006 02:08:08 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4421A43D45 for ; Thu, 20 Jul 2006 02:08:07 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [192.168.42.24] (andersonbox4.centtech.com [192.168.42.24]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k6K287BP090456 for ; Wed, 19 Jul 2006 21:08:07 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <44BEE590.2030905@centtech.com> Date: Wed, 19 Jul 2006 21:08:16 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.4 (X11/20060612) MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1609/Wed Jul 19 07:13:27 2006 on mh1.centtech.com X-Virus-Status: Clean Subject: Failed to flush worklist X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jul 2006 02:08:08 -0000 On one of my servers, I'm unable to change a mount point from rw to ro (something I do all the time). When I try it, I receive messages like: Jul 19 08:57:18 snapshot1 kernel: softdep_waitidle: Failed to flush worklist for 0xc9f37400softdep_waitidle: Failed to flush worklist for 0xc9f37400 Jul 19 08:57:18 snapshot1 kernel: softdep_waitidle: Failed to flush worklist for 0xc9f37400 Jul 19 08:57:53 snapshot1 kernel: softdep_waitidle: Failed to flush worklist for 0xc9f37400 Jul 19 08:58:28 snapshot1 kernel: softdep_waitidle: Failed to flush worklist for 0xc9f37400 # df /vol4 Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/label/vol4 1891668564 1769762782 -29427702 102% /vol4 # uname -a FreeBSD snapshot1.centtech.com 6.1-STABLE FreeBSD 6.1-STABLE #2: Tue May 16 08:34:43 CDT 2006 root@snapshot1.centtech.com:/usr/obj/usr/src/sys/SNAPSHOT i386 # mount /dev/label/vol4 on /vol4 (ufs, local, noatime, soft-updates) Now, here's how I think I broke it: Filesystem was mounted rw, and all was well. Perl program was running in the background, accessing data in a directory deep down inside the filesystem, maybe even its cwd was in there when it started. I did massive rm's, hard links, and writing of new files (see rsync and the link-dest option), probably including the directory the perl program was in. Later on, I tried to do: mount -u -oro /vol4 which failed, with either permission denied or operation not permitted. I cannot umount the filesystem, nor does the -f option help. Here's a snippet from fstat - nothing is currently running on that filesystem that I can find, and this is the only thing that looks odd: root perl 99912 root / 2 drwxr-xr-x 1024 r root perl 99912 wd / 2 drwxr-xr-x 1024 r root perl 99912 text / 238402 -rwxr-xr-x 10088 r root perl 99912 0 - - bad - root perl 99912 1 - - bad - root perl 99912 2 - - bad - root perl 99740 root / 2 drwxr-xr-x 1024 r root perl 99740 wd / 2 drwxr-xr-x 1024 r root perl 99740 text / 238402 -rwxr-xr-x 10088 r root perl 99740 0 - - bad - root perl 99740 1 - - bad - root perl 99740 2 - - bad - Any ideas?? Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------