From owner-freebsd-stable@FreeBSD.ORG Wed Jan 4 02:09:49 2006 Return-Path: X-Original-To: freebsd-stable@FreeBSD.org Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9CB1516A41F; Wed, 4 Jan 2006 02:09:49 +0000 (GMT) (envelope-from gcr+freebsd-stable@tharned.org) Received: from nc8000.tharned.org (rrcs-24-56-87-26.ma.biz.rr.com [24.56.87.26]) by mx1.FreeBSD.org (Postfix) with ESMTP id C6EEE43D49; Wed, 4 Jan 2006 02:09:48 +0000 (GMT) (envelope-from gcr+freebsd-stable@tharned.org) Received: from nc8000.tharned.org (gcr@localhost [127.0.0.1]) by nc8000.tharned.org (8.13.4/8.13.4) with ESMTP id k0429ibK025289; Tue, 3 Jan 2006 20:09:44 -0600 (CST) (envelope-from gcr+freebsd-stable@tharned.org) Received: from localhost (gcr@localhost) by nc8000.tharned.org (8.13.4/8.13.4/Submit) with ESMTP id k0429iRV025286; Tue, 3 Jan 2006 20:09:44 -0600 (CST) (envelope-from gcr+freebsd-stable@tharned.org) Date: Tue, 3 Jan 2006 20:09:44 -0600 (CST) From: Greg Rivers Sender: gcr@tharned.org To: Don Lewis In-Reply-To: <200601040156.k041u5Z9008271@gw.catspoiler.org> Message-ID: <20060103195956.L798@nc8000.tharned.org> References: <200601040156.k041u5Z9008271@gw.catspoiler.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-stable@FreeBSD.org, Kirk McKusick , Kris Kennaway Subject: Re: Recurring problem: processes block accessing UFS file system X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jan 2006 02:09:49 -0000 On Tue, 3 Jan 2006, Don Lewis wrote: >> db> show lockedbufs > > [snip] > > looks like this is the buf that pid 87150 is waiting for: > >> buf at 0xdc713f50 >> b_flags = 0xa00200a0 >> b_error = 0, b_bufsize = 16384, b_bcount = 16384, b_resid = 0 >> b_bufobj = (0xc8985610), b_data = 0xe0b7b000, b_blkno = 365094624 >> lockstatus = 2, excl count = 1, excl owner 0xcfeb5d80 >> b_npages = 4, pages(OBJ, IDX, PA): (0xc8984108, 0x2b85cdc, 0xa89e9000),(0xc8984108, 0x2b85cdd, 0xa852a000),(0xc8984108, 0x2b85cde, 0xa850b000),(0xc8984108, 0x2b85cdf, 0xa836c000) > > which is locked by this thread: > > Tracing command sendmail pid 87117 tid 101335 td 0xcfeb5d80 > sched_switch(cfeb5d80,0,1,fd1926a,640c65f9) at sched_switch+0x158 > mi_switch(1,0,c04d7b33,dc76fe8c,ec883b2c) at mi_switch+0x1d5 > sleepq_switch(dc76fe8c,ec883b60,c04bb9ce,dc76fe8c,4c) at sleepq_switch+0x16f > sleepq_wait(dc76fe8c,4c,c061e9ac,0,0) at sleepq_wait+0x11 > msleep(dc76fe8c,c0662f80,4c,c061e9ac,0) at msleep+0x3d7 > getdirtybuf(dc76fe30,c0662f80,1,ec883ba8,0) at getdirtybuf+0x221 > softdep_update_inodeblock(cd1bc528,dc713f50,1,4000,0) at softdep_update_inodeblo > ck+0x267 > ffs_update(cd953bb0,1,0,cd953bb0,ec883c78,c0529a59,0,0,0,4,1,cd953c2c) at ffs_up > date+0x27f > ffs_syncvnode(cd953bb0,1,4,ec883c78,c05f9a70) at ffs_syncvnode+0x52e > ffs_fsync(ec883cb4,ec883cd0,c052468a,c0646cc0,ec883cb4) at ffs_fsync+0x1c > VOP_FSYNC_APV(c0646cc0,ec883cb4,0,0,0) at VOP_FSYNC_APV+0x3a > fsync(cfeb5d80,ec883d04,4,cfeb5d80,ec883d2c) at fsync+0x1db > syscall(3b,3b,3b,80c7c1b,bfbfa6b0) at syscall+0x324 > Xint0x80_syscall() at Xint0x80_syscall+0x1f > --- syscall (95, FreeBSD ELF32, fsync), eip = 0x8830f63f, esp = 0xbfbfa66c, ebp > = 0xbfbfaf98 --- > > > Pid 87117 is playing with buf 0xdc76fe30 which is not locked, and is > sleeping on the buf's b_xflags member. It looks like 87117 is waiting > for an in-progress write to complete. There are a large number of other > sendmail processes waiting in this same place. > > How about "show buffer 0xdc76fe30"? > db> show buffer 0xdc76fe30 buf at 0xdc76fe30 b_flags = 0x200000a0 b_error = 0, b_bufsize = 16384, b_bcount = 16384, b_resid = 0 b_bufobj = (0xc8985610), b_data = 0xe1d6b000, b_blkno = 365086368 lockstatus = 0, excl count = 0, excl owner 0xffffffff b_npages = 4, pages(OBJ, IDX, PA): (0xc8984108, 0x2b858d4, 0xa8de1000),(0xc8984108, 0x2b858d5, 0xa8c62000),(0xc8984108, 0x2b858d6, 0xa8de3000),(0xc8984108, 0x2b858d7, 0xa8e64000) db> > This is getting into an area of the kernel that I do not understand > well. > For me, we've long since passed that point. :-) -- Greg