From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 6 11:57:44 2005 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A18A816A41F for ; Tue, 6 Sep 2005 11:57:44 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [204.156.12.53]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4D67543D45 for ; Tue, 6 Sep 2005 11:57:44 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by cyrus.watson.org (Postfix) with ESMTP id 21C2846B10; Tue, 6 Sep 2005 07:57:43 -0400 (EDT) Date: Tue, 6 Sep 2005 12:57:42 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Peter Jeremy In-Reply-To: <20050906081855.GA26550@cirb503493.alcatel.com.au> Message-ID: <20050906125551.I51625@fledge.watson.org> References: <17db6d3a0509051000622868bc@mail.gmail.com> <431C8D5B.7080309@samsco.org> <431C92F2.9090104@persistent.co.in> <431C93DD.20402@samsco.org> <17db6d3a0509052203b1da14a@mail.gmail.com> <20050906081855.GA26550@cirb503493.alcatel.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-hackers@freebsd.org, Nikhil Dharashivkar Subject: Re: Adding new option to ktrace X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2005 11:57:44 -0000 On Tue, 6 Sep 2005, Peter Jeremy wrote: > On Tue, 2005-Sep-06 10:33:53 +0530, Nikhil Dharashivkar wrote: >> Thanks for replying me. Basically what happend, while testing >> scsi driver on freebsd, at some point it crashes. So, there is no way >> to know how much IO is performed. To know the IO state just before the >> driver fails, i selected ktrace to print IO information whatever i ll >> get from dastrategy routine. > > It's not clear how ktrace is going to help here. The ktrXXX(9) > functions place ktr_request events in a queue. A kernel thread then > dumps the queue entries into a file via the normal buffer cache. The > data on disk is typically about 30 seconds behind real time. If the > system crashes, you will lose any events that are still in the buffer > cache or ktr_todo queue. > > Another problem is that since ktrace generates disk I/O, it is likely to > disturb your testing. > > A better approach would seem to be to build a circular buffer and store > the I/O requests in the buffer. When the system crashes, you can look > at the last entries in the buffer. ktrace is indeed probably the wrong layer to do this analysis, if it is firmly believed to be a SCSI layer problem, since process level I/O events often map weakly to device level I/O events -- i.e., directory operations are substantially divorced from the block operations that implement them due to soft updates, write buffering, write combining, etc. KTR(9) supports tracing of GEOM-layer I/O events to an in-memory buffer which can be retrieved from a coredump using ktrdump. Adding additional instrumentation, say in CAM or a device driver, is pretty straight forward and probably a useful way to approach this problem. Robert N M Watson