Date: Sun, 3 Jul 2005 19:13:32 +0100 From: Peter Edwards <peadar.edwards@gmail.com> To: Robert Watson <rwatson@freebsd.org> Cc: Peter Edwards <peadar@freebsd.org>, arch@freebsd.org Subject: Re: ktrace and KTR_DROP Message-ID: <34cb7c8405070311131cd1ca8a@mail.gmail.com> In-Reply-To: <20050701155757.A36905@fledge.watson.org> References: <20050701132104.GA95135@freefall.freebsd.org> <20050701155757.A36905@fledge.watson.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 7/1/05, Robert Watson <rwatson@freebsd.org> wrote: >=20 > There are two benefits to the current ktrace dispatch model: >=20 > (1) Avoiding untimely sleeping in the execution paths of threads that are > being traced. >=20 > (2) Allowing the traced thread to run ahead asynchronously, hopefully > impacting performance less. >=20 Agreed, as long as the rate it needs to trace at doesn't exceed the rate th= e trace can be recorded at: otherwise, it needs to be flow controlled, either= by calling vn_write, or by waiting for another thread to do so. Just to re-ite= rate, I understand that not slowing down the target process is probably beneficia= l in some circumstances, but you can only do that by loosing trace information, = and in the vast majority of cases that's exactly what you don't want. > One of the things I've been thinking for a few years is that I think I > actually preferred the old model better, there processes (now threads) > would hang a "current record" off of their process (now thread) structure= , > and fill it in as they went along. The upsides of this have to do with > the downsides of the current model: that you don't allow fully > asynchronous execition of the threads with respect to queueing the record= s > to disk, so you don't run into "drop" scenarios, instead slowing down the > process. Likewise, the downsides. >=20 > In the audit code, we pull from a common record queue, but we allocate th= e > record when the system call starts for each process -- if there aren't > records available (or various other reliability-related conditions fail, > such as adequate disk space), we stall the thread entering the kernel > until we can satisfy its record allocation requirements. I don't follow: looking at the code from RELENG_4, it looks pretty similar = to what's there already: create a requst, call ktrwrite, free request. By add= ing the flow control with the patch, you just end up waiting for the free queue= to be non-empty, rather than waiting to aquire the vnode lock and do the write= . Am I missing something? Do I need to go back further in time? >=20 > There are two cases where I really run into problems with the current > model: >=20 > (1) When I'm interacting with a slow file system, such as NFS over > 100mbps, I will always lose records, because it doesn't take long fo= r > the process to get quite ahead of the write-behind. It doesn't even need NFS: syscall throughput is much better than I/O throughput :-) >=20 > (2) When I trace more than one process at a time, the volume of records > overwhelms the write-behind. As long as you need to do physical writes for the syscalls, I don't know how you'l ever avoid that... >=20 > Write coalescing/etc is already provided "for free" by pushing the writes > down into the file system, so other than slowing down the traced process = a > little, I think we don't lose much by moving back to this model. And if > we pre-commit the record storage on system call entry (with the exception > of paths, which generally require potential sleeps anyway), we probably > won't hurt performance all that much, and avoid sleeping in bad places. I'm not sure I follow: are you saying you'd rather move the ktrace resource aquisition out of where the sleep will be more critical? I don't see how you can avoid having the ktrace'd thread wait for as long as it takes for the writer thread to catch up with the system to some point
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?34cb7c8405070311131cd1ca8a>