Date: Tue, 5 Jun 2007 15:39:22 -0400 From: John Baldwin <jhb@freebsd.org> To: freebsd-hackers@freebsd.org Cc: Lawrence Stewart <lstewart@room52.net>, Dag-Erling =?utf-8?q?Sm=C3=B8rgrav?= <des@des.no> Subject: Re: Writing a plain text file to disk from kernel space Message-ID: <200706051539.22662.jhb@freebsd.org> In-Reply-To: <46565781.2030407@room52.net> References: <4649349D.4060101@room52.net> <86odkcugev.fsf@dwp.des.no> <46565781.2030407@room52.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thursday 24 May 2007 11:26:58 pm Lawrence Stewart wrote: > Comments inline... >=20 > Dag-Erling Sm=C3=B8rgrav wrote: > > Lawrence Stewart <lstewart@room52.net> writes: > > =20 > >> Dag-Erling Sm=C3=B8rgrav <des@des.no> writes: > >> =20 > >>> Since you are writing kernel code, I assume you have KDB/DDB in your > >>> kernel and know how to use it. > >>> =20 > >> I don't know how to use them really. Thus far I haven't had a need for > >> really low level debugging tools... seems that may have changed > >> though! Any good tutorials/pointers on how to get started with kernel > >> debugging? > >> =20 > > > > The handbook and FAQ have information on debugging panics. Greg Lehey > > (grog@) does a tutorial on kernel debugging, you can probably find > > slides online (or just ask him) > > =20 >=20 >=20 > For reference, I found what looks to be a very comprehensive kernel=20 > debugging reference here:=20 > http://www.lemis.com/grog/Papers/Debug-tutorial/tutorial.pdf >=20 > Greg certainly knows the ins and outs of kernel debugging! >=20 > > =20 > >>> kio_write probably blocks waiting for the write to complete. You can= 't > >>> do that while holding a non-sleepable lock. > >>> =20 > >> So this is where my knowledge/understanding gets very hazy... > >> > >> When a thread blocks waiting for some operation to complete or event > >> to happen, the thread effectively goes to sleep, correct? > >> =20 > > > > It depends on the type of lock used, but mostly, yes. > > > > =20 > >> Looking at the kio_write code in subr_kernio.c, I'm guessing the lock > >> that is causing the trouble is related to the "vn_lock" function call? > >> =20 > > > > What matters is that kio_write() may sleep and therefore can't be called > > while holding a non-sleepable lock. > > > > =20 > >> I don't understand though why the vnode lock would be set up in such a > >> way that when the write blocks whilst waiting for the underlying > >> filesystem to signal everything is ok, it causes the kernel to panic! > >> =20 > > > > You cannot sleep while holding a non-sleepable lock. You need to find > > out which locks are held at the point where you call kio_write(), and > > figure out a way to delay the kio_write() call until those locks are > > released. > > > > =20 > >> How do I make the lock "sleepable" or make sure the thread doesn't try > >> go to sleep whilst holding the lock? > >> =20 > > > > You can't make an unsleepable lock sleepable. You might be able to > > replace it with a sleepable lock, but you would have to go through every > > part of the kernel that uses the lock and make sure that it works > > correctly with a sleepable lock. Most likely, it won't. > > > > =20 >=20 >=20 > Thanks for the explanations. I'm starting to get a better picture of=20 > what's actually going on. >=20 > So it seems that there is no way I can call kio_write from within the=20 > function that is acting as a pfil output hook, because it blocks at some= =20 > point whilst doing the disk write, which makes the kernel unhappy=20 > because pfil code is holding a non-sleepable mutex somewhere. >=20 > If you read my other message from yesterday, I still can't figure out=20 > why this only happens with outbound TCP traffic, but anyways... >=20 > I'll have a bit more of a think about it and get back to the list shortly= =2E.. Use a task to defer the kio_write() to a taskqueue. You have to malloc sta= te=20 (using M_NOWAIT, which can fail) to do this properly. If you are doing thi= s=20 for every packet, you are probably better off using malloc() to throw items= =20 into a queue and having a global task that drains the queue on each executi= on=20 doing kio_write()'s for each object. Regarding sleepable vs. non-sleepable locks. Getting preempted by an=20 interrupt is not considered "sleeping". Sleeping means voluntarily yieldin= g=20 the CPU to wait for an event such as via msleep() or a condition variable. = =20 Note that interrupt handlers can acquire non-sleepable locks. If you sleep= =20 while holding a non-sleepable lock, you may have an interrupt handler that= =20 can't run while it waits for some async event (like disk I/O) to complete. =2D-=20 John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200706051539.22662.jhb>