Date: Wed, 17 Nov 2010 21:38:12 +0000 From: Alexander Best <arundel@freebsd.org> To: John Baldwin <jhb@freebsd.org> Cc: freebsd-current@freebsd.org Subject: Re: kldunload(8) returns 0, although it fail Message-ID: <20101117213812.GA45491@freebsd.org> In-Reply-To: <20101117171154.GA1398@freebsd.org> References: <20101109114612.GA58585@freebsd.org> <201011101717.51062.jhb@freebsd.org> <20101117032012.GA93866@freebsd.org> <201011171006.10486.jhb@freebsd.org> <20101117171154.GA1398@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--ZPt4rx8FFjLCG7dd
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
On Wed Nov 17 10, Alexander Best wrote:
> On Wed Nov 17 10, John Baldwin wrote:
> > On Tuesday, November 16, 2010 10:20:12 pm Alexander Best wrote:
> > > On Wed Nov 10 10, John Baldwin wrote:
> > > > On Wednesday, November 10, 2010 5:07:21 pm Alexander Best wrote:
> > > > > On Wed Nov 10 10, Alexander Best wrote:
> > > > > > On Tue Nov 9 10, John Baldwin wrote:
> > > > > > > On Tuesday, November 09, 2010 9:10:28 am Alexander Best wrote:
> > > > > > > > On Tue Nov 9 10, John Baldwin wrote:
> > > > > > > > > On Tuesday, November 09, 2010 6:46:12 am Alexander Best wrote:
> > > > > > > > > > hi there,
> > > > > > > > > >
> > > > > > > > > > i posted this message on freebsd-questions@, but nobody could help me with it.
> > > > > > > > > > to me this looks like a bug, so i assume posting it again here on
> > > > > > > > > > freebsd-current@ might be better.
> > > > > > > > > >
> > > > > > > > > > please keep in mind that the issue here is not the fact that the second attempt
> > > > > > > > > > to unload sound.ko/netgraph.ko fails. it *should* fail, because both modules
> > > > > > > > > > have dependencies. however it should fail with the first attempt. right now
> > > > > > > > > > kldunloadf() returns zero, whereas it should actually return EBUSY (just like
> > > > > > > > > > the second attempt).
> > > > > > > > > >
> > > > > > > > > > i've attached two kdump outputs: one for the first 'kldunload' attempt and one
> > > > > > > > > > for the second. as you can see the problem is that for some reason kldunloadf()
> > > > > > > > > > returns zero, although it couldn't unload the module.
> > > > > > > > >
> > > > > > > > > Did you get an error message in dmesg? If you have manually loaded
> > > > > > > > > netgraph.ko (via netgraph_load="YES" in loader.conf or an explicit kldload)
> > > > > > > > > and then loaded other modules that depend on it (such as ng_foo.ko) then this
> > > > > > > > > is expected behavior. What your kldunload has done is to remove the manual
> > > > > > > > > reference from loader.conf or 'kldload netgraph.ko'. What this changes is what
> > > > > > > > > happens when you do 'kldunload ng_foo.ko'. If you unload ng_foo.ko now, then
> > > > > > > > > netgraph.ko will also be unloaded when its last reference drops (in this case
> > > > > > > > > it looks like you actually have two ng_*.ko objects loaded, so you would have
> > > > > > > > > to unload both of them, but I will assume a single ng_foo.ko to make the
> > > > > > > > > explanation simpler). If you had not done 'kldunload netgraph.ko' but had
> > > > > > > > > done 'kldunload ng_foo.ko', then the manual reference would have kept netgraph.ko
> > > > > > > > > loaded.
> > > > > > > > >
> > > > > > > > > All this logic exists so that if you do 'kldload foo.ko' and it auto-loads bar.ko
> > > > > > > > > as a dependency, then doing 'kldunload bar.ko' will unload both foo.ko and bar.ko.
> > > > > > > >
> > > > > > > > i have ng_ubt_load="YES" in my loader.conf and kldstat says:
> > > > > > > >
> > > > > > > > Id Refs Address Size Name
> > > > > > > > 1 37 0xffffffff80100000 a2ddb8 kernel
> > > > > > > > 2 1 0xffffffff80b2e000 295e8 snd_hda.ko
> > > > > > > > 3 1 0xffffffff80b58000 85110 sound.ko
> > > > > > > > 4 1 0xffffffff80bde000 cf79e0 nvidia.ko
> > > > > > > > 5 5 0xffffffff818d6000 418c0 linux.ko
> > > > > > > > 6 1 0xffffffff81918000 80e8 ng_ubt.ko
> > > > > > > > 7 2 0xffffffff81921000 fa78 ng_hci.ko
> > > > > > > > 8 2 0xffffffff81931000 2bd0 ng_bluetooth.ko
> > > > > > > > 9 3 0xffffffff81934000 15e68 netgraph.ko
> > > > > > > > 10 1 0xffffffff81a12000 3efb linprocfs.ko
> > > > > > > > 11 3 0xffffffff81a16000 4698 pseudofs.ko
> > > > > > > > 12 1 0xffffffff81a1b000 31b3 procfs.ko
> > > > > > > > 13 1 0xffffffff81a1f000 a37 linsysfs.ko
> > > > > > > > 14 1 0xffffffff81a20000 6f4 rtc.ko
> > > > > > > >
> > > > > > > > also the same happens with sound.ko. i have snd_hda_load="yes". and i think
> > > > > > > > snd_hda is the only dependecy for sound.ko.
> > > > > > > >
> > > > > > > > there was no error message in dmesg for the first kldunload attempt.
> > > > > > > >
> > > > > > > > i think i understand the logic, however the current behavior does not confirm
> > > > > > > > to the description in kldunload(2).
> > > > > > >
> > > > > > > Hmm, ok, fair enough. Do you get the same behavior if you kldload ng_ubt.ko
> > > > > > > after boot?
> > > > >
> > > > > just tested and the behavior is *not* the same. when i do kldload ng_ubt after
> > > > > boot everything seems fine. netgraph gets loaded and has a REF count of 2. if i
> > > > > try doing `kldunload netgraph` i get EBUSY the first time i run that command.
> > > > > when i add ng_ubt_load=yes to /boot/loader.conf kldstat reports a REF count of
> > > > > 3 for netgraph.ko and i get the behavior i described beforehand.
> > > >
> > > > Try doing 'kldload netgraph.ko' and then 'kldload ng_ubt.ko' (along with manually
> > > > loading any other netgraph modules so that nothing is loaded magically as a
> > > > dependency) and see what behavior that gives you.
>
> i did some more research regarding this matter:
>
> 1) DEPENDENCIES:
>
> netgraph => X (none)
> ng_ubt => ng_hci
> ng_hci => ng_bluetooth
> ng_buetooth => X (none; [no not even netgraph])
>
> 2) VARIOUS results:
>
> **
> kldload netgraph => REF_COUNT_NETGRAPH = 1
> kldunload netgraph => EBUSY, REF_COUNT_NETGRAPH = 1
> **
>
> **
> kldload netgraph => REF_COUNT_NETGRAPH = 1
> kldload ng_bluetooth => REF_COUNT_NETGRAPH = 1
> kldunload netgraph => EBUSY, REF_COUNT_NETGRAPH = 1
> (obvious, since ng_bluetooth doesn't depend on netgraph)
> **
>
> **
> kldload netgraph => REF_COUNT_NETGRAPH = 1
> kldload ng_bluetooth => REF_COUNT_NETGRAPH = 1
> kldload ng_hci => REF_COUNT_NETGRAPH = 2
> kldunload netgraph => OK, REF_COUNT_NETGRAPH = 1
> kldunload netgraph => EBUSY, REF_COUNT_NETGRAPH = 1 [1]
> **
>
> **
> kldload netgraph => REF_COUNT_NETGRAPH = 1
> kldload ng_bluetooth => REF_COUNT_NETGRAPH = 1
> kldload ng_hci => REF_COUNT_NETGRAPH = 2
> kldload ng_ubt => REF_COUNT_NETGRAPH = 3
> kldunload netgraph => OK, REF_COUNT_NETGRAPH = 2
> kldunload netgraph => EBUSY, REF_COUNT_NETGRAPH = 2 [1]
> **
>
> **
> kldload netgraph => REF_COUNT_NETGRAPH = 1
> kldload ng_bluetooth => REF_COUNT_NETGRAPH = 1
> kldload ng_hci => REF_COUNT_NETGRAPH = 2
> kldunload netgraph => OK, REF_COUNT_NETGRAPH = 1
> kldunload netgraph => EBUSY, REF_COUNT_NETGRAPH = 1 [1]
> kldload ng_ubt => REF_COUNT_NETGRAPH = 2
> kldunload netgraph => EBUSY, REF_COUNT_NETGRAPH = 2 [1]
> **
>
> **
> kldload netgraph => REF_COUNT_NETGRAPH = 1
> kldload ng_bluetooth => REF_COUNT_NETGRAPH = 1
> kldload ng_hci => REF_COUNT_NETGRAPH = 2
> kldunload netgraph => OK, REF_COUNT_NETGRAPH = 1
> kldload ng_ubt => REF_COUNT_NETGRAPH = 2
> kldunload netgraph => EBUSY, REF_COUNT_NETGRAPH = 2 [1]
> **
>
> [1] before EBUSY gets returned i see a notice that netgraph.ko cannot be
> unloaded, since it was loaded by the kernel. this doesn't happen in the
> last two cases.
>
> > >
> > > sorry it took me a while to check this out. i did some more research and it
> > > seems a few things are broken:
> > >
> > > 1) i have snd_hda in my loader.conf which will load sound as a dependency.
> > > unloading snd_hda should then automatically unload sound, which it *doesn't*.
> > > i need to unload sound manually.
> >
> > I clearly said earlier that this is what happens. It's the paragraph below
> > where I said that the kernel has no idea when it discovers the modules that
> > the loader loaded which ones were explicit and which ones were implicit. It
> > treats them all as explicit.
the following patch changes the semantics of kldunload to deal with removing
modules after all their dependencies have been removed too.
this is just a quick hack and probably contains several style(9) bugs.
cheers.
alex
>
> i'm really sorry. i completely forgot about that fact. so to make this easy for
> me to finally comprehend:
>
> if a module is loaded via loader.conf *no* implicit dependencies are present
> and unloading a module from a chain that got triggered by loader.conf will
> *never* unload anything except that one module i specified!
>
> > >
> > > 2) if i unload all ng_* modules i should then be able to also unload netgraph,
> > > because no dependencies are left. however kldunload fails with EBUSY!
> >
> > Perhaps netgraph.ko is not unloadable in general? Actually, that is exactly what
> > it does:
> >
> > case MOD_UNLOAD:
> > /* You can't unload it because an interface may be using it. */
> > error = EBUSY;
> > break;
> >
> > (from ng_base.c).
> >
> > > > When you load modules from the loader, they and all their dependencies are
> > > > treated as if you had manually loaded them similar to kldload. This because
> > > > the kernel can't determine which modules loaded by the loader were silent
> > > > dependencies.
> >
> > --
> > John Baldwin
>
> --
> a13x
--
a13x
--ZPt4rx8FFjLCG7dd
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="kldunload.diff"
diff --git a/sbin/kldunload/kldunload.c b/sbin/kldunload/kldunload.c
index 8a3ea61..50e215c 100644
--- a/sbin/kldunload/kldunload.c
+++ b/sbin/kldunload/kldunload.c
@@ -31,6 +31,7 @@ __FBSDID("$FreeBSD$");
#include <sys/linker.h>
#include <err.h>
+#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <sysexits.h>
@@ -53,7 +54,7 @@ int
main(int argc, char** argv)
{
struct kld_file_stat stat;
- int c, fileid, force, opt;
+ int c, error, fileid, force, opt;
char *filename;
filename = NULL;
@@ -109,7 +110,14 @@ main(int argc, char** argv)
else
force = LINKER_UNLOAD_NORMAL;
- if (kldunloadf(fileid, force) < 0)
+ error = kldunloadf(fileid, force);
+
+ if (error < 0 && errno == EDOOFUS) {
+ warnx("module scheduled for removal when refcount drops "
+ "to zero");
+ errno = EBUSY;
+ }
+ if (error < 0)
err(EXIT_FAILURE, "can't unload file");
}
diff --git a/sys/kern/kern_linker.c b/sys/kern/kern_linker.c
index 43ccd74..82128d0 100644
--- a/sys/kern/kern_linker.c
+++ b/sys/kern/kern_linker.c
@@ -62,6 +62,7 @@ __FBSDID("$FreeBSD$");
#include <sys/pmckern.h>
#endif
+#define KLD_DEBUG
#ifdef KLD_DEBUG
int kld_debug = 0;
SYSCTL_INT(_debug, OID_AUTO, kld_debug, CTLFLAG_RW,
@@ -592,7 +593,7 @@ linker_file_unload(linker_file_t file, int flags)
/* Easy case of just dropping a reference. */
if (file->refs > 1) {
file->refs--;
- return (0);
+ return (EDOOFUS);
}
KLD_DPF(FILE, ("linker_file_unload: file is unloading,"
@@ -1095,7 +1096,7 @@ kern_kldunload(struct thread *td, int fileid, int flags)
#endif
lf->userrefs--;
error = linker_file_unload(lf, flags);
- if (error)
+ if (error && error != EDOOFUS)
lf->userrefs++;
}
} else
--ZPt4rx8FFjLCG7dd--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20101117213812.GA45491>