From owner-freebsd-arch@FreeBSD.ORG Fri Sep 24 09:01:08 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CB1AB16A4CF for ; Fri, 24 Sep 2004 09:01:08 +0000 (GMT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1614643D53 for ; Fri, 24 Sep 2004 09:01:08 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.13.1/8.13.1) with ESMTP id i8O9158C041459 for ; Fri, 24 Sep 2004 11:01:06 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: arch@freebsd.org From: "Poul-Henning Kamp" Date: Fri, 24 Sep 2004 11:01:05 +0200 Message-ID: <41458.1096016465@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Subject: I'm counting my threads, one, two, three, four, five... [1] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2004 09:01:08 -0000 I've completed my sweep of the kernel and I belive that we now have a reliable count of the number of threads currently inside any particular cdev and consequently, with a bit of adding up, also for each cdevsw. There are two soft spots marked with XXX (devfs_rule and vm_mmap) and one big hack, snp(4). The next step is to add a new method to the cdevsw which purges threads from the driver. The best name I can come up with is d_evict(). The drivers should make this function wakeup(9) on anything a thread could be sleeping on for the relevant cdev and the tread on waking up should examine whatever bits it can, realize the hw/driver/whatever is gone and exit the driver with ENXIO error return. If this cdevsw method is present, destroy_dev() will use it and will sleep until all threads have cleared out. So for a typical driver things will look like this: foo_evict(struct cdev *dev) { struct foo_sc *sc; /* XXX: NB: no locks, only wakeups */ sc = dev->si_drv1; wakeup(sc->bing); wakeup(sc->bongle); wakeup(sc->bang); } foo_detach(dev) { ... sc->flags |= GONE; destroy_dev(sc->cdev); /* XXX: we now know there are no threads involved with this cdev */ ... } destroy_dev() will work more or less this way: ... dev_lock(); csw = cdev->si_devsw; cdev->si_devsw = NULL while (csw->d_evict != NULL && cdev->si_threacount > 0) { (csw->d_evict)(cdev); msleep(&dev_lock, ..., hz/10); } dev_unlock(); ... I will also add a convenience function called destroy_cdevsw() which will call destroy_dev() on all cdevs using the cdevsw in question. I belive this gives us the handle we need to unload drivers and remove hardware without panicing in the lower layers of the kernel. The higher layers may still have a thing or two to learn in this respect. Poul-Henning [1] It used to be possible to legally download the D:A:D tune which inspired the subject, but I can't find the link anymore, sorry :-( -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.