Date: Mon, 18 Jul 2005 19:59:04 +0100 (BST) From: Robert Watson <rwatson@FreeBSD.org> To: arch@FreeBSD.org Subject: Network-related device drivers and MPSAFEty: status, and plans for 7.0 Message-ID: <20050718183717.N9430@fledge.watson.org>
next in thread | raw e-mail | index | archive | help
Just in case you missed it, a significant activity in the FreeBSD source tree for the past five years has been SMPng, the movement from a single "Giant" lock around the kernel to alternative synchronization models that include atomic operations, fine-grained locking, per-CPU data structures, etc. A significant part of this work was to remove the Giant lock from the network stack, where it has now been largely removed. The remaining sections consist of: - Legacy network protocol code which needs to be removed or fixed. This includes KAME IPSEC (not FAST_IPSEC), some older ATM portions, and some tty-related parts (SLIP, kernel PPP). For most of these portions, which I'll send e-mail about in a week or two to cover the state of, we now either have owners or a plan. There are some notable exceptions. - A number of network device drivers, which run under a compatibility mode that requires acquiring Giant in the inbound path, and possibly deferring work to a Giant-holding context in the outbound path. This e-mail is about the second category -- device drivers which still run with or require Giant. These in turn can be broken out into two categories: - Device drivers that appear to have been modified to run MPSAFE, and are possibly running partly Giant-free (i.e., perhaps the outbound path but not interrupt handler, or vice versa). - Device drivers that haven't been touched and rely on Giant entirely. The plan for FreeBSD 7.0 (i.e., HEAD) is that all network device drivers will be required to *operate without Giant*. This is necessary because all network protocol parts will be running without Giant, and the compatibility shims that allow older device drivers to run with Giant will be removed. Obviously, the details are subject to discussion, but I thought it would be helpful to do a sweep of the drivers and identify ones that either require some further work, or are definitely on the chopping board. Nothing will be removed until within a few months of the RELENG_7 branch, so we have quite a bit of time (about a year) to get this cleaned up before device drivers start disappearing. This will not be my last e-mail on the topic, you will get intermittent updates on how things are going. Specifically, however: - All network device drivers should be running INTR_MPSAFE (exceptions for device driver frameworks that don't support it). - No network device drivers should be running IFF_NEEDSGIANT (possibly excepting device driver frameworks that don't support it). First, a list of device drivers in the "partly or mostly done" category. Some require a lot of testing and work; others require a bit of code deletion or changing a flag and a bit of testing. If you are the maintainer/owner of one of these drivers, or play one on TV, I would love to see this work done by the end of August. And ideally, I'd love to see it MFC'd to RELENG_6. In general, this is low-risk work. if_an Aironet Communications 4500/4800 wireless network adapter driver. Mutexes present and exercised; not marked IFF_NEEDSGIANT, but not marked INTR_MPSAFE. Maybe we can just throw the switch? if_cp Cronyx Tau-PCI WAN adapters -- runs MPSAFE, but still has frobts to run un-MPSAFE. That should be easy to fix. if_ct Cronyx-Tau adapter -- runs MPSAFE, but still has frobs to run un-MPSAFE. That should be easy to fix. if_cx Driver for synchronous/asynchronous Cronyx Sigma WAN adapters -- runs MPSAFE, but still has frobs to run un-MPSAFE. That should be easy to fix :-). if_dc Tulip driver -- mutexes present, not used unless IS_MPSAFE set to non-0, so un-exercised. if_el Ethernet driver for 3Com Etherlink 3C501 devices. Mutxes present and exercised, not marked MPSAFE. if_en Device driver for Midway-based ATM interfaces. Mutexes present and exercised; not marked IFF_NEEDSGIANT, but not marked INTR_MPSAFE. Maybe we can just throw the switch? if_fatm Device driver for Fore PCA200E ATM interfaces. Mutexes present and exercised; not marked IFF_NEEDSGIANT, but not marked INTR_MPSAFE. Maybe we can just throw the switch? if_fxp Intel EtherExpress PRO/100 Ethernet device driver. Mutexes present and exercised; not marked IFF_NEEDSGIANT, but not marked INTR_MPSAFE (see if_fxpvar.h#define). Also, Giant passed into busdma for call-back lock. Maybe we can just throw the switch? if_hatm Device driver for Fore/Marconi HE155 and HE622 ATM interfaces. Appears MPSAFE and ready to go, only it looks like it uses a non-default kenv value to key MPSAFEty. Maybe we can just throw the switch? if_hme Sun Microelectronics STP2002-STQ Ethernet interfaces device driver. Generally fine, but Giant referenced in budma callback. Can this be fixed easily? if_my Myson Technology Ethernet PCI driver. Mutexes present and exercised, not marked IFF_NEEDSGIANT, but not not marked INTR_MPSAFE. Maybe we can just throw the switch? if_nve NVIDIA nForce MCP Networking Adapter device driver. Mutexes present and exercised; not marked IFF_NEEDSGIANT, but not marked INTR_MPSAFE. Also, busdma callbacks reference Giant. Maybe we can just throw the switch? if_patm Device driver for IDT77252 based ATM interfaces (ProSum and IDT). Mutexes present and exercised, not marked IFF_NEEDSGIANT, but not marked INTR_MPSAFE. Maybe we can just throw the switch? if_pcn AMD Am79c972 fast ethernet PCI NIC driver. Mutexes present and exercised, not marked MPSAFE. if_pdq DEC PDQ FDDI Controller. Mutexes present and exercised. Not marked IFF_NEEDSGIANT, but also not marked INTR_MPSAFE. A property of EISA MPSAFEty? if_sf Adaptec AIC-6915 "Starfire" PCI fast ethernet driver. Mutexes present and exercised, not marked MPSAFE. if_ste Sundance Technologies ST201 Fast Ethernet device driver. Mutexes present and exercised, not marked MPSAFE. if_ti Alteon Networks Tigon PCI gigabit ethernet driver. Mutexes present and exercised, not marked MPSAFE. if_tl Texas Instruments ThunderLAN Ethernet device driver. Mutexes present and exercised, not marked MPSAFE. if_wb Winbond fast ethernet PCI NIC driver. Mutexes present and exercised, not marked MPSAFE. if_owi Lucent Hermes, Intersil PRISM and Spectrum24 IEEE 802.11 driver (old). Mutexes present and exercised, not marked MPSAFE. if_aue, if_axe, if_cdce, if_cue, if_kue, if_rue, if_udav, if_ural Countless USB device drivers -- locking present, exercised, but all running under Giant, presumably due to the USB framework. Someone has a big TODO item here. if_wl T1 speed ISA/radio lan card. Mutexes present and exercised, not marked IFF_NEEDSGIANT, but not marked INTR_MPSAFE. Easy to fix? The second category was device drivers that basically haven't been touched at all with respect to synchronization since they were introduced into the tree, and in most cases, these drivers were introduced into the tree before SMPng began. These drivers are often aging ISA device drivers, and are often quite simple in structure so could be made MPSAFE quite easily, at least if you had the hardware on hand to test. There's probably quite a bit of this hardware floating around, so my expectation is that at least a third of these drivers will be made MPSAFE before I start having to start setting deadlines. If you have, or better yet, regularly use one of these drivers, please take a look at doing the work! Here they are: if_ar synchronous Digi/Arnet device driver -- has mutexes, but not used really. if_arl Aironet Arlan 655 wireless network adapter driver -- no locks at all. if_awi AMD PCnetMobile IEEE 802.11 PCMCIA wireless network driver -- no locks at all. if_cm SMC Arcnet Ethernet device driver -- no locks at all. if_cnw Netwave AirSurfer wireless network driver -- no locks at all. if_cs Crystal Semiconductor CS8900 and CS8920 NICs -- no locks at all. if_de DEC 21040 PCI Ethernet Controller -- no locks at all. if_ed NE-2000 and WD-80x3 Ethernet driver -- no locks at all. if_ex Ethernet device driver for the Intel EtherExpress Pro/10 and Pro/10+ -- macros defined, but no locking at all. if_fe Fujitsu MB86960A/MB86965A based Ethernet adapters -- no locks at all. if_fwe Ethernet emulation driver for FireWire -- no locks at all. if_fwip IP over FireWire driver -- no locks at all. if_gem ERI/GEM/GMAC Ethernet device driver -- no locks at all. if_harp Pseudo physical interface driver for HARP -- no locks at all. if_ie 8-bit and 16-bit ISA ethernet cards based on the Intel i82586 chip -- no locks at all. if_ic I2C bus IP driver -- no locks at all. if_lge Level 1 LXT1001 NetCellerator PCI Gigabit Ethernet adapter driver -- no locks at all. if_lnc AMD Lance/PCnet Ethernet device driver -- no locks at all. if_mn Unlocked netgraph node? if_oltr Olicom Token Ring device driver -- no locks at all. if_plip Printer port Internet Protocol driver -- no locks at all. if_ray Raytheon Raylink/Webgear Aviator PCCard driver -- no locks at all. if_sbsh Granch SBNI16 SHDSL modem device driver -- no locks at all. if_sbni Granch SBNI12 leased line modem driver -- no locks at all. if_snc National Semiconductor DP8393X SONIC Ethernet adapter driver -- no locks at all. if_sr Synchronous RISCom/N2 / WANic 400/405 device driver -- no locks at all. if_tx SMC 83c17x Fast Ethernet device driver -- no locks at all. if_txp 3Com 3XP Typhoon/Sidewinder (3CR990) Ethernet interface -- has mutexes, but they're not used. if_vx 3Com EtherLink III / Fast EtherLink III (3c59x) Ethernet driver -- no locks at all. if_xe Xircom PCMCIA Ethernet device driver -- no locks at all. This list might have a few mis-categorized, I might have missed a few device drivers, etc. If you are using a device driver and it's not marked MPSAFE, and it's also not on this list, you should still worry, and ideally do something about it. We have about a year -- that's lots of time, if people start now. :-) Robert N M Watson
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050718183717.N9430>