From owner-freebsd-bugs@FreeBSD.ORG Sat Dec 15 13:20:03 2007 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 120B216A46C for ; Sat, 15 Dec 2007 13:20:03 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id DB96A13C47E for ; Sat, 15 Dec 2007 13:20:02 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id lBFDK2Hj013552 for ; Sat, 15 Dec 2007 13:20:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id lBFDK2n1013551; Sat, 15 Dec 2007 13:20:02 GMT (envelope-from gnats) Resent-Date: Sat, 15 Dec 2007 13:20:02 GMT Resent-Message-Id: <200712151320.lBFDK2n1013551@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Dan Lukes Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F5BA16A420 for ; Sat, 15 Dec 2007 13:19:32 +0000 (UTC) (envelope-from dan@kulesh.obluda.cz) Received: from smtp1.kolej.mff.cuni.cz (smtp1.kolej.mff.cuni.cz [78.128.192.4]) by mx1.freebsd.org (Postfix) with ESMTP id A488813C442 for ; Sat, 15 Dec 2007 13:19:31 +0000 (UTC) (envelope-from dan@kulesh.obluda.cz) Received: from kulesh.obluda.cz (openvpn.ms.mff.cuni.cz [195.113.20.87]) by smtp1.kolej.mff.cuni.cz (8.13.8/8.13.8) with ESMTP id lBFDJCpP020958 for ; Sat, 15 Dec 2007 14:19:16 +0100 (CET) (envelope-from dan@kulesh.obluda.cz) Received: from kulesh.obluda.cz (localhost. [127.0.0.1]) by kulesh.obluda.cz (8.14.2/8.14.2) with ESMTP id lBFDJAXU001392 for ; Sat, 15 Dec 2007 14:19:10 +0100 (CET) (envelope-from dan@kulesh.obluda.cz) Received: (from dan@localhost) by kulesh.obluda.cz (8.14.2/8.14.1/Submit) id lBFDJAcF001391; Sat, 15 Dec 2007 14:19:10 +0100 (CET) (envelope-from dan) Message-Id: <200712151319.lBFDJAcF001391@kulesh.obluda.cz> Date: Sat, 15 Dec 2007 14:19:10 +0100 (CET) From: Dan Lukes To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Cc: Subject: kern/118719: 'Giant not owned at ...' with 're' interface X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Dan Lukes List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2007 13:20:03 -0000 >Number: 118719 >Category: kern >Synopsis: 'Giant not owned at ...' with 're' interface >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Sat Dec 15 13:20:01 UTC 2007 >Closed-Date: >Last-Modified: >Originator: Dan Lukes >Release: FreeBSD 6.3-PRERELEASE i386 >Organization: Obludarium >Environment: FreeBSD 6.3-PRERELEASE i386 src/sys/net/if.c,v 1.234.2.21 2007/07/13 01:26:44 thompsa src/sys/security/mac/mac_net.c,v 1.117 2005/07/05 23:39:50 rwatson sys/net/bpf.c,v 1.153.2.12 2007/11/03 17:13:16 csjp src/sys/net/if_ethersubr.c,v 1.193.2.15 2007/09/17 17:50:49 julian src/sys/net/bpfdesc.h,v 1.29.2.3 2007/01/19 23:01:31 jhb src/sys/sys/mutex.h,v 1.79.2.4 2006/08/01 18:38:35 jhb src/sys/dev/re/if_re.c,v 1.46.2.36 2007/12/06 06:01:47 yongari src/sys/kern/subr_bus.c,v 1.184.2.6 2007/11/05 11:49:44 phk src/sys/kern/kern_intr.c,v 1.124.2.8 2007/10/29 21:10:03 emaste Kernel compiled with IPSEC (=> MPSAFE network stack forced disabled; IPSEC require Giant) Kernel compiled with INVARIANTS and INVARIANT_SUPPORT so missing lock trigger panic **** **** **** **** **** **** **** **** NOTE: This doesn't apply for CURRENT, but may be significant for 6.3-RELEASE / 6-STABLE **** **** **** **** **** **** **** **** >Description: Panic triggered every time ( a packet arrive on 're' interface) AND (a bpf is active) (for example - dhclient use bpf) panic: mutex Giant not owned at #if MAC option compiled in mac_net.c:382 #else bpf.c:1345 #fi On both places it is the BPFD_LOCK_ASSERT macro that call the panic() NOTE - there seems not to be changes in the if_re.c in past few weeks that can cause that problem - so the problem may affect all MP_SAFE network card drivers - but I'm didn't test it. Unfortunatelly, the kernel locks during memory dump, so I have no exact backtrace, but I hope I can reconstruct the possible flow by self. mac_net.c:382 = mac_check_bpfdesc_receive() bpf.c:1345 = catchpacket() they are called from bpf_mtap() or bpf_mtap2() they are called from ether_vlan_mtop() which is part of ETHER_BPF_MTAP macro the macro is used within device driver if_input method this method is called from receiving interrupt service routine in the 're' driver is interrupt declared as MP_SAFE so the Giant is not locked by the core. Is it isn't acquired later as weel, the panic will be triggered. >How-To-Repeat: Compile kernel with INVARIANTS/INVARIANT_SUPPORT + (IPSEC or set mpsafenet to 0) use dhclient or tcpdump on 're' interface. Wait for a packet >Fix: Workaround is simple: don't use IPSEC or BPF or use a IFF_NEEDSGIANT devices only you may also remove INVARIANTS to avoid panic on error, but race contition may occur if the Giant is really needed here Fix: we need to have Giant acquired in the packet input path, unless someone smarter than me claimed it's not necesarry to have it - then we need correct the within bpf's catchpacket() and mac's mac_check_bpfdesc_received routines The re's bus_setup_intr() is called with INTR_MPSAFE | INTR_FAST flags The INTR_MPSAFE is cleared by bus_setup_intr's logic, but INTR_FAST remain active. Then the re's ISR routine will be called without Giant unfortunatelly, I don know where is exact point of problem - if mpsafenet inactive we need either ----------- [1] bus_setup_intr shall clear the INTR_FAST as well * or * [2] Giant shall be acquired before calling of driver's ISR routine even if IH_FAST active * or * [3] correct the driver's ISR as it is responsible to acquire Giant by self ----------- in the later case I would like to note the msk and em drivers may have the same problem as they use INTR_FAST also but I didn't tried them >Release-Note: >Audit-Trail: >Unformatted: