From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 20 21:22:56 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B032010656A9 for ; Sat, 20 Nov 2010 21:22:56 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3561B8FC15 for ; Sat, 20 Nov 2010 21:22:55 +0000 (UTC) Received: by bwz2 with SMTP id 2so5133140bwz.13 for ; Sat, 20 Nov 2010 13:22:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:cc:subject:references :x-comment-to:date:in-reply-to:message-id:user-agent:mime-version :content-type; bh=No93VM1raPx+yLAxA5bq0kZJ+e94EWBrlKn7gf9yvPk=; b=sIt39lV+VL6RJYrxLU0FX906qf5UDmN3TgYXvZ2HtduXWUt+dutQVmki4gxcNmT9DD QN4R7LDDVyxFUe9Jtp59anDnf1ErkcQJPHCRqliiIbD8UORcPdqh1cfclBCLooTgkoK8 eBkPGe23hQKLyupSPmjvQ7eQez0jv2wAI5YBw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:subject:references:x-comment-to:date:in-reply-to :message-id:user-agent:mime-version:content-type; b=S3R9FpPUoKwOnivmsi8DQd8u/aQ2X9YiaN7CtU+t0jkfZT3Wz57k9ep6+WsO0os8lK CqjwuqO80wHn5/R9kv/jkfVTjx9fsg7iT9VosRwpU7skVRNyueQtePMkFZMoIX6nqHDx Sir9EyYe+BTfvWm9VOxzWMh6+gYEaRPEtjByY= Received: by 10.204.100.79 with SMTP id x15mr3611115bkn.10.1290288174936; Sat, 20 Nov 2010 13:22:54 -0800 (PST) Received: from localhost ([95.69.174.185]) by mx.google.com with ESMTPS id p34sm1554340bkf.15.2010.11.20.13.22.52 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sat, 20 Nov 2010 13:22:53 -0800 (PST) From: Mikolaj Golub To: "Bjoern A. Zeeb" References: <86pqu0nexd.fsf@kopusha.home.net> <20101120165604.T24596@maildrop.int.zabbadoz.net> <868w0nolqo.fsf@kopusha.home.net> <20101120200208.M24596@maildrop.int.zabbadoz.net> X-Comment-To: Bjoern A. Zeeb Date: Sat, 20 Nov 2010 23:22:50 +0200 In-Reply-To: <20101120200208.M24596@maildrop.int.zabbadoz.net> (Bjoern A. Zeeb's message of "Sat, 20 Nov 2010 20:04:35 +0000 (UTC)") Message-ID: <864obbofkl.fsf@kopusha.home.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Cc: freebsd-hackers@freebsd.org Subject: Re: flowtable_cleaner/flowtable_flush livelock X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Nov 2010 21:22:56 -0000 --=-=-= On Sat, 20 Nov 2010 20:04:35 +0000 (UTC) Bjoern A. Zeeb wrote: BAZ> How do you reproduce the crash? Is it just another ifioctl race as BAZ> from kern/146250? Using the same script I posted in my first mail, removing a jail and epair interface simultaneously: ifconfig epair0b vnet myjail jail -r myjail & ifconfig epair0a destroy For me it loooks like other thread is destroying interface improperly in that time. One time I saw crash in another thread: (kgdb) bt #0 doadump () at pcpu.h:231 #1 0xc04f2439 in db_fncall (dummy1=1, dummy2=0, dummy3=-1056689728, dummy4=0xc2ba5984 "") at /usr/src/sys/ddb/db_command.c:548 #2 0xc04f2831 in db_command (last_cmdp=0xc0e75cfc, cmd_table=0x0, dopager=1) at /usr/src/sys/ddb/db_command.c:445 #3 0xc04f298a in db_command_loop () at /usr/src/sys/ddb/db_command.c:498 #4 0xc04f48ad in db_trap (type=12, code=0) at /usr/src/sys/ddb/db_main.c:229 #5 0xc090face in kdb_trap (type=12, code=0, tf=0xc2ba5bf8) at /usr/src/sys/kern/subr_kdb.c:546 #6 0xc0c3d2bf in trap_fatal (frame=0xc2ba5bf8, eva=3735929066) at /usr/src/sys/i386/i386/trap.c:971 #7 0xc0c3d4f0 in trap_pfault (frame=0xc2ba5bf8, usermode=0, eva=3735929066) at /usr/src/sys/i386/i386/trap.c:893 #8 0xc0c3dca5 in trap (frame=0xc2ba5bf8) at /usr/src/sys/i386/i386/trap.c:568 #9 0xc0c24a9c in calltrap () at /usr/src/sys/i386/i386/exception.s:168 #10 0xc09ad219 in vnet_destroy (vnet=0xc2f24240) at /usr/src/sys/net/vnet.c:284 #11 0xc08b5922 in prison_deref (pr=0xc3640800, flags=Variable "flags" is not available. ) at /usr/src/sys/kern/kern_jail.c:2506 #12 0xc08b5ab0 in prison_complete (context=0xc3640800, pending=1) at /usr/src/sys/kern/kern_jail.c:2433 #13 0xc091c87b in taskqueue_run_locked (queue=0xc2dd6d80) at /usr/src/sys/kern/subr_taskqueue.c:247 #14 0xc091cf17 in taskqueue_thread_loop (arg=0xc0ebb8e8) at /usr/src/sys/kern/subr_taskqueue.c:379 #15 0xc08af558 in fork_exit (callout=0xc091ceb0 , arg=0xc0ebb8e8, frame=0xc2ba5d28) at /usr/src/sys/kern/kern_fork.c:835 #16 0xc0c24b44 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:275 (kgdb) fr 10 #10 0xc09ad219 in vnet_destroy (vnet=0xc2f24240) at /usr/src/sys/net/vnet.c:284 284 TAILQ_FOREACH_SAFE(ifp, &V_ifnet, if_link, nifp) { (kgdb) list 279 VNET_LIST_WUNLOCK(); 280 281 CURVNET_SET_QUIET(vnet); 282 283 /* Return all inherited interfaces to their parent vnets. */ 284 TAILQ_FOREACH_SAFE(ifp, &V_ifnet, if_link, nifp) { 285 if (ifp->if_home_vnet != ifp->if_vnet) 286 if_vmove(ifp, ifp->if_home_vnet); 287 } 288 (kgdb) p ifp $1 = (struct ifnet *) 0xdeadc0de Doesn't this need some lock protection? I tried the attached patch, but still observed crashes in ifioctl I posted earlier. -- Mikolaj Golub --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=vnet.c.patch Index: sys/net/vnet.c =================================================================== --- sys/net/vnet.c (revision 215576) +++ sys/net/vnet.c (working copy) @@ -268,7 +268,7 @@ vnet_alloc(void) void vnet_destroy(struct vnet *vnet) { - struct ifnet *ifp, *nifp; + struct ifnet *ifp; SDT_PROBE2(vnet, functions, vnet_destroy, entry, __LINE__, vnet); KASSERT(vnet->vnet_sockcnt == 0, @@ -281,10 +281,20 @@ vnet_destroy(struct vnet *vnet) CURVNET_SET_QUIET(vnet); /* Return all inherited interfaces to their parent vnets. */ - TAILQ_FOREACH_SAFE(ifp, &V_ifnet, if_link, nifp) { - if (ifp->if_home_vnet != ifp->if_vnet) + do { + IFNET_RLOCK(); + TAILQ_FOREACH(ifp, &V_ifnet, if_link) { + if (ifp->if_home_vnet != ifp->if_vnet) { + if_ref(ifp); + break; + } + } + IFNET_RUNLOCK(); + if (ifp != NULL) { if_vmove(ifp, ifp->if_home_vnet); - } + if_rele(ifp); + } + } while (ifp != NULL); vnet_sysuninit(); CURVNET_RESTORE(); --=-=-=--