From owner-freebsd-current@FreeBSD.ORG Wed Sep 15 21:57:52 2010 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 44036106566C for ; Wed, 15 Sep 2010 21:57:52 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id BB8E98FC0A for ; Wed, 15 Sep 2010 21:57:51 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Ovzz6-0007Oe-Vd for freebsd-current@freebsd.org; Wed, 15 Sep 2010 23:57:45 +0200 Received: from k.saper.info ([91.121.151.35]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 15 Sep 2010 23:57:44 +0200 Received: from saper by k.saper.info with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 15 Sep 2010 23:57:44 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-current@freebsd.org From: Marcin Cieslak Date: Wed, 15 Sep 2010 21:57:34 +0000 (UTC) Organization: http://saper.info Lines: 86 Message-ID: References: <201009151749.45038.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: k.saper.info User-Agent: slrn/0.9.9p1 (FreeBSD) X-Mailman-Approved-At: Wed, 15 Sep 2010 22:04:06 +0000 Subject: Re: tun(4) in -CURRENT: No buffer space available - race condition patch X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Sep 2010 21:57:52 -0000 Dnia 15.09.2010 John Baldwin napisał/a: > On Monday, September 13, 2010 9:10:01 pm Marcin Cieslak wrote: >> Output queue of tun(4) gets full after some time when sending lots of data. >> I have been observing this on -CURRENT at least since March this year. >> >> Looks like it's a race condition (same in tun(4) and tap(4)), >> the following patch seems to address the issue: > > This is a good find. I actually went through these drivers a bit further and > have a bit of a larger patch to extend the locking some. Would you care to > test it? Sure, I am installing it right now (for if_tun right now). There are also a LORs during tunclose (I always get one of them when closing the tunnel): -------8<-------------------------------------------------------------------------- lock order reversal: 1st 0xc24db8c8 rtentry (rtentry) @ /usr/src/sys/net/route.c:370 2nd 0xc2472a04 if_afdata (if_afdata) @ /usr/src/sys/netinet6/scope6.c:417 KDB: stack backtrace: db_trace_self_wrapper(c0a4b284,cce14714,c0723185,c0715b2b,c0a4d1f8,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0715b2b,c0a4d1f8,c0c4f308,c21186e8,cce1476c,...) at kdb_backtrace+0x29 _witness_debugger(c0a4d1f8,c2472a04,c0a539ec,c211dfe0,c0a63771,...) at _witness_debugger+0x25 witness_checkorder(c2472a04,9,c0a63771,1a1,0,...) at witness_checkorder+0x6aa _rw_wlock(c2472a04,c0a63771,1a1,0,c2472a04,...) at _rw_wlock+0x38 in6_setscope(cce148c4,c2472800,0,cce147fc,cce147e0,...) at in6_setscope+0x30 in6_purgeaddr(c2539600,0,0,c211e048,c2362364,...) at in6_purgeaddr+0x4af if_purgeaddrs(c2472800,2,0,1cd,c24ab4c0,...) at if_purgeaddrs+0xb0 tunclose(c24d1000,3,2000,c23622c0,1,...) at tunclose+0x197 giant_close(c24d1000,3,2000,c23622c0,c23622c0,...) at giant_close+0x6e devfs_close(cce14a78,cce14a9c,c07785fb,c0aae500,cce14a78,...) at devfs_close+0x2a9 VOP_CLOSE_APV(c0aae500,cce14a78,c0a532d0,12d,c0af0160,...) at VOP_CLOSE_APV+0x42 vn_close(c24ccaa0,3,c2160c80,c23622c0,c23622c0,...) at vn_close+0xdb vn_closefile(c24bc0e0,c23622c0,c24bc0e0,0,cce14b28,...) at vn_closefile+0xe4 devfs_close_f(c24bc0e0,c23622c0,3,0,c24bc0e0,...) at devfs_close_f+0x2b _fdrop(c24bc0e0,c23622c0,cce14b5c,c072327c,0,...) at _fdrop+0x43 closef(c24bc0e0,c23622c0,721,71e,c2362364,...) at closef+0x277 fdfree(c23622c0,0,c0a4549c,107,80000000,...) at fdfree+0x3ba exit1(c23622c0,0,cce14c7c,c071da4a,c23622c0,...) at exit1+0x465 sys_exit(c23622c0,cce14cec,stray irq7 c06a7bac,1,0,...) at sys_exit+0x1d syscallenter(c23622c0,cce14ce4,c06d6stray irq7 d5d,c0cae934,8,...) at syscallenter+0x25a syscall(cce14d28) at syscall+0x34 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (1, FreeBSD ELF32, sys_exit), eipstray irq7 = 0x28128acf, esp = 0xbfbfed80, ebp = 0xbfbfed8c --- ^Ctun0: link state changed to DOWN -------8<-------------------------------------------------------------------------- lock order reversal: 1st 0xc24db8c8 rtentrystray irq7 (rtentry) @ /usr/src/sys/net/route.c:370 2nd 0xc2482604 if_afdata (if_afdata) @ /usr/src/sys/netinet6/scope6.c:417 KDB: stack backtrace: db_trace_self_wrapper(c0a4912e,cce16714,c0723105,c0715aab,c0a4b0a2,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0715aab,c0a4b0a2,c0c4cb58,c21176e8,cce1676c,...) at kdb_backtrace+0x29 _witness_debugger(c0a4b0a2,c2482604,c0a51896,c211cfe0,c0a6137e,...) at _witness_debugger+0x25 witness_checkorder(c2482604,9,c0a6137e,1a1,0,...) at witness_checkorder+0x6aa _rw_wlock(c2482604,c0a613stray irq7 7e,1a1,c0793997,c2482604,...) at _rw_wlock+0x38 in6_setscope(cce168c4,c2482400,0,cce167fc,cce167e0,...) at in6_setscope+0x30 in6_purgeaddr(c253e000,0,0,c211d048,c2361364,...) at in6_purgeaddr+0x4af if_purgeaddrs(c2482400,2,0,1cd,c24aa5c0,...) at if_purgeaddrs+0xb0 tunclose(c2539a00,3,2000,c23612c0,1,...) at tunclose+0x136 giant_close(c2539a00,3,2000,c23612c0,c23612c0,...) at giant_close+0x6e devfs_close(cce16a78,cce16a9c,c077857b,c0aac0e0,cce16a78,...) at devfs_close+0x2a9 VOP_CLOSE_APV(c0aac0e0,cce16a78,c0a5117a,12d,c0aedac0,...) at VOP_CLOSE_APV+0x42 vn_close(c25d2770,3,c215fc80,c23612c0,c0b10180,...) at vn_close+0xdb vn_closefile(c24bba10,c23612c0,c24bba10,0,cce16b28,...) at vn_closefile+0xe4 devfs_close_f(c24bba10,c23612c0,3,0,c24bba10,...) at devfs_close_f+0x2b _fdrop(c24bba10,c23612c0,cce16b5c,c07231fc,0,...) at _fdrop+0x43 closef(c24bba10,c23612c0,721,71e,c2361364,...) at closef+0x277 fdfree(c23612c0,0,c0a43346,107,c2361364,...) at fdfree+0x3ba exit1(c23612c0,0,cce16c7c,c071d9ca,c23612c0,...) at exit1+0x465 sys_exit(c23612c0,cce16cec,28087460,1,0,...) at sys_exit+0x1d syscallenter(c23612c0,cce16ce4,c09a564e,c23612c0,cce16d28,...) at syscallenter+0x25a syscall(cce16d28) at syscall+0x34 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (1, FreeBSD ELF32, sys_exit), eip = 0x28128acf, esp = 0xbfbfed80, ebp = 0xbfbfed8c --- tun0: link state changed to DOWN -------8<-------------------------------------------------------------------------- --Marcin