From owner-freebsd-net@FreeBSD.ORG Sat Jan 24 17:53:32 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5ACC346; Sat, 24 Jan 2015 17:53:32 +0000 (UTC) Received: from mail-lb0-x234.google.com (mail-lb0-x234.google.com [IPv6:2a00:1450:4010:c04::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5431BC18; Sat, 24 Jan 2015 17:53:32 +0000 (UTC) Received: by mail-lb0-f180.google.com with SMTP id b6so2263774lbj.11; Sat, 24 Jan 2015 09:53:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=wtkApJZha50vu09vaYLrpnBs9AmWpti9Nbu3haYv10E=; b=Wj2yE5LGwkgLn3E4208O7ohzodt20tW5/BHvylBl5uL32OhWwaBJnV2jluovjvPABj bQ2+D1E0YFnEKVF6jkY9tdvMr00oDenxt8oJePMEhN2U5exjIrKr9i95LNKd8tzk7gyU kSXD7FVGZAP2tCzIJPGGCJaKTMQJUmWnSqDqmGeXbh7E7EIiFB5t6esBQ6rbKKaKD/PV n6BiEBWnLgC0tiMQhbc/Kf6DwFBZx1vwhYq84tWZwj1JP/QeJBRBUh88gh8kYPBiUWB+ 8rtjr6AGGOGk3ARl9lk7c4kFXSCBIPk221UfXcL4pXVVHoZa/u3mE9r/MQIrt6GHzH5G IFxg== X-Received: by 10.112.167.228 with SMTP id zr4mr13382214lbb.20.1422122010416; Sat, 24 Jan 2015 09:53:30 -0800 (PST) Received: from [10.0.1.7] (broadband-5-228-253-40.nationalcablenetworks.ru. [5.228.253.40]) by mx.google.com with ESMTPSA id p10sm1329078lap.10.2015.01.24.09.53.28 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 24 Jan 2015 09:53:29 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: network locks up with udp traffic From: Dmitry Sivachenko In-Reply-To: Date: Sat, 24 Jan 2015 20:53:27 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5B08402C-67A7-49E7-ADF8-390C94DCF1D7@gmail.com> To: Adrian Chadd X-Mailer: Apple Mail (2.1993) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Jan 2015 17:53:32 -0000 > On 24 =D1=8F=D0=BD=D0=B2. 2015 =D0=B3., at 20:29, Adrian Chadd = wrote: >=20 > Hi, >=20 > Can you compile your kernel with WITNESS and repeat this test? It > looks like it's stuck in some lock ordering thing between some UDP > paths and an inpcb lookup. >=20 > It should log in dmesg the first time a LOR occurs, regardless of > whether it actively causes a hang. Here is what I got in dmesg just before my net locked up: lock order reversal: 1st 0xffffffff80e79008 pcbinfohash (pcbinfohash) @ = /opt/WRK/src/sys/netinet6/udp6_usrreq.c:1202 2nd 0xffffffff80e78fb0 udp (udp) @ = /opt/WRK/src/sys/netinet6/in6_pcb.c:614 lock order reversal: 1st 0xffffffff80e79008 pcbinfohash (pcbinfohash) @ = /opt/WRK/src/sys/netinet6/udp6_usrreq.c:1202 2nd 0xffffffff80e78d58 tcp (tcp) @ = /opt/WRK/src/sys/netinet6/in6_pcb.c:614 lock order reversal: 1st 0xffffffff80e79008 pcbinfohash (pcbinfohash) @ = /opt/WRK/src/sys/netinet6/udp6_usrreq.c:1202 2nd 0xffffffff80e781c0 rip (rip) @ = /opt/WRK/src/sys/netinet6/in6_pcb.c:614 When I was saving this dmesg output to disk, I got another LOR: lock order reversal: 1st 0xfffffe0baf12fd78 bufwait (bufwait) @ = /opt/WRK/src/sys/kern/vfs_bio.c:3065 2nd 0xfffff80011416c00 dirhash (dirhash) @ = /opt/WRK/src/sys/ufs/ufs/ufs_dirhash.c:284 Please tell me if I can provide more information to help tracking this = down. Thanks!=