From owner-freebsd-current@FreeBSD.ORG Wed Nov 9 17:32:11 2005 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C3F0116A41F; Wed, 9 Nov 2005 17:32:11 +0000 (GMT) (envelope-from pckizer@nostrum.com) Received: from nostrum.com (magus.nostrum.com [69.5.195.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4C90843D46; Wed, 9 Nov 2005 17:32:11 +0000 (GMT) (envelope-from pckizer@nostrum.com) Received: from [165.91.250.64] (magus.tamu.edu [165.91.250.64]) (authenticated bits=0) by nostrum.com (8.12.11/8.12.11) with ESMTP id jA9HVcON054491 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NO); Wed, 9 Nov 2005 11:31:39 -0600 (CST) (envelope-from pckizer@nostrum.com) In-Reply-To: <0906B09C-B5A2-402E-BF39-57EBB20B2D4F@nostrum.com> References: <200510191623.j9JGNSfr007356@magus.nostrum.com> <20051019175020.S60849@fledge.watson.org> <20051025110453.L6720@fledge.watson.org> <2E18CEAE-2A72-4387-B92E-DAED7CC7FACD@nostrum.com> <33E53AA7-2A01-4BBE-9674-8F54E008D0A8@nostrum.com> <0906B09C-B5A2-402E-BF39-57EBB20B2D4F@nostrum.com> Mime-Version: 1.0 (Apple Message framework v746.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <425C901E-3315-41EC-B2D9-C372A2110FF0@nostrum.com> Content-Transfer-Encoding: 7bit From: Philip Kizer Date: Wed, 9 Nov 2005 11:31:37 -0600 To: Philip Kizer X-Mailer: Apple Mail (2.746.2) Received-SPF: pass (nostrum.com: 165.91.250.64 is authenticated by a trusted mechanism) Cc: freebsd-current@freebsd.org Subject: Re: Problem remains with FreeBSD 6.0-RELEASE as seen in RELENG_5 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Nov 2005 17:32:12 -0000 On Oct 29, 2005, at 11:01, Philip Kizer wrote: > Well, it's been a few days since the last time I heard from anyone > that had been helping me with this and I only recall seeing one > diagnostic patch to apply (that does not seem to have been > triggered in subsequent hangs). > Have there been any more patches I've missed that would help either > the problem or diagnostics? Well, with 6.0 going production, I updated my test system to that: FreeBSD host6 6.0-STABLE FreeBSD 6.0-STABLE #2: Tue Nov 8 08:40:32 CST 2005 since I figured you would do most of your development/patching against something more recent than RC2. I have now had another live-lock after upgrading: http://www.nostrum.com/hang/hang.RELENG_6-trace-2005-11-09-0.txt Any other suggestions or pointers on how to identify this livelock? As an added point, during boot I did see this LOR: Additional TCP options:. Starting background file system checks in 60 seconds. Tue Nov 8 15:31:14 CST 2005 FreeBSD/i386 (host6) (ttyd0) login: lock order reversal: 1st 0xc0982be0 UMA lock (UMA lock) @ /usr/src/sys/vm/uma_core.c:1494 2nd 0xc1060144 system map (system map) @ /usr/src/sys/vm/vm_kern.c:295 KDB: stack backtrace: kdb_backtrace(0,ffffffff,c0937670,c0937d00,c08c4344) at 0xc064d9b1 = kdb_backtrace+0x29 witness_checkorder(c1060144,9,c0879ac1,127) at 0xc065795c = witness_checkorder+0x5b0 _mtx_lock_flags(c1060144,0,c0879ac1,127) at 0xc062df1f = _mtx_lock_flags+0x5b _vm_map_lock(c10600c0,c0879ac1,127) at 0xc078dda6 = _vm_map_lock+0x26 kmem_malloc(c10600c0,1000,1,e6a03ba4,c078605d) at 0xc078d432 = kmem_malloc+0x32 page_alloc(c104a960,1000,e6a03b97,1,c06572a0) at 0xc078647e = page_alloc+0x1a slab_zalloc(c104a960,1,0,c3dd0680,c10491c8) at 0xc078605d = slab_zalloc+0xa1 uma_zone_slab(c104a960,1,1,10,6ecf) at 0xc07875d0 = uma_zone_slab+0xe8 uma_zalloc_bucket(c104a960,1) at 0xc07877e0 = uma_zalloc_bucket+0x12c uma_zalloc_arg(c104a960,0,1) at 0xc078749c = uma_zalloc_arg+0x2dc malloc(800,c08fee40,1,105,c1044460) at 0xc062c38a = malloc+0xae hash_alloc(e6a03c74,c1044468,0,c0878d7a,1ab) at 0xc07859f9 = hash_alloc+0x29 zone_timeout(c1040b40) at 0xc0785912 = zone_timeout+0x7e zone_foreach(c0785894,e6a03ce8,c0641065,0,c0785860) at 0xc0786df7 = zone_foreach+0x37 uma_timeout(0) at 0xc0785872 = uma_timeout+0x12 softclock(0) at 0xc0641065 = softclock+0x211 ithread_loop(c34d6c00,e6a03d38,c34d6c00,c06234c0,0) at 0xc0623604 = ithread_loop+0x144 fork_exit(c06234c0,c34d6c00,e6a03d38) at 0xc0622a10 = fork_exit+0xa0 fork_trampoline() at 0xc07e39fc = fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe6a03d6c, ebp = 0 --- FreeBSD/i386 (host6) (ttyd0) login: Thank you, Philip