From owner-freebsd-stable@FreeBSD.ORG Sun Jun 11 18:09:12 2006 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9F0EF16A41A for ; Sun, 11 Jun 2006 18:09:12 +0000 (UTC) (envelope-from freebsd@wcubed.net) Received: from mail.datausa.com (mail.datausa.com [216.150.220.134]) by mx1.FreeBSD.org (Postfix) with SMTP id 797B543D5C for ; Sun, 11 Jun 2006 18:09:11 +0000 (GMT) (envelope-from freebsd@wcubed.net) Received: (qmail 8020 invoked by uid 89); 11 Jun 2006 18:07:19 -0000 Received: from c-67-176-75-179.hsd1.co.comcast.net (HELO ?10.0.1.1?) (67.176.75.179) by mail.datausa.com with SMTP; 11 Jun 2006 18:07:19 -0000 Message-ID: <448C5C41.10302@wcubed.net> Date: Sun, 11 Jun 2006 12:09:05 -0600 From: Brad Waite User-Agent: Thunderbird 1.5.0.4 (Windows/20060516) MIME-Version: 1.0 To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: 6.1-stable hangs and LORs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jun 2006 18:09:12 -0000 Hi guys, I'm going to take another stab at getting some help. For the last 6 months my FBSD gateway has been locking up every few days, usually about once a week. No panic, no reboots, just a hard lock with no response on the console or over the net. I've replaced literally every piece of hardware with the exception of the case and power supply. No change. I've upgraded from 5.3- to 6.0- to 6.1-STABLE. No change. I've researched as much as I know how and still come up with hardly anything. I have turned on BREAK_TO_DEBUGGER, WITNESS and INVARIANTS and the only indication I've gotten is a lock order reversal that's *similar* to http://sources.zabbadoz.net/freebsd/lor/017.html. The line numbers in pf.c don't match up with LOR 017, but that's about all I can tell. I'm reasonably certain the issue is with pf, since I have 3 other non-gateway servers humming along with no problems. The hardware is nearly identical - their RAID cards are different, but I've tried running my gateway on just a single SCSI drive and had the same lockup issue. Of course, the issue could be somewhere else, but I'm at a loss as to how to find it. I'm running my console over serial so I can log anything that's necessary. I've been able to break to the debugger, but to be honest, I don't know what to look for. I've seen several posts on the lists about posting the output of debug commands, but I figured it to be in poor taste to just dump my output here before someone asked. I'm getting a lot of heat from the boss since our VoIP phones don't work when the gateway locks up. If someone can help identify and/or eliminate this issue, I'm more than happy to do everything I can to provide the necessary information. Thanks.