From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 11:04:07 2004 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 058C716A4CE for ; Thu, 7 Oct 2004 11:04:07 +0000 (GMT) Received: from w2xo.jcdurham.com (18.gibs5.xdsl.nauticom.net [209.195.184.19]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4B17643D45 for ; Thu, 7 Oct 2004 11:04:06 +0000 (GMT) (envelope-from durham@w2xo.jcdurham.com) Received: from w2xo.jcdurham.com (localhost [127.0.0.1]) by w2xo.jcdurham.com (8.12.11/8.11.6) with ESMTP id i97B4524048268 for ; Thu, 7 Oct 2004 07:04:05 -0400 (EDT) (envelope-from durham@w2xo.jcdurham.com) Received: from localhost (localhost [[UNIX: localhost]]) by w2xo.jcdurham.com (8.12.11/8.12.9/Submit) id i97B44XZ048267 for freebsd-hackers@freebsd.org; Thu, 7 Oct 2004 07:04:04 -0400 (EDT) (envelope-from durham) From: Jim Durham To: freebsd-hackers@freebsd.org Date: Thu, 7 Oct 2004 07:04:04 -0400 User-Agent: KMail/1.6.2 References: <20041004130415.G1034-100000@mxb.saturn-tech.com> <200410041604.00507.durham@jcdurham.com> In-Reply-To: <200410041604.00507.durham@jcdurham.com> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200410070704.04654.durham@jcdurham.com> Subject: Re: Sudden Reboots X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2004 11:04:07 -0000 On Monday 04 October 2004 04:04 pm, Jim Durham wrote: > On Monday 04 October 2004 03:06 pm, Doug Russell wrote: > > On Mon, 4 Oct 2004, Jim Durham wrote: > > > The reboots started out happening at 5.15 pm or so. I had them unplug > > > the server completely from AC and restart it and now it's happening > > > withing a few minutes of 12:40pm every day. > > > > > > The 'last' command output is the only thing showing anything log-wise. > > > Look at this: > > > > > > reboot ~ Mon Oct 4 12:33 > > > reboot ~ Sun Oct 3 12:37 > > > reboot ~ Sat Oct 2 12:42 > > > reboot ~ Fri Oct 1 12:45 > > > > That is strange.... > > What is that machine doing at that time? Does it start a backup job, or > > any kind of maintenance at that time? Anything at all? > > > > I'd monitor what the heck the machine is doing to some remote machine and > > see what goes on when it dies. > > > > Later....... > > Well... I'm trying to do too many things at the same time and not thinking. > I thought about it and also ran 'last -f /var/log/wmtp.1' and found the > whole sequence...and added my recent reboot info to the top and here is > what I got. > > > reboot ~ Mon Oct 4 14:54 > reboot ~ Mon Oct 4 12:33 > reboot ~ Sun Oct 3 12:37 > reboot ~ Sat Oct 2 12:42 > reboot ~ Fri Oct 1 12:45 > reboot ~ Thu Sep 30 12:33 > reboot ~ Thu Sep 30 08:54 > reboot ~ Wed Sep 29 08:41 > reboot ~ Tue Sep 28 17:20 > reboot ~ Mon Sep 27 17:28 > reboot ~ Sun Sep 26 17:28 > reboot ~ Sat Sep 25 17:37 > reboot ~ Fri Sep 24 17:38 > reboot ~ Thu Sep 23 17:42 > reboot ~ Wed Sep 22 17:50 > reboot ~ Tue Sep 21 17:52 > reboot ~ Mon Sep 20 17:57 > reboot ~ Sun Sep 19 17:57 > reboot ~ Sat Sep 18 17:56 > > Now, the machine was powered down on Sept 17 because of an all-day power > failure. It started doing this on the 18th ,starting at 5:56pm and working > earlier until Sept 29, when I upgraded from 4.9 to 4.10 p3. Then it started > rebooting at 8:45 aprox and I had the customer shut down the machine > completely on the 30th and remove the AC from the rear connector and wait a > few minutes and bring it back up. That's when it started rebooting at 12:30 > or so. > > Googling for "Freebsd reboots at same time every day" will produce hits > on people with the same problem so this is not just me. > > I checked crontab, etc and there's nothing coinciding and also, apparently > every time you reboot the machine, the cycle changes, so how can it be an > external event? Boy...I'm confused... 8-) > > There was one Google posting that maintained that this one guy fixed it by > removing IPV6 from the kernel. I'm trying this at the moment. Of course, I > had to reboot to change the kernel, so the times will probably change if it > keeps rebooting. > > It also just hit me that the time doesn't change much if the machine > reboots itself, but changes if you upgrade or do a controlled reboot. > Wow... This is a followup posting on the above problem.. After removing IPV6 from the kernel and recompiling ,(note that I had recompiled the kernel earlier during this troubleshooting series with no effect) , the problem has gone away. The machine has been up for over 2 days with no reboots. So, to recap: Machine reboots at aprox 23hrs 57 minutes of uptime at ~12:30pm.. Upgraded to latest 4.10p3 sources. Still reboots at 23hrs 57mins, but at different time of day, ~8:15am. Unplugged machine totally from AC and restarted. Still reboots at 23:57 intervals, but time is now changed again, ~2:55pm. Recompiled kernel, removing IPV6. Problem stops. Up now for over 2 days for first time in weeks. Note1: The 23:57 is the usual interval. Sometimes it varied by a few minutes. Note2: 'healthd' showed normal operation of Power Supply with no log entries. Note3: The IPV6 idea was gleaned from a Google search of earlier postings on same problem. Note4: Another posting claimed a cure of this problem by changing network card. Observation: When the machine reboots itself, the time of the reboots seems to stay very close to 23:57 intervals. When you reboot on command, the interval stays the same, but the time of day changes, and does not coincide with the commanded reboot time. I have no conclusion about this. Thanks to all who made suggestions. -Jim Durham