From owner-freebsd-stable@FreeBSD.ORG Sat Jan 19 10:30:20 2013 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 9EC08AF1; Sat, 19 Jan 2013 10:30:20 +0000 (UTC) (envelope-from dnaeon@gmail.com) Received: from mail-bk0-f42.google.com (mail-bk0-f42.google.com [209.85.214.42]) by mx1.freebsd.org (Postfix) with ESMTP id 0E642658; Sat, 19 Jan 2013 10:30:19 +0000 (UTC) Received: by mail-bk0-f42.google.com with SMTP id ji2so2411759bkc.1 for ; Sat, 19 Jan 2013 02:30:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=ioLY//4fsTdwuYPLya12x1a6QJ6Qe7kh0prVMIbzqq0=; b=rV78OsqBhzRIV7hmgOe4uClblybdJH9eJRXKAVGTml4CpxvjiCvZsuEO/69AF3e3x2 6PgSZ7nR6/i53YGpPkc0pkEEU29lrw8c+ZWhFn19kNQbBbUgLq776RBRyaW2een5iLG3 sne8h6sb6Bn+Qe4/bpP0/Pobn9rH4O3PPjPFr5OXu6z6RFeXDCg1qWtTSSbEo3RD0OQ4 iY+gz4XKiXX4SCMRGGzfRXGHqNEFwE4xTsodh1EjilberJRcSUJt82Nl+ELgtN9wyWLE lWLXIlj7J1+0N/pyFODNa3pNUI5OPnQndch5k9RFiKv0zhUPOyYbPl63QSrri1CMyVXE S5aw== MIME-Version: 1.0 X-Received: by 10.204.148.134 with SMTP id p6mr3186143bkv.75.1358591417181; Sat, 19 Jan 2013 02:30:17 -0800 (PST) Received: by 10.204.179.146 with HTTP; Sat, 19 Jan 2013 02:30:17 -0800 (PST) In-Reply-To: References: <1358527685.32417.237.camel@revolution.hippie.lan> <20130118173602.GA76438@neutralgood.org> Date: Sat, 19 Jan 2013 12:30:17 +0200 Message-ID: Subject: Re: Spontaneous reboots on Intel i5 and FreeBSD 9.0 From: Marin Atanasov Nikolov To: Warren Block Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: ml-freebsd-stable , Ian Lepore , kpneal@pobox.com, Ronald Klop X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jan 2013 10:30:20 -0000 Hi, Re-sending this one, as I've attached an image which was too large to pass the mailing lists, sorry about that :) After starting the system last night I kept monitoring the memory usage just in case I see something strange and I've noticed a significant memory drop of the free memory between 03:00am and 03:05am time. I've taken a screenshot of the graph, which you can also see at the link below: * http://users.unix-heaven.org/~dnaeon/memory-usage.jpg At 03:00am I can see that periodic(8) runs, but I don't see what could have taken so much of the free memory. I'm also running this system on ZFS and have daily rotating ZFS snapshots created - currently the number of ZFS snapshots are > 1000, and not sure if that could be causing this. Here's a list of the periodic(8) daily scripts that run at 03:00am time. % ls -1 /etc/periodic/daily 100.clean-disks 110.clean-tmps 120.clean-preserve 130.clean-msgs 140.clean-rwho 150.clean-hoststat 200.backup-passwd 210.backup-aliases 220.backup-pkgdb 300.calendar 310.accounting 330.news 400.status-disks 404.status-zfs 405.status-ata-raid 406.status-gmirror 407.status-graid3 408.status-gstripe 409.status-gconcat 420.status-network 430.status-rwho 440.status-mailq 450.status-security 460.status-mail-rejects 470.status-named 480.status-ntpd 490.status-pkg-changes 500.queuerun 800.scrub-zfs 999.local % ls -1 /usr/local/etc/periodic/daily 402.zfSnap 403.zfSnap_delete 411.pkg-backup smart I'll keep monitoring the memory usage and will see if the free memory drops again by more than 50% on the next periodic(8) daily run. If the memory drop keeps the current trend that would mean that the system should crash in the next 1-2 days, so if that happens and the memory was low at that time I'll start debugging the periodic(8) scripts and see which one might be causing this. Thanks and regards, Marin On Fri, Jan 18, 2013 at 10:23 PM, Warren Block wrote: > On Fri, 18 Jan 2013, kpneal@pobox.com wrote: > > On Fri, Jan 18, 2013 at 09:48:05AM -0700, Ian Lepore wrote: >> >>> I tend to agree, a machine that starts rebooting spontaneously when >>> nothing significant changed and it used to be stable is usually a sign >>> of a failing power supply or memory. >>> >> >> Agreed. >> >> But I disagree about memtest86. It's probably not completely without >>> value, but to me its value is only negative: if it tells you memory is >>> bad, it is. If it tells you it's good, you know nothing. Over the >>> years I've had 5 dimms fail. memtest86 found the error in one of them, >>> but said all the others were fine in continuous 48-hour tests. I even >>> tried running the tests on multiple systems. >>> >>> The thing that always reliably finds bad memory for me >>> is /usr/ports/math/mprime run in test/benchmark mode. It often takes 24 >>> or more hours of runtime, but it will find your bad memory. >>> >> >> I've had "good" luck with gcc showing bad memory. If compiling a new >> kernel >> produces seg faults then I know I have a hardware problem. I've seen >> compilers at work failing due to bad memory as well. >> >> Some problems only happen with particular access patterns. So if a >> compiler >> works fine then, like memtest86, it doesn't say anything about the health >> of the hardware. >> > > Most test tools are like that. They might diagnose something as bad, but > they often can't prove it is good. SMART has a reputation for not finding > any problems on disks that are failing, and capacitors that aren't swollen > or leaking still may not be working. > > But diagnostic tools can at least give a hint. In my case, memtest > indicated a problem--a big problem. I removed one DIMM at random (there > were only two) and the problems and memtest errors both went away. Replace > the DIMM, and both came back. > > ______________________________**_________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/**mailman/listinfo/freebsd-**stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@**freebsd.org > " > -- Marin Atanasov Nikolov dnaeon AT gmail DOT com http://www.unix-heaven.org/