From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 24 18:05:38 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6EC21106566B for ; Sun, 24 Jun 2012 18:05:38 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id F1F8F8FC0A for ; Sun, 24 Jun 2012 18:05:37 +0000 (UTC) Received: by eabm6 with SMTP id m6so1275361eab.13 for ; Sun, 24 Jun 2012 11:05:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=9Pd0UkJAesGAgdqXl29aCEG2h/euQV6yt2/moZJ/RD4=; b=BCI14DUn2aXU3Q16yA/T5l7IUk0bKg/wC7dLcxES4gdwDQg2ZsKSUm4sgzYykMCWDD jddBjZ9tKLl0bJU1GUQuMQ/RSaVLdbg3X5PBIEg5MzgfKFXVYAnzO0DIqFju5P9zwASq iHYb5CtamSETsYLB8biGHsns7v+nhAtNp36Mwe8kwTBSFuwe+fqS0pfraXqE4JETLDY3 xU2zSjfTcwJGBP/uiAAdqiCqMBUDR9tQaY1MGT8GUCGLRrkso9hLrxGqGF+ODcvN5aba zQBacfGfnVfLb6cTUkKG3pTGdzHB/WsfvBothWjmmLE2XYvq8JfE4wp5DRvdX8Ozkl7y F5Uw== Received: by 10.14.189.14 with SMTP id b14mr1354037een.141.1340561136878; Sun, 24 Jun 2012 11:05:36 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPS id h53sm131245567eea.1.2012.06.24.11.05.35 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 24 Jun 2012 11:05:35 -0700 (PDT) Date: Sun, 24 Jun 2012 20:05:27 +0200 From: Mateusz Guzik To: Wojciech Puchar Message-ID: <20120624180526.GA15899@dft-labs.eu> References: <20120623162415.303430@gmx.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-hackers@freebsd.org Subject: Re: reason for "magic" crashes. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Jun 2012 18:05:38 -0000 On Sun, Jun 24, 2012 at 07:05:35PM +0200, Wojciech Puchar wrote: > i've got third crash third week in a row. > > Every time in sunday after 18:00, every time with rsync process > (which means rsync based backup that is done every day, not just in > sunday!), > > you may see a crash (viewed from KVM) at > > http://www.tensor.gdynia.pl/~wojtek/crash.png > > what is important - syncing disk doesn't go on, system hangs here. > > For 99% system is not overheating at sunday, but i will be 100% sure > as i added ipmitool sensor logged from cron every 5 minutes. > > Please give me an idea what to check. > > > There is nothing in cron that is done at sunday. > > i don't run "periodic" stuff in /etc/crontab > Compile the kernel with the following: makeoptions DEBUG="-O0 -g" options KDB # Enable kernel debugger support. options DDB # Support DDB. options GDB # Support remote GDB. options DEADLKRES # Enable the deadlock resolver options INVARIANTS # Enable calls of extra sanity checking options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS options WITNESS # Enable checks to detect deadlocks and cycles options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed options DIAGNOSTIC After kernel panic ddb prompt will be waiting for you. Type in: dump reset Make sure you have swap that can handle crashdumps. See this for more details: http://www.freebsd.org/doc/en/books/developers-handbook/kerneldebug.html You can check if everything works correctly by issuing panic manually: sysctl debug.kdb.panic=1 then typing aforementioned ddb commands. After reboot you should get core in /var/crash. Also provide the following: - system version - filesystems involved in rsync with mount details (e.g. UFS with SU+J) - dmesg Hopefully this will be enough for someone to help. -- Mateusz Guzik