From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 2 20:12:04 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 59423494 for ; Tue, 2 Sep 2014 20:12:04 +0000 (UTC) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E6E201D7A for ; Tue, 2 Sep 2014 20:12:03 +0000 (UTC) Received: by mail-wi0-f179.google.com with SMTP id q5so8581561wiv.0 for ; Tue, 02 Sep 2014 13:12:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=8xUlJVx/HgBh1/BlVjLejFRUogUWXRFD63vSqQ/ysis=; b=VYCrbRcBU7OXzRzvrj7qv++kwFQ6sRNaNplifLiY6c1issGm6bqJ+RYZBWlP0qf5Al QoaHU7i43Y1+lUFjEFGVo0ORg7SoudntG4mAZoJsc0FQJwy5iXczkwTvCGz+PLBbXUB5 2lh7jlbN1dtTiPnPQ/7yRRs0d2kjjdAo/iRdxH0/rYKhDFu7MomY6IjQzCnhl6BEK3wT SlC8HonOSrjUdCV21KKe0MzO0SUIYOTM1uXZ2xoCVSaFEZ0R+k+Im1P7Z1gz7SjhwO0Y gRE+ga4a3c6iq4t+gOY936FpY36YQaVL3mI21njf7V2HYuJu0uthqXwcEAq/S6J51SNp 852w== X-Received: by 10.180.89.100 with SMTP id bn4mr31100042wib.34.1409688722138; Tue, 02 Sep 2014 13:12:02 -0700 (PDT) Received: from [192.168.1.17] ([89.191.152.141]) by mx.google.com with ESMTPSA id r19sm37879176wik.0.2014.09.02.13.12.00 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Sep 2014 13:12:01 -0700 (PDT) Message-ID: <54062495.4080601@gmail.com> Date: Tue, 02 Sep 2014 22:12:05 +0200 From: Maciej Gabryszak User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.0 MIME-Version: 1.0 To: "hackers@freebsd.org" Subject: FreeBSD 10.0 + kernel crash Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Sep 2014 20:12:04 -0000 Hi All, I configurated freeBSD 10.0 at my servers. Two have configuration: CARP+PF. Sometimes one of them crash. Hardware: Supermicro servers Debug log: Unread portion of the kernel message buffer: <118>Stopping powerd. <118>Waiting for PIDS: 2913. <118>Stopping devd. <118>Waiting for PIDS: 2578. <118>Writing entropy file:. <118>. <118>Terminated <118>Aug 29 19:54:04 rack02 syslogd: exiting on signal 15 spin lock 0xffffffff814fa030 (smp rendezvous) held by 0xfffff8035506d000 (tid 100157) too long panic: spin lock held too long cpuid = 2 KDB: stack backtrace: #0 0xffffffff808e7dd0 at kdb_backtrace+0x60 #1 0xffffffff808af8b5 at panic+0x155 #2 0xffffffff8089cb71 at _mtx_lock_spin_cookie+0x241 #3 0xffffffff80c7ef54 at smp_targeted_tlb_shootdown+0xf4 #4 0xffffffff80c80922 at pmap_invalidate_all+0x232 #5 0xffffffff80c8736f at pmap_remove_pages+0x6af #6 0xffffffff80b104c0 at vmspace_exit+0xa0 #7 0xffffffff8087c6af at exit1+0x65f #8 0xffffffff808b2eef at sigexit+0xb7f #9 0xffffffff808b35a9 at postsig+0x349 #10 0xffffffff808f6e97 at ast+0x437 #11 0xffffffff80c765c9 at doreti_ast+0x1f Uptime: 1d2h40m52s (ada0:ahcich0:0:0:0): STANDBY_IMMEDIATE. ACB: e0 00 00 00 00 40 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request is in progress (ada0:ahcich0:0:0:0): Error 5, Retries exhausted (ada0:ahcich0:0:0:0): Spin-down disk failed (ada1:ahcich1:0:0:0): STANDBY_IMMEDIATE. ACB: e0 00 00 00 00 40 00 00 00 00 00 00 (ada1:ahcich1:0:0:0): CAM status: CCB request is in progress (ada1:ahcich1:0:0:0): Error 5, Retries exhausted (ada1:ahcich1:0:0:0): Spin-down disk failed Dumping 1590 out of 32706 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% It looks like problem with standby and disks. I tried turn off it at disks by camcontrol cmd /dev/ada0 -a "EF 85 00 00 00 00 00 00 00 00 00 00" -v and same at second disk but it didn't change anything. Server crash after a few minuts/days. cat /etc/rc.conf zfs_enable="YES" hostname="rack02" keymap="pl_PL.ISO8859-2.kbd" sshd_enable="YES" ntpd_enable="YES" powerd_enable="YES" dumpdev="AUTO" cat /boot/loader.conf zfs_load="YES" ipmi_load="YES" vmm_load="YES" if_tap_load="YES" bridgestp_load="YES" if_bridge_load="YES" nmdm_load="YES" if_em_load="YES" pf_load="YES" pfsync_load="YES" pflog_load="YES" carp_load="YES" autoboot_delay="3" Can you suggest something how can i resolve issue ? Best Regards, Maciej