From owner-freebsd-stable@freebsd.org Mon Feb 13 17:49:51 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D10F6CDDCE8 for ; Mon, 13 Feb 2017 17:49:51 +0000 (UTC) (envelope-from thenomad@gmail.com) Received: from mail-ot0-x236.google.com (mail-ot0-x236.google.com [IPv6:2607:f8b0:4003:c0f::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 891B215FB for ; Mon, 13 Feb 2017 17:49:51 +0000 (UTC) (envelope-from thenomad@gmail.com) Received: by mail-ot0-x236.google.com with SMTP id 65so73937878otq.2 for ; Mon, 13 Feb 2017 09:49:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=/KdG0tMd4Ik117QcC+zuHUwjb1YcA4zKlWxlDofOkfk=; b=LEepHbw0KjlunxOJei1f5MlVYRb80sjklmdBlwWsuiLEOvSeSK155cte4jLYubO4eE 3ZPyHIeuCOP+kb3+7buBeNrpsYkx5mL76Onz4sijjesM8KajABaFGtoD2Zw0kivs6LT2 PhbQV9Bc3cNwVOz/YHu3gSazwXxYchx3b2RUdIkLXMd00HRWMCUwb4U3Wy8/WdXtckYR Iio7Hd6fQeo3o0dO3tTx8YhFD79eU2W+8ynqryr4swT1gJOZbakCW+db/MCHDEc0Nu0C yybvY5FD+fIj3qKQ1bBnff5+ll01Kjpim50PoFPxYx1uZgYKg1HioidYkIYL8aHb9b5e 7k4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=/KdG0tMd4Ik117QcC+zuHUwjb1YcA4zKlWxlDofOkfk=; b=kEyoTHC+KwrQmqzDGd0vqv28H/zk4BgSI8sPwbeEbNvnObOR1KHtD1Ua/Ro92M4oF1 xJ6wiTR2uMxHeeRUTtdxIqYKiIyS0XudW5s8l87kZoS+PNuhZoMLTCay8lckcPgM8TUN 86GQGTGPsNBj3HGD/hMyFhupMnzMW/DQhQAFF46Qapja5wBBtXJnYg5/Ttjpxezgkjpc 3cSitjcp27XGHQ7Mop9hOxk56Ce2YNiufhyuhmMQB0cuW8kUA6ZrI2awrEZOwzyiWFgE HJv95wKJDGmbfFIXb5VRy86qLphC9UTQuCgq+HLHVrAAaNYtywrmRzLNMrnl+JjvCCW1 uvjw== X-Gm-Message-State: AMke39mlUwilKLeTjQdeCeIQaF0rCou2HUxEmwHUr23jP5KJMADX/1mEw8MS/nspzx2fSw== X-Received: by 10.98.198.90 with SMTP id m87mr26757838pfg.153.1487008190292; Mon, 13 Feb 2017 09:49:50 -0800 (PST) Received: from vanyel.ee.washington.edu (vanyel.ee.washington.edu. [128.208.232.99]) by smtp.gmail.com with ESMTPSA id s8sm22222315pfj.30.2017.02.13.09.49.48 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Feb 2017 09:49:48 -0800 (PST) Sender: Lee Damon Subject: Re: FBSD 10.3 + ZFS + Sun x4500 = utter lock up. To: freebsd-stable@freebsd.org References: <44ecebcb-fb48-d828-7f08-47a981b732d2@castle.org> From: Lee Damon Message-ID: Date: Mon, 13 Feb 2017 09:49:53 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Feb 2017 17:49:51 -0000 In what was arguably a silly attempt I changed all IRQ interrupts to go to CPU0 and .. the host has stayed up through multiple attempts to crash it. I'm not calling it fixed yet but there appears to be hope. Right now I have a script -- /usr/local/etc/rc.d/cpuset.sh -- that's doing the work. This seems a sub-optimal place to do it as there is a possibility of crash before the script is executed on boot. Is there any option in bootloader or related for setting these or is cpuset(1) my only option? thanks, nomad >> FreeBSD [redacted] 10.3-STABLE FreeBSD 10.3-STABLE #2 r313008: Tue Jan >> 31 01:50:49 PST 2017 lvd@[redacted]:/usr/obj/usr/src/sys/GENERIC >> amd64 >> >> I'm trying to get FBSD 10.3 working on a Sun x4500 (don't ask) for use >> as a ZFS-based backup server. However, whenever any amount of data is >> put into a zpool and then zpool scrub is run the host locks up hard. On >> reboot it complains that a "Hyper transport sync flood occurred". >> >> I found >> https://lists.freebsd.org/pipermail/freebsd-stable/2012-January/065542.html >> >> which seems to match but when I try the cpuset command mentioned there I >> get an error: >> >> ; sudo cpuset -c -l 0 -x 58 >> cpuset: setaffinity: Invalid argument >> >> Looks like the -c was invalid. After removing that I was informed -x 58 >> wasn't valid. Sure enough, there's no mpt0 or IRQ 58 on the host: >> >> ; vmstat -i >> interrupt total rate >> irq17: ohci2 8578 2 >> irq18: ohci3 473 0 >> irq19: ohci0 ohci1+ 4924 1 >> irq24: mvs0 457 0 >> irq32: mvs1 453 0 >> irq38: mvs2 451 0 >> irq46: mvs3 8063 1 >> irq52: em0 152354 35 >> irq53: em1 140 0 >> irq68: mvs4 450 0 >> irq76: mvs5 454 0 >> cpu0:timer 208311 48 >> cpu1:timer 98318 23 >> cpu2:timer 105704 24 >> cpu3:timer 106202 24 >> Total 695332 162 >> >> Looking around with some help from #freebsd on efnet I found mvs0-5 >> which are connected to the Marvel drive controllers on the host. I then >> used >> ; sudo cpuset -l 0 -x ## >> where I replaced ## with 24, 32, 38, 46, 68, and 76. >> >> After rebuilding the zpool I started writing to it. It took a lot less >> time to crash - I didn't even need to run zpool scrub - but instead of >> completely locking up it just rebooted. I did not see reference to the >> hyper transport problem while watching it boot but given the poor >> performance of the serial console I can't be 100% sure it wasn't there. >> >> So now I turn here to ask for guidance. Is anyone currently successfully >> running 10.x on a x4500 and if so, how are you doing it? If not, how can >> I get this working? >> >> thanks, >> nomad >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"