From owner-freebsd-stable@freebsd.org Tue Feb 7 21:44:05 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 39BD0CD5870 for ; Tue, 7 Feb 2017 21:44:05 +0000 (UTC) (envelope-from thenomad@gmail.com) Received: from mail-pf0-x229.google.com (mail-pf0-x229.google.com [IPv6:2607:f8b0:400e:c00::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1452012BA for ; Tue, 7 Feb 2017 21:44:05 +0000 (UTC) (envelope-from thenomad@gmail.com) Received: by mail-pf0-x229.google.com with SMTP id y143so36072481pfb.0 for ; Tue, 07 Feb 2017 13:44:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:to:from:subject:message-id:date:user-agent:mime-version :content-transfer-encoding; bh=VH96h/GpA7l2ua7PwuTFIMmoheSKjC2Ya8eZXbd7XNY=; b=Mo9HVd/mLHqkOr+8n9a2SEly92BORc/7p1zteiY+W6a0jgnMQXT7rXzZkUPd2RQzXA /7Avys7BkFsnSXFLo1IAt9IP1U2HCW1emW1WHzH3WMWKtRzvFUibn0FLVGydCsbAJCsY FhQPgWKEDH3xFOlLVIFFEicIZX1JMW/Uevg1TbMVHaQiphiW511ckZ6q22gg5ekV/kym hywc8O/NHtfCHVtINnE18WIeG9PAASTZYu1jhV93btODl3n0cJrc+orT7maKSBRplEMC HQerYsNrSUl3fLZ3bQITLXEJfwjK66FtdBBfAcM96Fhg3dS5WA2j0FhGMMXAxCS7Nn6u V0Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:to:from:subject:message-id:date :user-agent:mime-version:content-transfer-encoding; bh=VH96h/GpA7l2ua7PwuTFIMmoheSKjC2Ya8eZXbd7XNY=; b=YlDdgCkv/wQ7MLD3MRM685F18hv2JQTSsqrYDyVgNmGSVrbYJLBl1aaoqVORKOLhRn XG5/vCCKnUZ/N6jFY3RAANh4FHh7tZYWz9nOLY96/x59q7iCqlnnRK3Nsu58GTwRuDYK FqW83knnXgmHcMkj0BH4fXGWhN/z246D+cst6AzUtsMi9O1Cs7V4vVa+kXt11QWDe72h vmv9TvwRJM6VsZ3G2BgykMuMB8Dca0NYNrfiz7uX9p6cbm9oG1frBN3kAxsso7y49ol9 pMwuxUKoEnZhaITf4R/AXPjbTMSiTf0gvrG/W1LFcrDv2iGKEHQ5mrmCTuHVkjy83QN2 4emQ== X-Gm-Message-State: AIkVDXI3F555TkfLPgBCY2/bEDIcCf61mRLkqnxQYID7/+UJhNv7mpTsn4KnIM9wa7nfwg== X-Received: by 10.99.116.22 with SMTP id p22mr23233989pgc.161.1486503844352; Tue, 07 Feb 2017 13:44:04 -0800 (PST) Received: from vanyel.ee.washington.edu (vanyel.ee.washington.edu. [128.208.232.99]) by smtp.gmail.com with ESMTPSA id e4sm13853866pgc.45.2017.02.07.13.44.01 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Feb 2017 13:44:02 -0800 (PST) Sender: Lee Damon To: freebsd-stable@freebsd.org From: Lee Damon Subject: FBSD 10.3 + ZFS + Sun x4500 = utter lock up. Message-ID: <44ecebcb-fb48-d828-7f08-47a981b732d2@castle.org> Date: Tue, 7 Feb 2017 13:44:01 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Feb 2017 21:44:05 -0000 FreeBSD [redacted] 10.3-STABLE FreeBSD 10.3-STABLE #2 r313008: Tue Jan 31 01:50:49 PST 2017 lvd@[redacted]:/usr/obj/usr/src/sys/GENERIC amd64 I'm trying to get FBSD 10.3 working on a Sun x4500 (don't ask) for use as a ZFS-based backup server. However, whenever any amount of data is put into a zpool and then zpool scrub is run the host locks up hard. On reboot it complains that a "Hyper transport sync flood occurred". I found https://lists.freebsd.org/pipermail/freebsd-stable/2012-January/065542.html which seems to match but when I try the cpuset command mentioned there I get an error: ; sudo cpuset -c -l 0 -x 58 cpuset: setaffinity: Invalid argument Looks like the -c was invalid. After removing that I was informed -x 58 wasn't valid. Sure enough, there's no mpt0 or IRQ 58 on the host: ; vmstat -i interrupt total rate irq17: ohci2 8578 2 irq18: ohci3 473 0 irq19: ohci0 ohci1+ 4924 1 irq24: mvs0 457 0 irq32: mvs1 453 0 irq38: mvs2 451 0 irq46: mvs3 8063 1 irq52: em0 152354 35 irq53: em1 140 0 irq68: mvs4 450 0 irq76: mvs5 454 0 cpu0:timer 208311 48 cpu1:timer 98318 23 cpu2:timer 105704 24 cpu3:timer 106202 24 Total 695332 162 Looking around with some help from #freebsd on efnet I found mvs0-5 which are connected to the Marvel drive controllers on the host. I then used ; sudo cpuset -l 0 -x ## where I replaced ## with 24, 32, 38, 46, 68, and 76. After rebuilding the zpool I started writing to it. It took a lot less time to crash - I didn't even need to run zpool scrub - but instead of completely locking up it just rebooted. I did not see reference to the hyper transport problem while watching it boot but given the poor performance of the serial console I can't be 100% sure it wasn't there. So now I turn here to ask for guidance. Is anyone currently successfully running 10.x on a x4500 and if so, how are you doing it? If not, how can I get this working? thanks, nomad