From owner-freebsd-stable@FreeBSD.ORG Tue Jan 17 23:29:16 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B6D6D1065676 for ; Tue, 17 Jan 2012 23:29:16 +0000 (UTC) (envelope-from healer@rpi.edu) Received: from smtp8.server.rpi.edu (smtp8.server.rpi.edu [128.113.2.228]) by mx1.freebsd.org (Postfix) with ESMTP id 751728FC28 for ; Tue, 17 Jan 2012 23:29:16 +0000 (UTC) Received: from [128.113.242.162] (ford-tri-motor-03.dynamic2.rpi.edu [128.113.242.162]) (authenticated bits=0) by smtp8.server.rpi.edu (8.13.1/8.13.1) with ESMTP id q0HMMMC6014169 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 17 Jan 2012 17:22:23 -0500 Message-ID: <4F15F4A0.8030007@rpi.edu> Date: Tue, 17 Jan 2012 17:22:24 -0500 From: Bob Healey User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <201201171859.10812.peter@hk.ipsec.se> <20120117220912.GA32330@icarus.home.lan> In-Reply-To: <20120117220912.GA32330@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Bayes-Prob: 0.0001 (Score 0) X-RPI-SA-Score: 1.40 (*) [Hold at 12.00] RATWARE_GECKO_BUILD,22490(-25) X-CanItPRO-Stream: outgoing X-Canit-Stats-ID: Bayes signature not available X-Scanned-By: CanIt (www . roaringpenguin . com) on 128.113.2.228 Subject: Re: about thumper aka sun fire x4500 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jan 2012 23:29:16 -0000 The X4500s are oldish systems built around a pair Opteron 290 chips with 16 GB RAM and 6 PCI-X Marvell SATA controllers with 8 ports each supporting 48 drives in the machine. Only the first and 4th drive on the I think 4th controller are bootable. Are you using the latest firmware? If not, you're going to have to pay Oracle for the privilege of updating it as there is no way the machines are still under warranty. I'd find a copy of the OpenSolaris Live CD and see if that boots and supports all your drives. Hope this helps you, or helps someone else on the list with more knowledge of debugging older AMD systems point you in the right direction. Bob Healey Systems Administrator Biocomputation and Bioinformatics Constellation and Molecularium healer@rpi.edu (518) 276-4407 On 1/17/2012 5:09 PM, Jeremy Chadwick wrote: > On Tue, Jan 17, 2012 at 06:59:08PM +0100, peter h wrote: >> I have been beating on of these a few days, i have udes freebsd 9.0 and 8.2 >> Both fails when i engage> 10 disks, the system craches and messages : >> "Hyper transport sync flood" will get into the BIOS errorlog ( but nothing will >> come to syslog since reboot is immediate) >> >> Using a zfs radz of 25 disks and typing "zpool scrub" will bring the system down in seconds. >> >> Anyone using a x4500 that can comfirm that it works ? Or is this box broken ? > I do not have one of these boxes / am not familiar with them, but > HyperTransport is an AMD thing. The concept is that it's a bus that > interconnects different pieces of a system to the CPU (and thus the > memory bus). ASCII diagram coming up: > > +-----------------------+ > | RAM | > +----------+------------+ > | > +----------+------------+ > | CPU (w/ on-die MCH) | > +----------+------------+ > | > +----------+------------+ +-----------------------------+ > | HyperTransport bridge +-----+ PCI Express bus (VGA, etc.) | > +----------+------------+ +-----------------------------+ > | > +----------+---------------+ > | Southbridge (SATA, etc.) | > +--------------------------+ > > ZFS is memory I/O intensive. Your controller, given that it consists of > 25 disks, is probably sitting on the PCI Express bus, and thus is > generating an equally high amount of I/O. > > Given this above diagram, I'm sure you can figure out how "flooding" > might occur. :-) I'm not sure what "sync flood" means (vs. I/O > flooding). > > Googling turns up *tons* of examples of this on the web, except every > time they involve people doing overclocking or having CPU-level problems > pertaining to voltage. > > There may be a BIOS option on your system to help curb this behaviour, > or at least try to limit it in some way. I know on our AMD systems at > work the number of options in the Memory section of the BIOS is quite > large, many of which pertaining to interactivity with HyperTransport. > > If you want my advice? Bring the issue up to Sun. They will almost > certainly be able to assign the case to an engineer, who although may > not be familiar with FreeBSD, hopefully WILL be familiar with the bus > interconnects described above and might be able to help you out. >