Date: Tue, 17 Jan 2012 14:09:12 -0800 From: Jeremy Chadwick <freebsd@jdc.parodius.com> To: peter h <peter@hk.ipsec.se> Cc: freebsd-stable@freebsd.org Subject: Re: about thumper aka sun fire x4500 Message-ID: <20120117220912.GA32330@icarus.home.lan> In-Reply-To: <201201171859.10812.peter@hk.ipsec.se> References: <201201171859.10812.peter@hk.ipsec.se>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Jan 17, 2012 at 06:59:08PM +0100, peter h wrote:
> I have been beating on of these a few days, i have udes freebsd 9.0 and 8.2
> Both fails when i engage > 10 disks, the system craches and messages :
> "Hyper transport sync flood" will get into the BIOS errorlog ( but nothing will
> come to syslog since reboot is immediate)
>
> Using a zfs radz of 25 disks and typing "zpool scrub" will bring the system down in seconds.
>
> Anyone using a x4500 that can comfirm that it works ? Or is this box broken ?
I do not have one of these boxes / am not familiar with them, but
HyperTransport is an AMD thing. The concept is that it's a bus that
interconnects different pieces of a system to the CPU (and thus the
memory bus). ASCII diagram coming up:
+-----------------------+
| RAM |
+----------+------------+
|
+----------+------------+
| CPU (w/ on-die MCH) |
+----------+------------+
|
+----------+------------+ +-----------------------------+
| HyperTransport bridge +-----+ PCI Express bus (VGA, etc.) |
+----------+------------+ +-----------------------------+
|
+----------+---------------+
| Southbridge (SATA, etc.) |
+--------------------------+
ZFS is memory I/O intensive. Your controller, given that it consists of
25 disks, is probably sitting on the PCI Express bus, and thus is
generating an equally high amount of I/O.
Given this above diagram, I'm sure you can figure out how "flooding"
might occur. :-) I'm not sure what "sync flood" means (vs. I/O
flooding).
Googling turns up *tons* of examples of this on the web, except every
time they involve people doing overclocking or having CPU-level problems
pertaining to voltage.
There may be a BIOS option on your system to help curb this behaviour,
or at least try to limit it in some way. I know on our AMD systems at
work the number of options in the Memory section of the BIOS is quite
large, many of which pertaining to interactivity with HyperTransport.
If you want my advice? Bring the issue up to Sun. They will almost
certainly be able to assign the case to an engineer, who although may
not be familiar with FreeBSD, hopefully WILL be familiar with the bus
interconnects described above and might be able to help you out.
--
| Jeremy Chadwick jdc@parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, US |
| Making life hard for others since 1977. PGP 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20120117220912.GA32330>