Date: Sun, 22 Feb 2004 14:26:34 +0100 From: "Willem Jan Withagen" <wjw@withagen.nl> To: <freebsd-amd64@freebsd.org> Subject: Re: System advice requested Message-ID: <062501c3f947$7dfc5930$4d1b3dd4@digiware.nl> References: <0b6801c3f21f$1fd470b0$471b3dd4@dual>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format. ------=_NextPart_000_0622_01C3F94F.DF6CADC0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit ----- Original Message ----- From: "Willem Jan Withagen" <wjw@withagen.nl> > If I were to get a dual opteron board/system what things would be > to watch out for, or would be required. Hardware wise that is. > > In the beginning I would like to use is as: > 1) FreeBSD/windows/linux desktop > and lateron it would turn into my > 2) Home NFS/Samba-server running FBSD > > In case 2) I'd remove any fancy user-interface stuff... > > All suggestions welcomed, To follow-up on myself... The attachement contains all(??) comments about hardware on this list in the last 4 months. Currently totally completely unorganised. (Other than the most recents comments are first in the file) Given some more time, I'll try to organise it more sensible. And of course: comments welcome. --WjW ------=_NextPart_000_0622_01C3F94F.DF6CADC0 Content-Type: text/plain; name="dual-opteron.txt" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="dual-opteron.txt" Gobrien at freebsd.org: you want to populate memory in pairs per CPU. This is so you can use both channels of the DDR memory controller. If you only put one DIMM per CPU, you're missing 1/2 your potential bandwidth.=20 =3D=3D=3D=3D james-freebsd-amd64 at jrv.org=20 > Note 5.2.1 also adds the Silicon Image 3114 SATA support needed to > install on a SATA drive in many Opteron systems. Is anyone else having trouble with this? The 3114 is found no problem but the drive is not seen. The culprit may be this code in ata_reset(). =3D=3D=3D=3D billsf at curacao.n2it.nl: > Is anyone using an Adptec 29320? Mine is not working. > I suspect is may be a BIOS problem on the Tyan K8W S2885. > I also have a Highpoint 1820 that does not work right. Not anymore. It worked for awhile and after a few days it just stopped. Next i was given a PERC 320/DC Adaptec FSA RAID (aac) This was much = better but still an Adaptec. I'm using a 29160 now and it is going to go! These cards just don't seem to be up to a busy Unix environment. In all cases the firmware was the latest. (Surprisingly FreeDOS runs quite well on=20 the amd64, and needed to run the flash.exe stuff.) I'm getting a real Cirrus Logic card. These cost a third what Adaptec costs and my = experience has only been good with these cheap (smb) cards. I work with servers and most BOFH's swear by them and deplore Adaptec. (Therefore all those = 'free' cards to try. ;-) Gladly stump up a hundred Euros or so for piece of = mind. =20 Presently I'm using an Asus K8 motherboard for an Opteron-100 marketed = as "Athlon-FX" or something. Not bad for Taiwan, but apparently Taiwan = thought amd64 was going to be Windows and therefore some really goofy features.=20 All AMD products seem to work better slightly overclocked. 2250MHz for a = 2200MHz chip is perfectly inline with the 3 - 5% rule. When increasing = the=20 bus speed, Adaptec cards give out first, then SATA and then the video = and=20 so on. Sorry, this is a bit offtopic. =3D=3D=3D=3D bob at immure.com: Well, my early Tyan Athlon MBs (Tiger S2460) didn't like Crucial DIMMs much, and my later Tyan boards (Tiger s2466's) won't work with the Crucial DIMMs at all (I have about 8 of them on a shelf that I can't use at all now). I never have seen any ECC errors (there registered ECC DIMMs) with these, just varying degrees of system unstability (random memory erors/panics). A couple of years ago I spent quite a bit of time on the phone with Crucial's tech support folks (shortly after I bought the memory) with no real resolution (the tech support guy tried to be helpful but couldn't really help). Following this experience, I have come to worry much more than I previously did about memory compatibility issues; and I began to wonder just how much good ECC memory really does in these systems when you wind up with all kinds of system instability. I now worry more about compatibility than do ECC vs. non-ECC memory. Note, that my newer Tyan dual MB systems (those with the S2466 MBs that can run with unbufferred/non-ECC DIMMs) have been running with unbufferred/non-ECC DIMMs for some time (over a year) and I have had much better stability with these systems than when I tried the Crucial reg/ECC DIMMs (generally, they either wouldn't boot at all or only ran for a short time, depending on how many DIMMs I installed--more meant shorter time to crash). Note that I never did try other brands of reg/ECC memory on these systems since they were stable with the cheaper stuff and I was growing tired of spending money on them. =3D=3D=3D=3D obrien at freebsd.org: Tyan GX28 support? > The onboard sATA on the S2885 isn't supported. The GX28 > uses a Promise controller chip that is more likely to work. > I have no idea if the LSI SCSI controller is supported. > It's a BCM5704c ethernet: mine is a 5705. Both the LSI SCSI and BCM5704c are supported. =3D=3D=3D=3D=3D obrien at freebsd.org: > what motherboard do you guys recommend for use on a dual cpu ssytem.=20 Server or workstation?? Workstation get the Tyan Thunder K8W or Iwill DK8X. Server get Arima RioWorks, or Tyan Thunder K8S PRO. > would prefer ddr non registered memory capability and sata drive = support. ALL Opterons require registered ECC memory. =3D=3D=3D=3D james-freebsd-amd64 at jrv.org: My Tyan K8W S2885 works fine with its onboard Broadcom. Will's problems appear caused by ACPI problems, which is a property of the particular BIOS the motherboard interacting with whatever bugs exist in FreeBSD's ACPI interpreter. Your Tyan probably works as-is. =3D=3D=3D=3D obrien at freebsd.org: > I downloaded the 5.2-RC2 AMD64 miniinst ISO to try on a MSI K8T800 > based dual opteron system today. What is the board model number? I am not aware of any K8T800 based SMP boards. The only MSI 2P board I know if is the K8D Master MS-9131 which is AMD 8100. =3D=3D=3D=3D chat95 at mbox.kyoto-inet.or.jp: Hm, I know it, but unfortunately, there are no data for Tyan thunder k8w (S2885) and Rioworks HDAMB-WO... anyway, for Rioworks HDAMB-WO, o RIOWORKS HDAMB-WO http://www.rioworks.co.jp/motherboard/popup/hdamb.html I found a PDF (though it was written in Japanese). o AMD Opteron Processor Compatible DDR Memory Modules AMD Tested Memory Validated DIMMs - AMD Opteron Processor = http://www.amd.com/us-en/Processors/ProductInformation/0,,30_118_8796_881= 9%5E9394,00.html =3D=3D=3D=3D obrien at freebsd.org: At work we're pretty sloppy about what random no-name DIMM's we use. Only the Shuttle box has been picky. That said, Micron (MT) / Crucial DIMM's are probably the best quality. Samsung DIMM's also. These are the two brands AMD uses in the development systems it sends out. I've got a case load of Kingston 1GB ValueRAM in which 1 out of 4 DIMM's is bad. =3D=3D=3D=3D Markus at kpnqwest.ch: > The motherboard has the same Sil3114 SATA controller as the Tyan = Thunder K8W=20 > (S2885). I have seen other posts to this list that indicate success = in=20 > installing FreeBSD on the Thunder K8W, so I'm wondering why I can't = see the=20 > drives on my system. I made a simple modification to the driver which essentially maps the = 3114 as a 3112, which works, sort of (not with the recent changes though, I'm = getting those "TIMEOUT - ATAPI_IDENTIFY retrying messages" that others had = mentioned as well). The drives do not go into SATA DMA mode (150Mbps), but I have = the=20 slight suspicion the code doesn't really support those yet with the SiL = chipsets. On very high load on the drives, I sometimes get=20 ad4: WARNING - WRITE_DMA UDMA ICRC error (retrying request) style errors, but all in all, it's working very nicely. =3D=3D=3D=3D obrien at freebsd.org: > You might be able to save some money by using an Athlon64 CPU and > motherboard instead of Opteron, but registered DIMM might not work. Registered(buffered) DIMM's will not work in Athlon64 motherboards. =3D=3D=3D=3D obrien at freebsd.org: > Adaptec 3950U2B hosting a pair of LVD drives > Adaptec 2400A IDE RAID hosting a quartet of WD 120GB Sp. Editions > ATI Radeon 7200 (aka Radeon VIVO) > Netgear FA310TX serving the outside world > Netgear FA302T serving the inside LAN All of this will work. =3D=3D=3D=3D WillS at housing.ufl.edu: If you used an Opteron 240 on an MSI K8T Master2-FAR board you would spend ~$600. That board is ATX-sized so it doesn't = require any special case. Your RAM is too slow, and I don't know if you can run the memory bus=20 asynchronously, so if it were me I would figure on buying more RAM. I = think you can get 1GB of Corsair Registered ECC DDR400 in a 'matched pair' of=20 512MB modules for less than $400, so that would put you under your = spending limit. =3D=3D=3D=3D obrien at freebsd.org: > Your RAM is too slow, and I don't know if you can run the memory bus=20 > asynchronously, so if it were me I would figure on buying more RAM. Huh? I don't follow, why is running PC2100 in any AMD64 machine running the memory bus asynchronously?? PC1600 memory will work fine any any AMD64 mobo, so will PC2100 and PC2700. You'll need the latest BIOS's and a rev.C CPU to run DDR400. =3D=3D=3D=3D obrien at freebsd.org: Well tested are Matrox G4[05]0 and nVidia GeForce{3,4}. I have no ATI experience on FreeBSD/amd64. =3D=3D=3D=3D tilman at arved.at: I use an none0 at pci1:0:0: class=3D0x030000 card=3D0x596412ab chip=3D0x59641002 = rev=3D0x01 hdr=3D0x00 vendor =3D 'ATI Technologies' device =3D 'Radeon 9200 Series' class =3D display subclass =3D VGA =3D=3D=3D=3D obrien at freebsd.org: I don't quite follow you. You are using the 2015S ZCR in 32-bit or 64-bit FreeBSD? It should work with FreeBSD/i386 on Opteron HW. asr(4) isn't 64-bit clean and thus asr(4) devices aren't supported in the FreeBSD/amd64 OS. =3D=3D=3D=3D tomh at waterloo.equitrac.com: All right, so I gave up on my Adaptec 2100S RAID controller, and managed = to locate a used HP NetRAID 3Si (aka MegaRAID 438). The NetRAID is only = U2W (80 MB/s, as I recall?), but FreeBSD/amd64 seems to be much happier with = it. =3D=3D=3D=3D peter at wemm.org: The only major chunk of hardware that I know of that has big issues is = the=20 Promise SX6000. The driver tries to use a 32 bit field in a hardware = data=20 structure to store a 64 bit pointer. =3D=3D=3D=3D james-freebsd-amd64 at jrv.org: I'm using a Tyan S2885, which appears somewhat similar to this. FreeBSD has run flawlessly for me - no reboots, crashes or other headaches. I'm using FreeBSD 5.2 RC2. The onboard sATA on the S2885 isn't supported. The GX28 uses a Promise controller chip that is more likely to work. I have no idea if the LSI SCSI controller is supported. It's a BCM5704c ethernet: mine is a 5705. I'm using 8 GB of memory (8x Crucial part #CT12872Y335) and a HighPoint 1542 sATA controller. =3D=3D=3D=3D obrien at freebsd.org: > On the Tyan 2880 motherboard. I know you've already bought it... but to everyone else I strongly recommend not buying this motherboard. The 4+2 DIMM configuration[*] is sub-optimal and you can get 4+4 DIMM configuration from Tyan (K8S Pro) and other makers. [*] I fully know the bad history of Opteron 4+2 DIMM configurations... but can't speak publically about it. :-( =3D=3D=3D=3D peter at wemm.org: The ones I have personal experience with are: 940 pin: Asus SK8N: works (nForce3 chipset, no driver for onboard nVidia nic, = onboard promise 378 raid+SATA) Rioworks HDAMA: works (AMD chipset, 2 cpu, onboard if_bge gigabit, in = use on the freebsd.org cluster as sledge.freebsd.org and at = work) 754 pin: Asus K8V: works (VIA K8T800 chipset, onboard ide not recognized without patch which has integrated SATA support, decent onboard 3com/syskonnect gigabit nic (if_sk), onboard promise = 378 raid+SATA) Gigabyte K8NPro: works (nForce3, lousy tiny bios interface, realtek 8110 gigabit nic (if_re) - supposedly not bad. onboard = nVidia ethernet isn't connected (but we dont have a driver = anyway)). AMD Solo4, Solo5 and Solo7 (developer reference boards) I really like both of the ASUS boards for personal use and the Rioworks = for servers. I have not seen any of the MSI boards so I can't vouch for = them. The MSI 2xCPU board only has 2 dimm slots on one of the cpus so I'm not that interested in their server board. Their consumer board appears to have similar specs to the asus K8V but I haven't tried it. All of these boards work with ACPI, minus some quirks with ordering the serial ports. Some of them list the COM2 port first in the tables which means we assign 'sio0' to it. Thats only a minor annoyance though. The onboard nVidia ethernet is annoying because they only give out a = binary driver blob that you write an OS shim to use. This works for i386, but obviously isn't going to work too well for an amd64 kernel. The only = board affected by this is the Asus SK8N, but ethernet cards are not a big = deal. I really really dislike the bios on the Gigabyte, and really like the = ones on the ASUS boards. The Gigabyte bios interface is almost worthless and gives you very little control over anything. If it had to be named, I'd call it a 'BIOS for dummies'. On the other hand, the ASUS bios' give = you more than enough rope. For example, Gigabyte have 'PNP OS: Yes' by = default and dont give you the ability to turn it off. ASUS do let you turn it = off. =3D=3D=3D=3D obrien at freebsd.org: I've been asked a few times what case one can use in building a = dual-proc Opteron workstation. So I thought I'd answer it publically for all. AMD uses the InWin Q500 http://www.pcstats.com/articleview.cfm?articleID=3D570 with the internal = 3" drive cage removed. This case has tons of room and even with the 3" drive cage removed there are still 5 drive bays left. This case is US$69.32 from http://www.pricetool.com/pr-In_Win_IW_Q500_IW-Q500_ATX_Full_Tower_Case?sp= =3Dgcm&tid=3D921103 If you want a shorter case, I have also installed both Tyan K8W (s2885) and MSI-9131 mobos into the Antec SX835II Performance Series II = mid-Tower case http://www.antec-inc.com/pro_details_enclosure.php?ProdID=3D80843. Again with the internal 3" drive cage removed. =20 =3D=3D=3D=3D eric at fnordsystems.com: The "Antec" cases are also available under a variety of other names, = they are actually manufactured by: http://www.chieftec.com/products/products.htm =3D=3D=3D=3D WillS@housing.ufl.edu: At least with my board (an 'awful' 4+0 MSI K8T Master2-FAR board) the = onboard broadcom=20 NIC does not work correctly with bge. It does work with ndis though. = Also for whatever=20 reason -current as of last weekend would lock the machine up hard after = ~ 30 minutes.=20 Haven't had time to work on why. -Will =3D=3D=3D=3D On Sat, Feb 14, 2004 at 11:26:06PM +0100, Adriaan de Groot wrote: > Take a look at the archives of this list (via > <http://lists.freebsd.org/mailman/listinfo/freebsd-amd64>; ), there's = lot of > hardware mentioned. I think a lot depends on how much memory you need = to > support your users: Athlon64 boards (single CPU) tend to top out at 4G > theoretical and 3G practical (like my Asus k8v, with 3 slots so I = could > stick 3G of DDR266 in it) ; Opteron boards for single CPU with 4 slots = top > out at 8G theoretical (4x2G Registered), but I hear tales of an sk8n > supporting only 3.5G. Dual boards run up to 8 slots, for 16G of = memory. Not just SK8N's. Just about every AMD64 mobo will "loose" physical memory from 3.5-4GB. So one would get 17.5GB in your example of 8x2GB DIMM's. The reasons have been explained in this list before. =3D=3D=3D=3D nVidia nForce3 [for AMD64] chipsets are very problematic for Unix(BSD)/Linux. I would avoid them if you want to run a non-MS-Windows operating system. --=20 -- David (obrien@FreeBSD.org <mailto:obrien@FreeBSD.org>) =3D=3D=3D=3D From: "Haapanen, Tom" <tomh@waterloo.equitrac.com> Andrew Gallatin: > Thanks. But are you saying the VIA K8T800 chipset is flakey too, and I = > should only consider AMD-8000 based boards? Just from scanning the=20 > last month or so of archives, it looks like the bad hardware is pretty = > evenly distributed across chipsets, but perhaps that's not enough=20 > perspective. I'm new here, after all.. Absolutely not saying that. I don't have any experience with = K8T800-based boards at all. But many people have been highly successful with the Tyan boards (I have been running a K8S Pro-based server in production for more than two = weeks now, and the mysqld process is approaching 100 hours of CPU time.)=20 So I am only suggesting them as an option ... =3D=3D=3D=3D Tyan K8W here with dual 242's. Love it. Cheers, Dylan Carlson [absinthe@pobox.com] ------=_NextPart_000_0622_01C3F94F.DF6CADC0--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?062501c3f947$7dfc5930$4d1b3dd4>