From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 13 18:14:53 2009 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 95CED1065679 for ; Fri, 13 Feb 2009 18:14:53 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from lorca.tdx.co.uk (lorca.tdx.co.uk [62.13.128.6]) by mx1.freebsd.org (Postfix) with ESMTP id 117E58FC0A for ; Fri, 13 Feb 2009 18:14:52 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from Octa64 (rainbow.tdx.co.uk [62.13.130.232] (may be forged)) (authenticated bits=0) by lorca.tdx.co.uk (8.14.0/8.14.0/Kp) with ESMTP id n1DI0Ojm011954 for ; Fri, 13 Feb 2009 18:00:25 GMT Date: Fri, 13 Feb 2009 18:00:31 +0000 From: Karl Pielorz To: freebsd-hackers@FreeBSD.org Message-ID: <12320CD678FB9B76CA7A29F1@Octa64> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: Subject: Tyan S2895 7.1 amd64 >4Gb RAM support? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Feb 2009 18:14:54 -0000 Hi, I've a Tyan S2895 (bios 1.04), w/10Gb of ECC RAM onboard using 2 * Opteron 285's. The machine used to run WinXP x64, and Vista x64 (mostly doing video production, ray tracing etc.) I recently switched this machine to FreeBSD 7.1 amd64 - to run ZFS on it, but I've been having horrific problems with it. Basically, with more than ~3Gb of RAM usable in the system - it shows signs of chronic RAM problems (everything from sig11's through to failing to compile the kernel with 'weird' errors - as well as kernel panics, and spontaneous reboots). I've tested all the RAM (ECC is enabled on the BIOS) - it all tests fine (even if I jumble it up between different simms in different sockets etc.) By setting: hw.physmem="3G" In loader.conf - I get a stable system. I've not setup any ZFS pools or anything yet, until I get the system stable. I've also tried changing the BIOS settings for the Memory Hole, IOMMU, MTRR etc. - all to no avail (nor does a BIOS 'use safe defaults' make any difference). It boots off the onboard nVidia RAID controller (a pair of 36Gb drives configured as RAID1), this shows up as: " atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1400-0x140f at device 6.0 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] atapci1: port 0x1440-0x1447,0x1434-0x1437,0x1438-0x143f,0x1430-0x1433,0x1410-0x141f mem 0xc0002000-0xc0002fff irq 22 at device 7.0 on pci0 atapci1: [ITHREAD] ata2: on atapci1 ata2: [ITHREAD] ata3: on atapci1 ata3: [ITHREAD] atapci2: port 0x1458-0x145f,0x144c-0x144f,0x1450-0x1457,0x1448-0x144b,0x1420-0x142f mem 0xc0003000-0xc0003fff irq 23 at device 8.0 on pci0 atapci2: [ITHREAD] ata4: on atapci2 ata4: [ITHREAD] ata5: on atapci2 ata5: [ITHREAD] ... ad8: 35304MB at ata4-master SATA150 ad10: 35304MB at ata5-master SATA150 ... ar0: 35304MB status: READY ar0: disk0 READY (master) using ad10 at ata5-master ar0: disk1 READY (mirror) using ad8 at ata4-master Trying to mount root from ufs:/dev/ar0s1a " Anyone got any ideas? - At this time, I can't prove 100% whether it's the disk controller messing up and corrupting data as it's loaded into RAM, or data getting corrupt once in RAM that's causing the crashes - all I know is with ~3Gb RAM - either by physically pulling SIMMs or using the hw.physmem option - it works fine. I tried booting 8.0-CURRENT-200812-amd64-disc1.iso - to see if anything was different with this hardware in 8.0 - but unfortunately that doesn't boot past the BTX loader on this machine, regardless of how much RAM is / isn't in it :( -Kp