Date: Mon, 16 Apr 2007 23:28:50 +0200 From: Solon Luigi Lutz <solon@pyro.de> To: freebsd-questions@freebsd.org Subject: Re[2]: Terabyte harddisks, GELI, AMD64, Samba and Zen... Message-ID: <1133543067.20070416232850@pyro.de> In-Reply-To: <evigul$o8f$1@sea.gmane.org> References: <1387491461.20070411043619@pyro.de> <evigul$o8f$1@sea.gmane.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi again, after some troubleshooting and some hours of memory tests, it finaly seems to be a hardware problem... The machine is based on an ASUS M2N4-SLI (Nforce4) and since the heat-sink on the north/southbridge is rather small and passive, the chip seems to get too hot. I manufactured a massive one from a IGBT heat-sink and since 20 hours the machine is doing ftp-transfers without any reboots - I keep my fingers crossed... BTW a "fsck_ufs -y -f /dev/da0.eli" without sofupdates on this 10 TB volume takes only 3 hours to complete. Thanks for your help. Solon P.S. Does anybody know a way to do some performance-tuning on GELI? >> -------Sequential Output-------- ---Sequential Input-- --Random-- >> -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- >> Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU >> 1000 37922 31.5 48829 12.0 23827 5.0 30054 36.0 43666 6.0 1120.0 2.5 IV> Interesting - your CPU doesn't look overwhelmed much. >> But I have not been able to get more than 17 MB/s when using Samba to >> transfer data - FTP maxes out around 27 MB/s. I also tried that on >> i386 32-bit and found it to be 8 MB/s and 17 MB/s - not good, but >> nothing to worry. >> >> What made me feel really uncomfortable was the fact, that just some >> minutes ago some 3000 1GB files suddenly disappeared while working >> in a directory. They where gone, but the filesystem did not report >> some additional 3TB to be free and after unmounting and remounting the >> filesystem the files were back where the belonged... >> This just happened some minutes later again, now with only 2500 files >> dis- and -reappearing again. IV> This can mean either file system corruption (which fsck fixed on boot?), IV> a bug (read cache bug, where the memory representation of the directory IV> doesn't agree with on-disk state) or a hardware memory error. Of these, IV> hardware errors are easiest to check in your case. Download a memtest86 IV> boot CD ISO, burn it and let it run for a few hours. Next, you can try a IV> "full" fsck, which would probably a few last days on such a big array IV> (big arrays are inconvenient to have without journaling). If both fail, IV> we may look for a bug somewhere. >> Questions until now: >> >> 1. 10TB as a single volume, too big for good? (fsck time: 30 min with softupdates) IV> Yes, too big. Softupdates doesn't even do a full fsck - if you tried a IV> full fsck it will require about a dozen GB of memory (or memory+swap) IV> and take a really long time. If you're not scared of it, you should run IV> 7-current and re-create the file system with gjournal, or even ZFS. >> 2. GELI unstable on big disks and/or AMD64? IV> You're the first to complain :) >> 3. Why is Samba so slow? IV> Search Google... Samba is notoriously slow on FreeBSD, but there are few IV> ways to tune it which will help. >> 4. Does the crypto-framwork gain speed advantages from dual-core CPUs? IV> No, and the same goes for most GEOM classes. >> 5. Will the GPT-stuff change over the next releases in a way I need to >> do DUMP/RESTORE? IV> I don't think so, except if someone discovers an incompatibility in the IV> way FreeBSD handles GPT wrt other OSs. Shouldn't happen.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1133543067.20070416232850>