Date: Tue, 10 Apr 2007 20:22:22 -0700 From: "Jeff Mohler" <speedtoys.racing@gmail.com> To: "Solon Luigi Lutz" <solon@pyro.de> Cc: freebsd-questions@freebsd.org Subject: Re: Terabyte harddisks, GELI, AMD64, Samba and Zen... Message-ID: <a969fbd10704102022x7fd72237tdac1f5259a98dc4b@mail.gmail.com> In-Reply-To: <1387491461.20070411043619@pyro.de> References: <1387491461.20070411043619@pyro.de>
next in thread | previous in thread | raw e-mail | index | archive | help
Was your 17mb/sec a drag & drop from a windows client? 17Mb/sec is about right, as Windows deals with that as a single threaded I/O operation. You can stuff a GigE pipe from a windows machine to a netapp or some other solid CIFS server if you can fire up multiple threads to copy with. So..dont feel bad, thats just the cap of a single threaded copy from windows. On 4/10/07, Solon Luigi Lutz <solon@pyro.de> wrote: > > Hello freebsd-questions, > > I've setup a Freebsd 6.2 server with a Areca 1280 24-port SATA2-Raid > controller and some 24 500gb harddisks. CPU is a AMD64 Sempron 3000+. > > --- > FreeBSD rad.net 6.2-STABLE FreeBSD 6.2-STABLE #0: Sun Apr 8 12:39:46 UTC > 2007 > root@rad.net:/usr/obj/usr/src/sys/RADIUM amd64 > --- > > --- > Filesystem Size Used Avail Capacity Mounted on > /dev/ad10s1a 496M 70M 386M 15% / > devfs 1.0K 1.0K 0B 100% /dev > /dev/ad10s1e 496M 18K 456M 0% /tmp > /dev/ad10s1f 67G 3.9G 58G 6% /usr > /dev/ad10s1d 1.9G 1.0G 761M 58% /var > /dev/da0.eli 9.9T 4.0T 5.1T 44% /mnt > --- > > The raid contains one 10 terabyte volume, encrypted with GELI > > --- > Geom name: da0.eli > EncryptionAlgorithm: AES-CBC > KeyLength: 128 > Crypto: software > UsedKey: 0 > Flags: NONE > Providers: > 1. Name: da0.eli > Mediasize: 10999996342272 (10T) > Sectorsize: 8192 > Mode: r1w1e1 > Consumers: > 1. Name: da0 > Mediasize: 10999996350464 (10T) > Sectorsize: 512 > Mode: r1w1e1 > --- > > As I started to fill up the volume, I experienced random crashes - > always while doing high-speed transfers over gigabit ethernet between > this and another server. Mostly 1-2 per day - might be a hardware > problem or not. > > Due to the slow CPU and software encryption, the maximum speed is somewhat > limited (raw speed is somewhat around 600 MB/s): > > --- > File './Bonnie.2117', size: 1048576000 > Writing with putc()...done > Rewriting...done > Writing intelligently...done > Reading with getc()...done > Reading intelligently...done > Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done... > -------Sequential Output-------- ---Sequential Input-- > --Random-- > -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- > --Seeks--- > Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec > %CPU > 1000 37922 31.5 48829 12.0 23827 5.0 30054 36.0 43666 6.0 > 1120.0 2.5 > --- > > But I have not been able to get more than 17 MB/s when using Samba to > transfer data - FTP maxes out around 27 MB/s. I also tried that on > i386 32-bit and found it to be 8 MB/s and 17 MB/s - not good, but > nothing to worry. > > What made me feel really uncomfortable was the fact, that just some > minutes ago some 3000 1GB files suddenly disappeared while working > in a directory. They where gone, but the filesystem did not report > some additional 3TB to be free and after unmounting and remounting the > filesystem the files were back where the belonged... > > This just happened some minutes later again, now with only 2500 files > dis- and -reappearing again. > > Questions until now: > > 1. 10TB as a single volume, too big for good? (fsck time: 30 min with > softupdates) > 2. GELI unstable on big disks and/or AMD64? > 3. Why is Samba so slow? > 4. Does the crypto-framwork gain speed advantages from dual-core CPUs? > 5. Will the GPT-stuff change over the next releases in a way I need to > do DUMP/RESTORE? > > Thanks for your patience... > > Best regards, > > Solon Lutz > > > _______________________________________________ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to " > freebsd-questions-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?a969fbd10704102022x7fd72237tdac1f5259a98dc4b>