Date: Thu, 17 Dec 1998 05:05:34 -0600 From: George Wenzel <gee2@realtime.net> To: freebsd-isp@FreeBSD.ORG Subject: Usenet performance issues (was Re: RAID solutions?) Message-ID: <3678E57E.12B2@realtime.net> References: <l03110705b29e428c72d1@[192.168.1.10]>
next in thread | previous in thread | raw e-mail | index | archive | help
Matthew Patton wrote: > > for a news spool you want SPEED, SPEED, and SPEED!! You do *NOT* want RAID5 > (it's damn slow on writes) and most definately NOT in software. Get a slew > of disks and use raid 10 (stripe + mirror) for utmost speed. Given modern > disks, 4 on a channel will tend to saturate the bus. Theoretical > performance numbers will not be reached due to firmware bugs in drives > (common) or inefficient drivers. > This is all very true... Unless the entire application is designed from top to bottom to take advantage of every cpu cycle and every disk rotation, theoretical perfprmance numbers mean nearly nothing. The last INN machine I ran went through a file by file analysis of performance so we could tune things to the best advantage. We went so far as to have mirrored load-spread filesystems for single files. In INN1.4 days, the active file was 85% of all IO, so our solution was to use the smallest drive we could find that had the largest on-drive cache we could find. We ended up with the active file being smaller than the memory cache on the drives, so we avoided rotational latency issues and though the file was still served off the scsi bus, we were able to squeek past one of the theoretical performance limits of the drives. As we saw Usenet growing, we were very concerned that INN wasn't growing with Usenet. INN was improving, but Usenet was growing faster. For a fraction of the cost of yahu (yet another hardware upgrade) we purchased better software. The result? I'm using Dnews, a commercial software package costing less than $500... and my hardware appears idle. I pull in about 2 megabits average on this machine, yet the load remains very low, and the filesystem has plenty of idle cycles. This machine has the appearance of doing a fraction of the work my old INN workhorse was doing. Dnews was more than a major performance gain. In the INN days, being a news administrator was a big part of my daily job. With Dnews I check on news when *I* have time. I never find myself doing the kind of work i did under INN. It took me 30 minutes to bring the Dnews machine on-line, and over the course of a year and a half I have spent a total of about 4 hours maintaining it. I hazard to guess I would have spent 400 to 1000 hours had I still been running INN, and there would have been numerous software upgrades and other events resulting in downtime. We did invest about 20 hours of programming time wiring Dnews into our user authentication system, which included custom software mods added by the Dnews programmers. The Dnews guys did my mods for free, turning them around in about 2 days! The <$500 price tag I paid for the software was way too low for what I got. Dnews is not the only software out there that is better than INN... it is just the one I tried. I have heard similar things about several other packages, but I'll let those software users speak for themselves. I have pointed other ISP's at Dnews, and so far I have heard nothing but good things from them. I run my Dnews on a dual pentium pro (180MHZ) system, using the Dmulti feature of Dnews to run 4 copies of the executable... 1 for incoming news, and three for readers. The SMP was not needed for this application after all, were I rebuilding the machine today I would stick to a single CPU. Dnews handles article storage across multiple filesystems using a small number of files per drive (article files are grouped together in larger clusters called "buckets" keeping the filecount low). It is just a guess, but I'd venture that a 100mhz pentium with a medium speed filesystem would be enough to make a full feed work through Dnews. Raid 10 on super-servers with lots and lots of performance tuning.... Seems like too much work and money to me. Oh and the Dnews guys do mail servers too... You guys being shut-down because spammers are hammering your sendmail... There ARE performance options for you out there... Find out about Dnews and Dmail from www.netwinsite.com. While we are talking about news, I should mention SkyCache. (www.skycache.com) Through Skycache, for a fraction of what I nomally pay for bandwidth, I get a full news feed pulled off of a dish sitting on my roof. The news I get off of the dish is fresh and always here before news from any of my NNTP peers and backbone feeds (CWIX and Sprint). For a small monthly payment these guys take a solid two megabits off of my backbone load... and provide the hardware. Skycach is meant to be a web cach pre-load service, but I won't get into that part here... find out more about skycache on your own by going to their web site. Bottom line, a few well spent dollars on the right products can make your life easier. INN was a GREAT improvement over Cnews (how many of you out there remember Cnews?)... and the guys at ISC deserve a lot of thanks for the work they have done. Dnews is a departure from the old ways of doing things, so if you are willing to accept some new ideas about news, Dnews is what you need. One suggestion though... Dnews has this concept of a "sucking" feed... If you want to be a productive member of the Usenet community, I skip the part about the sucking feed and set Dnews up with normal NNTP peer sessions... sucking feeds are for end users running their own news server at home (yes, Dnews is targeted for end users who want to run mini servers on their win95 machine at home, with a <$50 price tag for the home user). (and yes, an inexperienced ISP could be very successful running Dnews on a win95 machine) George (very happy Netwin and Skycache customer) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3678E57E.12B2>