Date: Thu, 7 Mar 1996 10:40:50 -0500 (EST) From: Rashid Karimov <rashid@rk.ios.com> To: angio@aros.net (Dave Andersen) Cc: jgreco@solaria.sol.net, mtaylor@cybernet.com, hackers@FreeBSD.org Subject: Re: Proper FreeBSD news machine Message-ID: <199603071540.KAA19159@rk.ios.com> In-Reply-To: <199603070345.UAA21122@terra.aros.net> from "Dave Andersen" at Mar 6, 96 08:45:28 pm
next in thread | previous in thread | raw e-mail | index | archive | help
hi there folx, > > Lo and behold, Joe Greco once said: > > For comparison: > > > Let me sketch out my news operation here. I have multiple news peers, very > > few readers, I retain about a million articles (5 to 7 days retention).. > > We have 6 peers, and about the same amount of retention. I have 3 days and ~500.000 articles There is good home grown method to determine if you are getting full news feed. Say one is fed by 5 sites. Assuming they provide equally good and uniformly distributed news feed, you should be getting 20% from each peer , so 80% of articles should be rejected from each peer because one already has 'em. It's easy to see in the log file > > > good deal, not because CPU was much of an issue. I'd say I was running > > 30-40% idle. That suggests a DX2/66 would be squeaky, though. The box is > > now a P90 and I notice a slight performance improvement. I have PPro-200Mhz and it's mostly idle too (I peak at 200-250 nnrpd readers + 4 simult. newsfeeds ). > > We're running on a P100 and typically have 90% idle, except during the > news.daily run (which admittedly takes about 7 hours or more). > > > > B) how much RAM? 32 Mb enough? > > > > Memory: 16MB + sizeof(history.pag) * 2MB + numclients * 1MB + numfeeds * 1MB. > > > > "clients" are expected simultaneous nnrp's. feeds are outbound feeds, > > innxmit or nntplink, no matter. sizeof(history.pag) in megabytes. > > It almost sounds like you developed this forumla before the > sharedactive patch meandered its way around. I'd have agreed with you > before (it would say we needed about 56 megs of ram, and we were actually > using about 49 during non-expire times), but the sharedactive patch > reduces the memory usage from 1.3M to .3M per client (roughly) on our > system. Obviously, the larger your active file, the more benefit you'll > achieve from this. On a 64MB machine, we run with about 28 megs used for > cache on average. This is with between 5 and 20 clients. > > > it will begin to fight for pages with INN, and both your innd and expire > > processes will slow to a crawl. You also must factor in memory for other > > running processes (i.e. clients and feeds), and the OS itself needs some RAM > > (16MB, let's say, for kernel, cache, scratch, etc). > > Quite fair. I think you can squeak by on a bit less than this if > you're .. ahh, willing to put up with some burps and slowness, but given > that this machine is also going to act as a router -- I agree completely. I have two machines - one with 256Mb and one with 128M - both work fine , but the former one is more heavily loaded. There's no swapping , but expire _is slow > > > > C) would separate SCSI busses help? (I plan to put a second 4.3Gb HD > > > in for the rest of the news spool) > > > > Go PCI SCSI if you can. Also, the more disks, the merrier (I have 14 but > > then I'm a performance freak). I've tried CCD - which implemented in software kinda RAID array. What you do is you combine a few partitions ( would be best if they're from different HDs on different adapters ) to get _very fast and very big partitions. I've played with 4 HDs on 2 Aha2940 ( one was wide) adapters and I was able to peak 12+Mb read or write , and average 9+Mb/sec read and write ! This is great ! So now I'd like to try 4 4Gb wide Barracudas on 3 SCSI adaptors on a real news server. > > As a side note, be sure to keep the history* files on a separate spool > from the rest of the world. Do the same with the overfiew files if you > keep them. It'll make your life a lot happier. > > > > D) whose SCSI card has the 'best' performance? > > > > I've had good luck with the AHA3940 and NCR-810 based cards. The AHA2940 > > should work well too. > > It's a good performer, though we have occasional stability problems > with it. I think it's more due to one of the drives -- a 9gb micropolis > for storing alt.binaries -- than anything else. > > > Pay close attention to the memory advice. I see so many people try to get > > by without enough memory. It doesn't pay. I run Exec-PC's news operation > > and they try to squeeze 150-200 nnrp clients onto a box with 128MB RAM. > > They complain to me that it "takes forever to connect". I wonder why. ;-) > > That's masochism for you. :-) it often depends on the version of innd you run there. Inn1.4secunoff3 is well know for this slow nnrp start ( it pays its all attention to the newsfeeds ). There is a patched version of it which makes nnrp faster. I compiled it and one can get it from /incoming at ftp.freebsd.org Some other interesting problem is that on heavily loaded news machines the bottleneck could in the default length of the queue for TCP connections , which is "only" `5` . Making it bigger makes WWW servers mucho faster for example, so think there a way to make innd faster for reading too. Rashid
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199603071540.KAA19159>