Date: Wed, 15 Feb 1995 03:11:59 -0500 (EST) From: Rashid Karimov <rashid@rk.ios.com> To: taob@io.org (Brian Tao) Cc: lray@aurora.liunet.edu, freebsd-questions@freebsd.org, freebsd-isp@freebsd.org Subject: Re: Building a large FreeBSD news server Message-ID: <199502150811.DAA00277@rk.ios.com> In-Reply-To: <Pine.BSF.3.91.960215014407.24170d-100000@zip.io.org> from "Brian Tao" at Feb 15, 96 02:00:27 am
index | next in thread | previous in thread | raw e-mail
Hi again :), > > > what is expirience with running FreeBSD PCs as INND > > servers with full news feed ? Is it capable of handling > > the load ? Any IO/CPU limitations ? > > I have the same questions, since I'm just starting out here. My > server will be going into full production mode within a few days (it's > been running with about a dozen beta testers for a week now). > > > I run P6-200 here as news server - runs just fine, load > > averages are ~0.3-0.5 , readers peak at ~200-300, total > > of ~23.000 newsgroups in active, 3 days expire average, > > 370.000 - 420.000 inodes used in /usr/spool/uucp. > > P133, 128 megabytes RAM, 3 NCR53c810 controllers and nine disks > to be split up thusly (I don't have all the controllers plugged in > yet): > At the moment, I have the news spool spread over two 2GB Quantum > Atlas drives and a 4GB Quantum Grand Prix, running off a single > Buslogics BT-946C controller. It receives about 120,000 articles a > day (900MB or so), with 20000 groups in the active file. Only about > 7000 of those actually have articles in them so far. ;-) > > I have the history database, the overview files and the > alt.binaries spool on a single disk on the current machine, and that > disk is getting hit pretty hard according to iostat. That's being > split across three spindles and three controllers in the production > machine, so disk I/O should not be a problem. > > A delayed expire run takes less than 5 minutes to generate the > list of articles to delete, which fastrm processes in about 30 > minutes. I expect that to rise once we get a hundred readers on it at > once, and more articles to expire. > > It seems INN is only spooling about 2 articles per second, peaking > at around 5/sec for very short periods of time. Perhaps this will > improve once I'm using multiple controllers and more disks? It's interesting thing , actually ... I was trying to get some estimates on articles/sec rate. Folx at n.s.n say that it should indeed peak at 5 arts/sec for a decent feed, so I decided to check what an ftp rate would be on a local 10Mb LAN. I've tried it on FreeBSD <> FreeBSD ( P166 <> P6-200) Sun20 <> FreeBSD Sun20 <> Sun10 LocalFreeBSD <> LocalFreeBSD with 100 small files(2K) with prompt off. Guess what ! It was steady (!!!) 100 files in 20 secs , or ~5 files/sec in ALL CASES. I've tried to do ftp to the same PC from 2 different machines at the same time. It took ~23 sec to upload 200 files (100 from each PC). So it looks like for small articles/multiple feeds (and fast links :) the upper limit could be bigger than 5art/sec ( putting aside the difference " ftp versus nntp"). Other thing is that on th eame link it will take ~0.25 sec to transfer single 200K file , and even less if the thing is compressed , so batched newsfeed looks like reasonable alternative, especially for a stub news server, when one knows for sure that there won't be no "blanks" in the batch - I mean the articles the site already has. With multiple feeds the effective loss of the BW will be much bigger , especially if the server ( say, small ISP) will provide transit for the newsfeed from one major peer to another. Probably it has sense to inject only articles which were originated locally , but it's often not the point when all peers are equally interested in getting full newsfeed from each other trying to get news articles as fast as possible. Rashidhome | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199502150811.DAA00277>