From owner-freebsd-isp Sat Oct 5 13:09:19 1996 Return-Path: owner-isp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id NAA05217 for isp-outgoing; Sat, 5 Oct 1996 13:09:19 -0700 (PDT) Received: from foo.primenet.com (ip238.lax.primenet.com [204.212.59.238]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id NAA05201 for ; Sat, 5 Oct 1996 13:09:11 -0700 (PDT) Received: (from bkogawa@localhost) by foo.primenet.com (8.7.5/8.6.12) id NAA09425; Sat, 5 Oct 1996 13:14:02 -0700 (PDT) Date: Sat, 5 Oct 1996 13:14:02 -0700 (PDT) Message-Id: <199610052014.NAA09425@foo.primenet.com> To: bkogawa@foo.primenet.com Subject: How to solve the news server problem From: "Bryan K. Ogawa" Cc: dror@dnai.com, freebsd-isp@FreeBSD.ORG, pjchilds@imforei.apana.org.au (Peter Childs) X-Newsreader: NN version 6.5.0 #1 (NOV) Sender: owner-isp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk >Xref: nosuchsite localhost.freebsd.isp:402 >Newsgroups: localhost.freebsd.isp >Path: nosuchsite!bkogawa >From: bvt@mp.aha.ru (Boris Tyshkiewitch) >Subject: Re: How to solve the news server problem >Approved: bkogawa@localhost >Content-Type: text/plain; charset=KOI8-R >X-Original-Sender: owner-isp@FreeBSD.ORG >To: pjchilds@imforei.apana.org.au (Peter Childs) >Sender: bkogawa@primenet.com (Bryan K. Ogawa) >Content-Transfer-Encoding: 8bit >Organization: Sirius Cybernetics, Sirius City branch >Precedence: bulk >Message-ID: <199610051431.RAA06429@mp.aha.ru> >X-Mailer: ELM [version 2.4 PL24alpha5] >References: <> >X-Uidl: 6e4e778ce85020846f8f19f90dc0ec60 >X-Loop: FreeBSD.org >Mime-Version: 1.0 >In-Reply-To: <199610050759.RAA09217@al.imforei.apana.org.au> from "Peter Childs" at Oct 5, 96 05:29:15 pm >Cc: dror@dnai.com, freebsd-isp@FreeBSD.ORG >Date: Sat, 5 Oct 1996 19:25:24 GMT > > : I believe that my news server is spending most of its time receiving, > : writing to disk, organizing, and then removing files that NONE OF > : MY USERS WILL EVER LOOK AT. > > Yup :) > > : We could quite easily figure out which newsgroups our users subscribe > : to, accept only articles for these newsgroups and reduce the traffic, > : the disk space, the memory etc to ... 5%? 10%? 30%? The problem is > : that we want to have newsgroups available when our users want to > : subscribe to something new. > > This works ok in some cases, but not really when your other sites > expect you to provide them with a full feed. Yes, but many sites work as leaf nodes, doing all the work for the newsreaders and uucp links. (Hmm, is uucp still alive in the US?) As to this problem, I have done a small patch for nnrpd, which creates tail of read newsgroups ( touch file in special dir). So I have a list of newsgroups that are actually being read. I can expire them this way by "find . +mtime 4 -exec rm {};". Next, what can I do with this list? The first idea is to create a list for some nntp sucker (I use slurp). It's good, it works, but sucking a list of ~1000 groups is not a right thing. (transfer a 1000 group list to server, get big the list of new msgids, get the articles). It works on my server for a 2 month, and I disable it. The second idea is more interesting. In my higher news server GUP is running - a program that regulates newsfeeds' contens via email. And the only thing I have to do - it sends messages like "subscribe to this.group", "unsubscribe from that.group". This apporoch works fine. All the clients are satisfied. I have only 700-1400 _really_active_ groups (full active file is 10000 groups) It is really easy to implement - 10 line patch to nnrpd, small shell script that generate subscribe/unsubscribe messages, and GUP installed on the higher server. Boris. -- bryan k ogawa http://www.primenet.com/~bkogawa/