From owner-freebsd-questions@FreeBSD.ORG Sun Dec 30 17:31:37 2007 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6211116A418 for ; Sun, 30 Dec 2007 17:31:37 +0000 (UTC) (envelope-from phatbuckett@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.178]) by mx1.freebsd.org (Postfix) with ESMTP id 368E713C44B for ; Sun, 30 Dec 2007 17:31:37 +0000 (UTC) (envelope-from phatbuckett@gmail.com) Received: by wa-out-1112.google.com with SMTP id k17so7452570waf.3 for ; Sun, 30 Dec 2007 09:31:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=PNWx6w8pydWn9xihByMsQK/dKlxFk5NjYPfBaMkhTBI=; b=bPKEjYXRBjRNFBgCiAQCco9S0/Y8BMevM521f0ssBTKbnzJc/puGmSQ+mX0DoTr+wXtMBwuO7oBws4YeRggaTQqiXgy5w89ldLqznjp0uJxRHMInvTe0baOyCHv89T606FJfTaLl+os9VLZ9a2b18i+IDIKzl212TQRzCUHVEoo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=mahMAFbC7ROFmz2T1XsfTbfeoZUQ0Wr9rZ/xvSwmzRelxQ0XL+7Dphu6OuEi+m44gv0S0kXCmMt4MnxqiEkl1QWNAvwCY/FUZcRGcS34S2Clo1Xv43KcznX7EWrt2anIpsTTNXxe1nxpT8ktMERO7PaZ1zwfvclYqGhHRvGyMGg= Received: by 10.114.88.1 with SMTP id l1mr10570636wab.79.1199035896910; Sun, 30 Dec 2007 09:31:36 -0800 (PST) Received: by 10.114.47.12 with HTTP; Sun, 30 Dec 2007 09:31:36 -0800 (PST) Message-ID: <839aec700712300931l18d892dak3fe16175405fa65f@mail.gmail.com> Date: Sun, 30 Dec 2007 10:31:36 -0700 From: "Darren Spruell" To: "Maxim Khitrov" In-Reply-To: <26ddd1750712300852p5835e094nf20f85a9f8e0356d@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <26ddd1750712271246j14795cf3wf8e9727f0f7cc148@mail.gmail.com> <47744048.6020202@daleco.biz> <47752446.8090908@gmail.com> <26ddd1750712300852p5835e094nf20f85a9f8e0356d@mail.gmail.com> Cc: Rob , User Questions Subject: Re: Blocking undesirable domains using BIND X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Dec 2007 17:31:37 -0000 On Dec 30, 2007 9:52 AM, Maxim Khitrov wrote: > > I was trying to do something similar. I didn't research too hard, but figured the only way to use Bind would be to make my server authoritative for all those domains, which meant a huge config file and potential overhead, as well as > > possibly breaking access to desirable servers in the domains. > > > > So hosts seemed easier, but apparently Bind never looks at hosts. I did find that Squid (which I already had installed and in limited use) has its own DNS resolver, and it does look at hosts first before going to the nameserver. > > > > Then I found this site: http://everythingisnt.com/hosts.html and put their list in hosts, and now client PCs get a squid error in place of ad junk. Works ok for me ;) > Well... you were right about overhead. In the last two days I wrote a > script that would fetch a list of domains from several different > sites, and output a valid BIND configuration file that could be > included in the main config. I just ran the second test and the > results are extremely poor. With only 27,885 blocked domains the > server is now consuming 208 MB of ram. The first time I tried > reloading the full list of domains (91,137 of them) and that nearly > crashed my server. Had to kill bind, remove two of the largest > sources, and try a second time. Nearly 100,000 zones on that server is a fairly impressive amount. Give it credit for what you're trying to do. :) Nonetheless, crashing is unacceptable. > Honestly, I can't figure out what BIND could possibly be using so much > memory for. It's taking up about 7 KB for each zone. The zone file > itself is not even 1 KB, and given that all the records are pointing > to the exact same thing it seems to be needlessly wasting memory. In > addition to that, if I comment out the blacklist config file and run > rndc reload, it only frees up about 16 MB. So it doesn't even release > memory when it is no longer needed. My experience, albeit with a smaller number of zones, is a bit different. First you need to account for main program memory and memory utilized by the nameserver's cache, if any. You may also be running your own authoritative zones which will add memory utilization outside of that. You can't account for all of the utilized memory in your additional blocking zones. Without my blocking zones loaded, I have 6 native zones on my nameserver and the resident memory size of named is 2.2 MB. After a fresh server startup, I expect minimum memory for cached records, so that comes out to be about 375 KB/zone, unscientifically. If I restart named (kill and start server fresh) with my blocking zones in the config, I come out with 17239 zones and a resident process memory size of 59 MB. (Unscientifically again,) this breaks down to about 3.5 KB/zone. In my configuration, each of these blocking zones points to a simple zone file 244B in size on disk: $TTL 86400 @ IN SOA ns.local. admin.local. ( 1 ; serial 1h ; refresh 30m ; retry 7d ; expiration 1h ) ; minimum IN NS ns.local. IN A 127.0.0.1 * IN A 127.0.0.1 So all told, I seem to notice somewhat slimmer utilization than you (roughly half the memory utilization per zone, and though I have 61% as many zones loaded my named takes only 28% of the memory yours does.) > It looks like my plan of using BIND for filtering purposes will not > work. Given how poorly it performed on this test I'm actually inclined > to try another name server to see if something else would be more > memory-efficient. You will almost certainly find most of the popular alternatives to be much more resource efficient. djbdns in particular would be my next choice if memory efficiency and stability are concerns. DS