From owner-freebsd-performance@FreeBSD.ORG Tue Jan 8 01:37:56 2008 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 826C816A419; Tue, 8 Jan 2008 01:37:56 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from mail-out4.apple.com (mail-out4.apple.com [17.254.13.23]) by mx1.freebsd.org (Postfix) with ESMTP id 60E3E13C46E; Tue, 8 Jan 2008 01:37:56 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from relay14.apple.com (relay14.apple.com [17.128.113.52]) by mail-out4.apple.com (Postfix) with ESMTP id 1914B1E2EE67; Mon, 7 Jan 2008 17:22:54 -0800 (PST) Received: from relay14.apple.com (unknown [127.0.0.1]) by relay14.apple.com (Symantec Mail Security) with ESMTP id 0223A28088; Mon, 7 Jan 2008 17:22:54 -0800 (PST) X-AuditID: 11807134-9cdecbb0000065e4-ae-4782d06d47be Received: from cswiger1.apple.com (cswiger1.apple.com [17.214.13.96]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by relay14.apple.com (Apple SCV relay) with ESMTP id BDC8328085; Mon, 7 Jan 2008 17:22:53 -0800 (PST) Message-Id: From: Chuck Swiger To: Kris Kennaway In-Reply-To: <47803998.6020808@FreeBSD.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v915) Date: Mon, 7 Jan 2008 17:22:52 -0800 References: <47803998.6020808@FreeBSD.org> X-Mailer: Apple Mail (2.915) X-Brightmail-Tracker: AAAAAA== Cc: performance@freebsd.org Subject: Re: DNS zone query data X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2008 01:37:56 -0000 Hi, Kris-- Jan 5, 2008, at 6:14 PM, Kris Kennaway wrote: > Some months ago someone on this list offered to provide to me a data > set of DNS query data and the corresponding zone file for > benchmarking of BIND performance as an authoritative server. > Unfortunately I have lost the email and forgot who it was who made > the offer :) If it was you, please contact me again privately as I > would like to proceed with this. Was it this thread: Begin forwarded message: > From: Chuck Swiger > Date: June 4, 2007 1:21:51 PM PDT > To: Kris Kennaway > Cc: Doug Barton , freebsd-current@freebsd.org > Subject: Re: HEADS UP: BIND 9.4.1 imported > On Jun 2, 2007, at 7:27 PM, Kris Kennaway wrote: >>> For the vast majority of users, this should be a noop. Please test, >>> especially if you have a heavier loaded name server, and report any >>> issues. >> >> Also I'll remark that we remain very interested in getting access to >> either a busy nameserver or the data stream from one, in order to >> profile FreeBSD kernel activity and look for places to optimize >> performance. > > I've mentioned this before, but the dns/adns port provides some > handy utilities for putting a DNS server under high loads. > > Something like the following command will generate anywhere from 200 > queries/sec to 1500+ queries/sec, depending on the IPs involved in > the logfile you use, and how rapidly the remote nameservers respond: > > /usr/local/bin/adnslogres -c 500 < /var/log/httpd-access.log >! / > var/log/httpd-access.log.dns > > -- > -Chuck ----- Begin forwarded message: > From: Chuck Swiger > Date: June 14, 2007 4:53:01 PM PDT > To: Kris Kennaway > Cc: performance@FreeBSD.org, smp@FreeBSD.org, current@FreeBSD.org > Subject: Re: BIND 9.4.1 performance on FreeBSD 6.2 vs. 7.0 > > Hi, Kris-- > > This was interesting, thanks for putting together the testing and > graphs. > > On Jun 14, 2007, at 1:48 AM, Kris Kennaway wrote: >> I have been benchmarking BIND 9.4.1 recursive query performance on an >> 8-core opteron, using the resperf utility (dns/dnsperf in ports). >> The >> query data set was taken from www.freebsd.org's httpd-access.log with >> some of the highly aggressive robot IP addresses pruned out (to avoid >> huge numbers of repeated queries against a small subset of addresses, >> which would skew the results). > > It's at least arguable that doing queries against a data set > including a bunch of repeats is "skewed" in a more realistic > fashion. :-) A quick look at some of the data sources I have handy > such as http access logs or Squid proxy logs suggests that (for > example) out of a database of 17+ million requests, there were only > 46000 unique IPs involved. > > You might find it interesting to compare doing queries against your > raw and filtered datasets, just to see what kind of difference you > get, if any. > >> Testing was done over a broadcom gigabit ethernet cable connected >> back-to-back between two identical machines. named was restarted in >> between tests to flush the cache. > > What was the external network connectivity in terms of speed? The > docs suggest you need something like a 16MBs up/8 Mbs down > connectivity in order to get up to 50K requests/sec.... > > [ ... ] >> It would be interesting to test BIND performance when acting as an >> authoritative server, which probably has very different performance >> characteristics; the difficulty there is getting access to a suitably >> interesting and representative zone file and query data. > > I suppose you could also set up a test nameserver which claims to be > authoritative for all of in-addr.arpa, and set up a bunch (65K?) /16 > reverse zone files, and then test against real unmodified IPs, but > it would be easier to do something like this: > > Set up a nameserver which is authoritative for 1.10.in-addr.arpa > (ie, the reverse zone for 10.1/16), and use a zonefile with the > $GENERATE directive to populate your PTR records: > > $TTL 86400 > $origin 1.10.in-addr.arpa. > > @ IN SOA localhost. hostmaster.localhost. ( > 1 ; serial (YYYYMMDD##) > 3h ; Refresh 3 hours > 1h ; Retry 1 hour > 30d ; Expire 30 days > 1d ) ; Minimum 24 hours > > @ NS localhost. > > $GENERATE 0-255 $.0 PTR ip-10-1-0-$.example.com. > $GENERATE 0-255 $.1 PTR ip-10-1-1-$.example.org. > $GENERATE 0-255 $.2 PTR ip-10-1-2-$.example.net. > ; ...etc... > > ...and then feed it a query database consisting of PTR lookups. If > you wanted to, you could take your existing IP database, and glue > the last two octets of the real IPs onto 10.1 to produce a > reasonable assortment of IPs to perform a reverse lookup upon. > > -- > -Chuck