From owner-freebsd-questions@FreeBSD.ORG Tue Feb 5 02:08:40 2008 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 83B3F16A418 for ; Tue, 5 Feb 2008 02:08:40 +0000 (UTC) (envelope-from bc979@lafn.org) Received: from zoot.lafn.org (zoot.lafn.ORG [206.117.18.6]) by mx1.freebsd.org (Postfix) with ESMTP id 5E6C513C44B for ; Tue, 5 Feb 2008 02:08:40 +0000 (UTC) (envelope-from bc979@lafn.org) Received: from [10.0.1.2] (pool-71-109-145-228.lsanca.dsl-w.verizon.net [71.109.145.228]) (authenticated bits=0) by zoot.lafn.org (8.13.6/8.13.4) with ESMTP id m1528IK5014417 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 4 Feb 2008 18:08:21 -0800 (PST) (envelope-from bc979@lafn.org) Message-Id: <1E7F8770-4E0A-4B6A-842F-9B6D6321E5D7@lafn.org> From: Doug Hardie To: Olivier Nicole In-Reply-To: <200802050127.m151Ro3r087662@banyan.cs.ait.ac.th> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v915) Date: Mon, 4 Feb 2008 18:08:18 -0800 References: <200802050127.m151Ro3r087662@banyan.cs.ait.ac.th> X-Mailer: Apple Mail (2.915) X-Virus-Scanned: ClamAV 0.88.7/5690/Mon Feb 4 17:11:39 2008 on zoot.lafn.org X-Virus-Status: Clean Cc: freebsd-questions@freebsd.org Subject: Re: Problem with makemap and hash tables X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Feb 2008 02:08:40 -0000 On Feb 4, 2008, at 17:27, Olivier Nicole wrote: > Hi, > > I am having problem with /usr/sbin/makemap hash on huge databases > under FreeBSD 5.5 and 6.3 while it is working on 4.11. > > By huge I mean around 380,000 lines in the /etc/mail/access file. > > After approximately 375,000 lines added into access.db I get: > > makemap: access.db: line 375135: key abcd.efgh: put error: > Operation not permitted > > and no more line will add. > > I know that having 380,000 REJECT rules in access.db may not be the > best choice, but it is easy and it was working si far (still works > under 4.11). > > Did anything changed in makemap since 4.11? Any limit in the size of > the DB? > > It seems that my message yesterday was zapped by SpamAssassin (because > I included names of infamous spamers as part of my examplke), so I > rewrote it. I am just under 300K lines in that map at present. While I have always wondered where hash was going to break, I hadn't found the limit yet. However, the time to build the hash table is extremely large with that many entries. I ran some tests a couple months ago using btree for the map type (subject was 'Map size'). I didn't test over 300K entries, but it sure built the maps a whole lot faster. You might try that and see if it builds or not.