From owner-freebsd-current@FreeBSD.ORG Thu Sep 20 21:17:55 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7DC9616A419; Thu, 20 Sep 2007 21:17:55 +0000 (UTC) (envelope-from BORJAMAR@SARENET.ES) Received: from proxypop1.sarenet.es (proxypop1.sarenet.es [194.30.0.99]) by mx1.freebsd.org (Postfix) with ESMTP id E029713C633; Thu, 20 Sep 2007 21:16:39 +0000 (UTC) (envelope-from BORJAMAR@SARENET.ES) Received: from [127.0.0.1] (matahari.sarenet.es [192.148.167.18]) by proxypop1.sarenet.es (Postfix) with ESMTP id EADDD5C58; Thu, 20 Sep 2007 12:02:08 +0200 (CEST) In-Reply-To: References: <5870F83F-7174-47AA-98AE-C1DE8972E0C8@SARENET.ES> <613318C3-6B66-4758-A0D4-97405D6A1914@SARENET.ES> <46F23166.8070908@FreeBSD.org> Mime-Version: 1.0 (Apple Message framework v752.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Borja Marcos Date: Thu, 20 Sep 2007 12:02:11 +0200 To: Borja Marcos X-Mailer: Apple Mail (2.752.2) Cc: Kris Kennaway , freebsd-current@freebsd.org, Ivan Voras Subject: Re: Memory allocation problems (ZFS/NFS/amd64) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2007 21:17:55 -0000 I've just tried cranking up both vm.kmem_size and vm.kmem_size_max to the same value (1073741824), and I end up with the same problem. In order to have a repeatable load I'm using httperf --hog --num- conn=100000 --rate 600 --timeout 5 retrieving a small jpg file. The ESTABLISHED TCP sessions I see at the victim machine peak at about 800, and I start to see the same "memory allocation" messages in /var/log/messages. I'm clueless, I must admit :/ Just in case it was the problem I've set up kern.ipc.somaxconn to 4096, but the result is still the same. And disabled syncookies as well. I'm looking at the whole sysctl variables tree and I cannot see a clue. Just to refresh it, it's -CURRENT from yesterday, amd64, apache22 compiled from ports. I've just tried using the echo service from inetd, and the server is keeping 1024 established connections without that error. I just don't understand why this can fail with Apache, but it seems it's an issue with -current, as we have a similar setup with 6.2(i386), same hardware, and it works. Any ideas? So far I'm getting lost :/ Borja. On 20 Sep 2007, at 11:05, Borja Marcos wrote: > > On 20 Sep 2007, at 10:37, Kris Kennaway wrote: > >> Borja Marcos wrote: >>> On 19 Sep 2007, at 19:35, Ivan Voras wrote: >>>> Borja Marcos wrote: >>>> >>>>> These are not innocuous messages, the machine is rejecting >>>>> connections >>>>> like crazy. Any ideas? >>>>> The number of established TCP connections was around 490, and the >>>>> machine has 2 GB of RAM. >>>> >>>> Just a guess: what is your vm.kmem_size_max and have you tried >>>> increasing it? >>> It's the first thing I thought, and I cranked it to a very high >>> value just in case: >>> vm.kmem_size_max: 1073741824 >> >> You actually wanted to tune vm.kmem_size too or it may not >> actually change the value used (_max is just a ceiling for >> autotuning). However if this is i386 you can't set it that high >> without also adjusting KVA_PAGES too (which has other effects). > > It's an amd64. I understand that i386 is mostly out of the question > if I want to play reasonably safe with ZFS :) > > Oh, ok. I will try with both, then. Should I try the same value? > Perhaps it's a bit high, but I understand that with a 64 bit > address space I can set it sort of arbitrarily high without many > side effects. > > ---------------- "The thing he realised about the windows was this: because they had been converted into openable windows after they had first been designed to be impregnable, they were, in fact, much less secure than if they had been designed as openable windows in the first place." Douglas Adams, "Mostly Harmless"