From owner-freebsd-stable Mon Apr 30 14: 3:20 2001 Delivered-To: freebsd-stable@freebsd.org Received: from noid.org (para.noid.org [209.247.165.213]) by hub.freebsd.org (Postfix) with ESMTP id 56C0437B629; Mon, 30 Apr 2001 14:03:04 -0700 (PDT) (envelope-from mrlist@noid.org) Received: from insomnia (scream.wlv.netzero.net [209.247.163.9]) by noid.org (8.9.3/8.9.3) with SMTP id OAA25755; Mon, 30 Apr 2001 14:07:40 -0700 Message-ID: <026b01c0d1b9$161f0320$3ba2640a@int.netzero.net> From: "Nick's Lists" To: Cc: Subject: panic: pipeinit: cannot allocate pipe -- out of kvm -- code = 3 Date: Mon, 30 Apr 2001 14:03:33 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG The box is a dual P3 with 2g ram, running Qmail which is handling inbound email. I'm running 4.2 Release (though I've had similar problems with 3.4 Stable a few months back, and never solved them either). After a 36 - 48hrs, the box will panic with "panic: pipeinit: cannot allocate pipe -- out of kvm -- code = 3". It doesn't seem to happen during periods of peak load, the last crash occurred at 3:45AM and there have been days when the box has made it through our peak load times with no problems. When the box was initially installed, it was running out of files. I built a new kernel with maxusers = 512. The box seemed to last about 48 hours or so before crashing with the out of kvm panic. After a bit of digging around, I came to the conclusion (erroneously?) that the problem was caused by bumping maxusers up too high and chewing through too much memory allocating space for a large number of files and processes. As a result, I've tried maxusers at 256, 128, and 64. With maxusers at 64, I've bumped maxfiles to 5000 via sysctl. That didn't seem to make any difference, the box seemed to crash every 36 - 48 hours with the same panic. It was suggested to me a couple of days ago, to try running maxusers = 1024. I tried that after the box crashed this last time. With that setting, the box only stayed up for 6 hours, from 3:45AM to 9:45AM. The last few lines of vmstat 2 prior to the crash are: procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ml0 md0 in sy cs us sy id 10 8 0 24248 778400 3022 0 0 0 2453 0 171 0 744 10659 1586 7 28 65 5 5 0 20900 780140 3353 0 0 0 3610 0 295 0 953 8824 2595 7 39 54 0 5 0 19096 780644 1339 0 0 0 1437 0 95 0 512 3188 886 1 15 84 1 4 0 17460 781020 940 0 0 0 973 0 83 0 417 3143 711 3 14 83 1 5 0 17384 780988 1842 0 0 0 1762 0 172 0 646 5964 1271 4 21 75 5 8 0 19908 779424 1527 0 0 0 1249 0 110 0 541 5080 799 4 17 79 I was also running vmstat -m every 5 seconds - the last output of that is: Mon Apr 30 03:43:54 PDT 2001 Memory Totals: In Use Free Requests 62374K 12435K 73429925 The only notable event I've noticed is that a few minutes prior to each crash, the number of established connections (netstat -n | grep -c EST) seems to spike up pretty dramatically. About 15 min prior to the 3:45AM crash, the number of established connections spiked from the 90 - 100 range to about 250, and then dropped back to the 90 - 100 range. Another, non BSD box, running at the same weight behind the load balancer stayed in the 90 - 100 connections range at the time the FBSD box's connections spiked up. I've seen similar behavior with all the prior crashes. I have savecore enabled, but it doesn't seem to be leaving a core when it reboots. My next step is going to be to cvsup to 4.3-stable and enable ddb. I'm doing this mainly in an effort to be on the latest in the hopes that this was fixed between 4.2 and 4.3. Does anyone have any suggestions? TIA! - Nick To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message