From owner-freebsd-stable@FreeBSD.ORG Mon Sep 2 06:16:48 2013 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 41046A3F for ; Mon, 2 Sep 2013 06:16:48 +0000 (UTC) (envelope-from trtrmitya@gmail.com) Received: from mail-lb0-x236.google.com (mail-lb0-x236.google.com [IPv6:2a00:1450:4010:c04::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B97AF208F for ; Mon, 2 Sep 2013 06:16:47 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id c11so2952706lbj.13 for ; Sun, 01 Sep 2013 23:16:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=KOKkEM+6nlHVIggYSMNtVFSg+kPiKQ2iDGQQQTzQiO4=; b=gmzQXUQmzUVEjvRx0+HX16LCaHiHA1/5SbbP31kzzr2Oc7B2EHy8n66o3BiMval4Mt TzHiqBm8JyHdFb/iNzPNnPMYql6FC5xVAeM2YW76+Tuccrs2Jih0AIFLjy+aKIcFyuOU q3P5+cX+h6BRtVDPHVErBR6nCIcQ38Qffckk9PcQAq5cxGG9nPdQHz8FiAtB3Cues141 qeuEA+uMm/U7gM65qscejiSJBPOOShO/C6hHG8QqbRnLZ4K5vMdsqFqAhem2GNGk5OAi CEuboeYdzEPjfDWYVStpIN3d5Hgaobf0XdptcfHdMtd1n39UbdB7ohcZx1KZM94BkJFX m2Eg== X-Received: by 10.152.21.225 with SMTP id y1mr20265933lae.18.1378102605540; Sun, 01 Sep 2013 23:16:45 -0700 (PDT) Received: from dhcp174-208-red.yandex.net (dhcp174-208-red.yandex.net. [95.108.174.208]) by mx.google.com with ESMTPSA id w10sm4989314lbv.6.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 01 Sep 2013 23:16:44 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Subject: Re: 9-STABLE panic on intensive fork From: Dmitry Sivachenko In-Reply-To: <20130829184515.GN4972@kib.kiev.ua> Date: Mon, 2 Sep 2013 10:16:43 +0400 Content-Transfer-Encoding: quoted-printable Message-Id: <99023A4B-AA44-45DD-B163-D9FF23E7AEA4@gmail.com> References: <20130829184515.GN4972@kib.kiev.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.1508) Cc: stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Sep 2013 06:16:48 -0000 On 29.08.2013, at 22:45, Konstantin Belousov = wrote: > On Wed, Aug 28, 2013 at 06:20:29PM +0400, Dmitry Sivachenko wrote: >> Hello! >>=20 >> I am using very recent FreeBSD-9-STABLE snapshot: >> 9.2-PRERELEASE FreeBSD 9.2-PRERELEASE #0 r254986: Wed Aug 28 17:18:57 = MSK 2013 >>=20 >> I run uwsgi program (ports/www/uwsgi) on that machine. >>=20 >> When uwsgi starts, it forks pre-configured number of worker = processes. >> If I raise workers parameter high enough (128), I get kernel panic = (100% reproducible): >>=20 >> Fatal trap 12: page fault while in kernel mode >>=20 >> If I compile kernel with KDB enabled, I get the following stack: >>=20 >> pmap_demote_pde_locked() >> pmap_copy() >> vmspace_fork() >> fork1() >> sys_fork() >>=20 >> I have only remote console for that machine, so I made 2 screenshots: >>=20 >> 1) http://people.freebsd.org/~demon/screen1.jpg >> Panic screen when kernel has no KDB support compiled in >>=20 >> 2) http://people.freebsd.org/~demon/screen2.jpg >> Panic screen (2nd part) with the above stack shown. > Look up the source line for the pmap_demote_pde_locked()+0x471 for = your > kernel. Dump the core from the panic. Kernel dump is not generated (despite it is configured at boot), there = is no "Dumping...." message on console. These screenshots shows everything I see on console. I performed some more investigations on this: I have several (14) totally identical configured machines running = exactly the same software. Hardware is a bit different though. I tried to analyze motherboard = differences but failed to find common things for the affected machines. Under conditions described in my initial e-mail, some of them crash = (exactly the same way), some of them do not. I am confident there is no hardware problems, these machines run for = months without reboot, as for now I discovered the only way to crash = them. I updated one of the affected servers to 10-current and I can state it = does not crash anymore with the same usage scenario.