From owner-freebsd-stable@FreeBSD.ORG Sun Oct 21 12:14:08 2012 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4BD9F900 for ; Sun, 21 Oct 2012 12:14:08 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id D5F868FC08 for ; Sun, 21 Oct 2012 12:14:07 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q9LCE8G4090657; Sun, 21 Oct 2012 15:14:08 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q9LCDu78032654; Sun, 21 Oct 2012 15:13:56 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q9LCDuPd032652; Sun, 21 Oct 2012 15:13:56 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 21 Oct 2012 15:13:56 +0300 From: Konstantin Belousov To: David Wolfskill Subject: Re: stable/9 @r241776 panic: REDZONE: Buffer underflow detected... Message-ID: <20121021121356.GJ35915@deviant.kiev.zoral.com.ua> References: <20121020141019.GW1817@albert.catwhisker.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="oKQo6H1tQBaoPMj5" Content-Disposition: inline In-Reply-To: <20121020141019.GW1817@albert.catwhisker.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Oct 2012 12:14:08 -0000 --oKQo6H1tQBaoPMj5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Oct 20, 2012 at 07:10:19AM -0700, David Wolfskill wrote: > This seems ... fairly weird to me. >=20 > Yesterday, I built & booted: >=20 > FreeBSD g1-227.catwhisker.org 9.1-PRERELEASE FreeBSD 9.1-PRERELEASE #274 = 241726M: Fri Oct 19 05:40:05 PDT 2012 root@g1-227.catwhisker.org:/usr/o= bj/usr/src/sys/CANARY i386 >=20 > and used the machine all day; nothing unusual (including various > reboots (e.g. when I disembarked the train for the final leg of my > commute home, so I powered the laptop off). >=20 > This morning, I built: >=20 > FreeBSD g1-227.catwhisker.org 9.1-PRERELEASE FreeBSD 9.1-PRERELEASE #275 = 241776M: Sat Oct 20 04:34:45 PDT 2012 root@g1-227.catwhisker.org:/usr/o= bj/usr/src/sys/CANARY i386 >=20 > and on first reboot, I got a panic. >=20 > After a bit of experimentation, it appears that I get a panic @r241776 > if I attempt a normal boot into multi-user mode, but if I first boot to > single-user mode, then exit single-user mode, it comes up without a > problem. >=20 > I don't have a serial console, so I started to write down some of the > panic information, but my patience ran a bit short. Here's whet I > recorded (warning: hand-transcripted -- twice!): >=20 > ... > Starting devd. > REDZONE: Buffer underflow detected. 1 byte corrupted before 0xced40080 (= 4294966796 bytes allocated). > Allocation backtrace: > #0 0xc0ceac8f at redzone_setup+0xcf > #1 0xc0a5d5c9 at malloc+0x1d9 > ...[about 20 more such lines I didn't record]... >=20 > > bt > Tracing pid 901 tid 100106 td 0xd2b99000 > kdb_enter(...) > panic(...) > free(...) > devread(ce8c2d00,f7274c0c,0,c0b1e4f0,d279e380,...) at devread+0x1a6 > giant_read(...) at giant_read+0x87 > devfs_read(...) at devfs_read+0xc6 > dofileread(...) at dofileread+0x99 > sys_read(...) at sys_read+0x98 > syscall(f7274d08) at syscall+0x387 >=20 > Within the bounds described above, this appears to be quite reproducible > -- on my laptop. My build machine (updated in parallel, at the same > GRNs) does not exhibit the panic. >=20 > I was unable to get a crash dump; I have >=20 > dumpdev=3D"AUTO" >=20 > in /etc/rc.conf, and the panic was occurring well after swap was > enabled. (Yes, I know I have swap over-allocated. I plan to do > something about it at some point.) >=20 > I've attached a copy of dmesg.boot. >=20 > Anyone else seeing this? Any ideas how to diagnose it? devread is the method of devctl(4) which passes devd notifications from the kernel to userland (to devd, specifically). There were no changes to devctl(4) for quite a time. The corruption is, most likely, in some unrelated piece of code. Could you try to bisect the stable to catch the offender ? The bisect is not guaranteed to work, obviously, since the random corruption effects are unpredictable. --oKQo6H1tQBaoPMj5 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlCD5wMACgkQC3+MBN1Mb4hYTQCfXTxexn6qLhv3U/5jttWNkMuh mO8AoKLn8GJLomWs4Zqg0YpmPYIpQSAt =cp/P -----END PGP SIGNATURE----- --oKQo6H1tQBaoPMj5--