Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 22 Oct 2012 00:09:08 +0200
From:      Mateusz Guzik <mjguzik@gmail.com>
To:        David Wolfskill <david@catwhisker.org>
Cc:        stable@freebsd.org
Subject:   Re: stable/9 @r241776 panic: REDZONE: Buffer underflow detected...
Message-ID:  <20121021220908.GA20958@dft-labs.eu>
In-Reply-To: <20121020141019.GW1817@albert.catwhisker.org>
References:  <20121020141019.GW1817@albert.catwhisker.org>

next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Oct 20, 2012 at 07:10:19AM -0700, David Wolfskill wrote:
> This seems ... fairly weird to me.
> 
> Yesterday, I built & booted:
> 
> FreeBSD g1-227.catwhisker.org 9.1-PRERELEASE FreeBSD 9.1-PRERELEASE #274 241726M: Fri Oct 19 05:40:05 PDT 2012     root@g1-227.catwhisker.org:/usr/obj/usr/src/sys/CANARY  i386
> 
> and used the machine all day; nothing unusual (including various
> reboots (e.g. when I disembarked the train for the final leg of my
> commute home, so I powered the laptop off).
> 
> This morning, I built:
> 
> FreeBSD g1-227.catwhisker.org 9.1-PRERELEASE FreeBSD 9.1-PRERELEASE #275 241776M: Sat Oct 20 04:34:45 PDT 2012     root@g1-227.catwhisker.org:/usr/obj/usr/src/sys/CANARY  i386
> 
> and on first reboot, I got a panic.
> 
[..]
> 
> ...
> Starting devd.
> REDZONE: Buffer underflow detected.  1 byte corrupted before 0xced40080 (4294966796 bytes allocated).
> Allocation backtrace:
> #0 0xc0ceac8f at redzone_setup+0xcf
> #1 0xc0a5d5c9 at malloc+0x1d9
> ...[about 20 more such lines I didn't record]...
> 
> > bt
> Tracing pid 901 tid 100106 td 0xd2b99000
> kdb_enter(...)
> panic(...)
> free(...)
> devread(ce8c2d00,f7274c0c,0,c0b1e4f0,d279e380,...) at devread+0x1a6
> giant_read(...) at giant_read+0x87
> devfs_read(...) at devfs_read+0xc6
> dofileread(...) at dofileread+0x99
> sys_read(...) at sys_read+0x98
> syscall(f7274d08) at syscall+0x387
> 

This looks a lot like issue you reported a couple of months earlier,
even affected buffer address matches.

At least part of REDZONE metadata placed directly before the buffer is
corrupted. So the idea is to set a watchpoint at a place that is known
to contain wrong data (in this case allocation size) and wait for some
code to try to modify it.

I hacked up the following (really ugly, but should do the job):
http://people.freebsd.org/~mjg/patches/watchpoint-hack.diff

Note: this assumes that address of affected buffer is always the same.

Assuming I didn't mess anything up, instructions are simple:
Just try to reproduce the issue, at some point you should be dropped to
the debugger. If that happens when dumpdevice is configured, please get a
core. Otherwise just a backtrace ("bt" command).

Note 2: this code does no clear the watchpoint, so if it fails to catch
the offending case, it may catch completely legitimate code later.



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20121021220908.GA20958>