Date: Tue, 14 Oct 2008 05:24:38 GMT From: Pekka Savola <pekkas@netcore.fi> To: freebsd-gnats-submit@FreeBSD.org Subject: ports/128082: megarc binary causes memory corruption Message-ID: <200810140524.m9E5OcZl056886@www.freebsd.org> Resent-Message-ID: <200810140530.m9E5U1tE028278@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 128082
>Category: ports
>Synopsis: megarc binary causes memory corruption
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: freebsd-ports-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Tue Oct 14 05:30:01 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator: Pekka Savola
>Release: 7.1-PRERELEASE
>Organization:
CSC/FUNET
>Environment:
FreeBSD sixpack.funet.fi 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #12: Fri Oct 10 13:12:58 EEST 2008 root@sixpack.funet.fi:/usr/obj/usr/src/sys/SIXPACK i386
s
>Description:
The ports/sysutils/megarc binary appears to cause memory corruption, leading to process core dumps and kernel crashes (with corrupted stack). This can be seen during one day of operation. When I didn't run megarc binary, the system was stable for a month.
How this gets triggered: I run the following commands every 10 minutes (a nagios check):
/usr/local/sbin/megarc -AllAdpInfo -nolog
/usr/local/sbin/megarc -ldInfo -a0 -Lall -nolog
I have 'Dell PowerEdge Expandable RAID Controller 4e/Si' with the latest firmware (A19).
I believe this port needs to be marked broken and/or removed.
Examples of crashes are below:
Unread portion of the kernel message buffer:
upt enabled, resume, IOPL = 0
current
<110>ipfw: 20 Deny TCP [2001:0:d5c7:a2ca:2cc0:337c:aa63:xxx]:64831 [2001:0:4137:9e50:3cb9:20d7:bd7a:yyy]:59757 in via stf0
process = 1589 (megarc)
trap number = 12
panic: page fault
cpuid = 0
Uptime: 40m45s
Physical memory: 2039 MB
Dumping 173 MB: 158 142 126 110 94 78 62 46 30 14
#0 doadump () at pcpu.h:196
196 __asm __volatile("movl %%fs:0,%0" : "=r" (td));
(kgdb) bt
#0 doadump () at pcpu.h:196
#1 0xc058e577 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
#2 0xc058e849 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:574
#3 0xc073d74c in trap_fatal (frame=0xe77b8b84, eva=3217031168) at /usr/src/sys/i386/i386/trap.c:939
#4 0xc073d9d0 in trap_pfault (frame=0xe77b8b84, usermode=0, eva=3217031168) at /usr/src/sys/i386/i386/trap.c:852
#5 0xc073e34c in trap (frame=0xe77b8b84) at /usr/src/sys/i386/i386/trap.c:530
#6 0xc0723eab in calltrap () at /usr/src/sys/i386/i386/exception.s:159
#7 0xc073948f in pmap_remove_pages (pmap=0xc4d10b6c) at /usr/src/sys/i386/i386/pmap.c:3077
#8 0xc06e206c in vmspace_exit (td=0xc53d08c0) at /usr/src/sys/vm/vm_map.c:404
#9 0xc0568db3 in exit1 (td=0xc53d08c0, rv=0) at /usr/src/sys/kern/kern_exit.c:305
#10 0xc056a10d in sys_exit (td=Could not find the frame base for "sys_exit".
) at /usr/src/sys/kern/kern_exit.c:109
#11 0xc073dd09 in syscall (frame=0xe77b8d38) at /usr/src/sys/i386/i386/trap.c:1090
#12 0xc0723f10 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:255
#13 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)
Unread portion of the kernel message buffer:
uid = 1; apic id = 06
fault virt
<110>ipfw: 20 Deny TCP [2001:0:d5c7:a2ca:23:1c6:a78f:xxx]:55195 [2001:0:4137:9e50:3c78:f15:42e1:xxx]:61615 in via stf0
ual address = 0x4
fault code = supervisor read, page not present
instruction pointer
<110>ipfw: 20 Deny TCP [2001:0:d5c7:a2ca:1cff:20df:a78c:xxx]:61853 [2001:0:d5c7:a2ca:0:f133:af21:yyy]:6113 in via stf0
= 0x20:0xc0736c5a
stack pointer = 0x28:0xe7763a84
frame pointer = 0x28:0xe7763a98
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
<110>ipfw: 20 Deny TCP [2001:0:d5c7:a2ca:1431:1feb:a46e:xxx]:22052 [2001:0:4137:9e50:2c9c:1098:ba7a:xxx]:53842 in via stf0
processor eflags =
<110>ipfw: 20 Deny TCP [2001:0:d5c7:a2ca:2082:17e1:ab04:xxx]:60104 [2001:0:d5c7:a2ca:24c2:2e61:a51a:xxx]:58599 in via stf0
interrupt enabled, resume, IOPL = 0
current process =
<110>ipfw: 20 Deny TCP [2001:0:d5c7:a2ca:20ff:31a7:3c6b:xxx]:50204 [2001:0:d5c7:a2ca:10a0:db1:a725:xxx]:61429 in via stf0
81306 (sudo)
trap number = 12
panic: pag
<110>ipfew: 2f0 aDeuny lTCtP [2
time: 3d12h3m53s
Physical memory: 2039 MB
Dumping 184 MB:
<110>ipfw: 20 Deny TCP [2001:0:d5c7:a2ca:308a:1153:a7aa:xxx]:59817 [2001:0:d5c7:a2ca:243f:14dc:ab05:xxx]:62555 in via stf0
169 153 137 121 105 89
<110>ipfw: 20 Deny TCP [2001:0:d5c7:a2ca:4db:2aae:af23:xxx]:55758 [2001:0:d5c7:a2ca:1ceb:b39:a5e3:xxx]:23505 in via stf0
73 57 41 25 9
#0 doadump () at pcpu.h:196
196 __asm __volatile("movl %%fs:0,%0" : "=r" (td));
(kgdb) bt
#0 doadump () at pcpu.h:196
#1 0xc058e577 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
#2 0xc058e849 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:574
#3 0xc073d74c in trap_fatal (frame=0xe7763a44, eva=4) at /usr/src/sys/i386/i386/trap.c:939
#4 0xc073d9d0 in trap_pfault (frame=0xe7763a44, usermode=0, eva=4) at /usr/src/sys/i386/i386/trap.c:852
#5 0xc073e34c in trap (frame=0xe7763a44) at /usr/src/sys/i386/i386/trap.c:530
#6 0xc0723eab in calltrap () at /usr/src/sys/i386/i386/exception.s:159
#7 0xc0736c5a in pmap_remove_entry (pmap=0xc55a75fc, m=0xc10e76b0, va=671641600) at /usr/src/sys/i386/i386/pmap.c:1918
#8 0xc0739eac in pmap_enter (pmap=0xc55a75fc, va=671641600, m=0xc13e3660, prot=3 '\003', wired=0) at /usr/src/sys/i386/i386/pmap.c:2424
#9 0xc06da5dc in vm_fault (map=0xc55a7570, vaddr=671641600, fault_type=2 '\002', fault_flags=8) at /usr/src/sys/vm/vm_fault.c:882
#10 0xc073d8bb in trap_pfault (frame=0xe7763d38, usermode=1, eva=671642892) at /usr/src/sys/i386/i386/trap.c:829
#11 0xc073e1d7 in trap (frame=0xe7763d38) at /usr/src/sys/i386/i386/trap.c:397
#12 0xc0723eab in calltrap () at /usr/src/sys/i386/i386/exception.s:159
#13 0x280638c6 in ?? ()
Previous frame inner to this frame (corrupt stack?)
# gdb alpine alpine.core
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...(no debugging symbols found)...
Core was generated by `alpine'.
Program terminated with signal 6, Aborted.
Reading symbols from /lib/libcrypt.so.4...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypt.so.4
Reading symbols from /usr/lib/libpam.so.4...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libpam.so.4
Reading symbols from /lib/libcrypto.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypto.so.5
Reading symbols from /lib/libncurses.so.7...(no debugging symbols found)...done.
Loaded symbols for /lib/libncurses.so.7
Reading symbols from /usr/lib/libssl.so.5...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libssl.so.5
Reading symbols from /lib/libthr.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib/libthr.so.3
Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.7
Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done.
Loaded symbols for /libexec/ld-elf.so.1
#0 0x28689c0b in thr_kill () from /lib/libc.so.7
[New Thread 0x8602400 (LWP 100085)]
[New Thread 0x8601100 (LWP 100122)]
(gdb) bt
#0 0x28689c0b in thr_kill () from /lib/libc.so.7
#1 0x2863d5b6 in pthread_kill () from /lib/libthr.so.3
#2 0x2863b163 in raise () from /lib/libthr.so.3
#3 0x2871beaa in abort () from /lib/libc.so.7
#4 0x081f255b in ?? ()
#5 0x00000101 in ?? ()
#6 0x00000101 in ?? ()
#7 0xbfbfc168 in ?? ()
#8 0x0806717f in ?? ()
#9 0x28744fc0 in __tsd_lock () from /lib/libc.so.7
#10 0x081fab00 in ?? ()
#11 0xbfbfc064 in ?? ()
#12 0xbfbfc190 in ?? ()
#13 0xbfbfc104 in ?? ()
#14 0x626f7250 in ?? ()
#15 0x206d656c in ?? ()
#16 0x65746564 in ?? ()
#17 0x64657463 in ?? ()
#18 0x5222203a in ?? ()
#19 0x69656365 in ?? ()
#20 0x20646576 in ?? ()
#21 0x726f6261 in ?? ()
#22 0x69732074 in ?? ()
#23 0x6c616e67 in ?? ()
#24 0x67697328 in ?? ()
#25 0x2931313d in ?? ()
#26 0x410a2e22 in ?? ()
#27 0x6e69706c in ?? ()
#28 0x78452065 in ?? ()
#29 0x6e697469 in ?? ()
#30 0x00002e67 in ?? ()
#31 0x00000000 in ?? ()
#32 0x00000000 in ?? ()
#33 0x00000000 in ?? ()
#34 0x00000000 in ?? ()
#35 0x00000000 in ?? ()
#36 0x00000000 in ?? ()
#37 0x00000000 in ?? ()
#38 0x00000000 in ?? ()
#39 0x00000000 in ?? ()
#40 0x00000000 in ?? ()
#41 0x00000000 in ?? ()
#42 0x00000000 in ?? ()
#43 0x00000000 in ?? ()
#44 0x00000000 in ?? ()
#45 0x00000000 in ?? ()
#46 0x00000000 in ?? ()
#47 0x00000000 in ?? ()
#48 0x00000000 in ?? ()
#49 0x00000000 in ?? ()
#50 0x00000000 in ?? ()
#51 0x00000000 in ?? ()
#52 0x00000000 in ?? ()
#53 0x00000000 in ?? ()
#54 0xbfbfc1ad in ?? ()
#55 0xbfbfc114 in ?? ()
#56 0x00000046 in ?? ()
#57 0xffff0208 in ?? ()
#58 0xbfbfc190 in ?? ()
#59 0x00000063 in ?? ()
#60 0x00000000 in ?? ()
#61 0x00000000 in ?? ()
#62 0x00000000 in ?? ()
#63 0x00000000 in ?? ()
#64 0x00000001 in ?? ()
#65 0x00000002 in ?? ()
#66 0x00000000 in ?? ()
#67 0x00000000 in ?? ()
#68 0xbfbfc070 in ?? ()
#69 0x00000000 in ?? ()
#70 0x00000001 in ?? ()
#71 0xbfbfc168 in ?? ()
#72 0x080fe445 in ?? ()
#73 0x00000002 in ?? ()
#74 0x00000001 in ?? ()
#75 0x00000001 in ?? ()
#76 0xbfbfc17c in ?? ()
#77 0x00bfc190 in ?? ()
#78 0xbfbfc190 in ?? ()
#79 0xbfbfc1f8 in ?? ()
#80 0x080fe950 in ?? ()
#81 0xbfbfc190 in ?? ()
#82 0x00000064 in ?? ()
#83 0x082869df in ?? ()
#84 0x0000000b in ?? ()
#85 0xbfbfc824 in ?? ()
#86 0xbfbfc9b8 in ?? ()
#87 0xbfbfc1a8 in ?? ()
#88 0x2871141f in open () from /lib/libc.so.7
#89 0x65636552 in ?? ()
#90 0x64657669 in ?? ()
#91 0x6f626120 in ?? ()
#92 0x73207472 in ?? ()
#93 0x616e6769 in ?? ()
#94 0x6973286c in ?? ()
#95 0x31313d67 in ?? ()
#96 0x28710029 in arc4random_addrandom () from /lib/libc.so.7
Previous frame inner to this frame (corrupt stack?)
>How-To-Repeat:
Run megarc binary a lot with the two commands and start observing kernel and process crashes.
>Fix:
LSI Logic only distributes a binary, so this problem doesn't appear to be fixable. I suggest removing this port.
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200810140524.m9E5OcZl056886>