Date: Fri, 31 Aug 2012 11:21:06 GMT From: "Eugene M. Zheganin" <emz@norma.perm.ru> To: freebsd-gnats-submit@FreeBSD.org Subject: kern/171202: [lor] multiple LOR's during the startup Message-ID: <201208311121.q7VBL6LJ027166@red.freebsd.org> Resent-Message-ID: <201208311130.q7VBU2cX099975@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 171202
>Category: kern
>Synopsis: [lor] multiple LOR's during the startup
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Fri Aug 31 11:30:02 UTC 2012
>Closed-Date:
>Last-Modified:
>Originator: Eugene M. Zheganin
>Release: 9.1-PRERELEASE
>Organization:
Vivat-Trade LLC
>Environment:
FreeBSD taiga 9.1-PRERELEASE FreeBSD 9.1-PRERELEASE #0: Fri Aug 17 13:55:19 YEKT 2012 emz@taiga:/usr/obj/usr/src/sys/TAIGADBG amd64
>Description:
During the investigation of the possible reasons of periodic hangups of the server, I build the WITNESS/INVARIANTS kernel. I receive multiple LORs during the startup of the server. bz@ kindly told me it's worth to report. So I'm doing this. I'm not aware if these LOR's actually are connected with my server's hangups.
Target machine is a IBM x3650 server. It runs IPv4, IPV6 via gif tunnel to the tunnelbroker, pf with a hundred of rules, pf nat, OSPFv6 with net/bird6, OSPFv4 with net/quagga, zfs.
# dmesg
Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.1-PRERELEASE #0: Fri Aug 17 13:55:19 YEKT 2012
emz@taiga:/usr/obj/usr/src/sys/TAIGADBG amd64
WARNING: WITNESS option enabled, expect reduced performance.
CPU: Intel(R) Xeon(R) CPU E5405 @ 2.00GHz (1995.04-MHz K8-class CPU)
Origin = "GenuineIntel" Id = 0x10676 Family = 6 Model = 17 Stepping = 6
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0xce33d<SSE3,DTES64,MON,DS_CPL,VMX,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1>
AMD Features=0x20100800<SYSCALL,NX,LM>
AMD Features2=0x1<LAHF>
TSC: P-state invariant, performance statistics
real memory = 4294967296 (4096 MB)
avail memory = 4089982976 (3900 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <IBM SERVALNT>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
cpu2 (AP): APIC ID: 2
cpu3 (AP): APIC ID: 3
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: <IBM SERVALNT> on motherboard
acpi0: Power Button (fixed)
ipmi0: KCS mode found at io 0xca8 on acpi
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x73 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 450
Event timer "HPET1" frequency 14318180 Hz quality 440
Event timer "HPET2" frequency 14318180 Hz quality 440
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x588-0x58b on acpi0
pcib0: <ACPI Host-PCI bridge> on acpi0
pcib0: Length mismatch for 3 range: c0000000 vs ffffffffc0000000
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
pci26: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> at device 0.0 on pci26
pci27: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 0.0 on pci27
pci28: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> irq 17 at device 1.0 on pci27
pci36: <ACPI PCI bus> on pcib4
pcib5: <PCI-PCI bridge> at device 0.3 on pci26
pci37: <PCI bus> on pcib5
pcib6: <ACPI PCI-PCI bridge> at device 3.0 on pci0
pci4: <ACPI PCI bus> on pcib6
aac0: <IBM ServeRAID-8k> port 0x5000-0x50ff mem 0xc9e00000-0xc9ffffff,0xc7fe0000-0xc7ffffff irq 17 at device 0.0 on pci4
aac0: Enabling 64-bit address support
aac0: Enable Raw I/O
aac0: Enable 64-bit array
aac0: New comm. interface enabled
aac0: ServeRAID 8k , aac driver 2.1.9-1
pcib7: <ACPI PCI-PCI bridge> at device 4.0 on pci0
pci16: <ACPI PCI bus> on pcib7
pcib8: <PCI-PCI bridge> at device 5.0 on pci0
pci69: <PCI bus> on pcib8
pcib9: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci7: <ACPI PCI bus> on pcib9
pcib10: <PCI-PCI bridge> at device 7.0 on pci0
pci68: <PCI bus> on pcib10
pci0: <base peripheral> at device 8.0 (no driver attached)
pcib11: <ACPI PCI-PCI bridge> at device 28.0 on pci0
pci2: <ACPI PCI bus> on pcib11
pcib12: <PCI-PCI bridge> at device 0.0 on pci2
pci3: <PCI bus> on pcib12
bce0: <Broadcom NetXtreme II BCM5708 1000Base-T (B2)> mem 0xce000000-0xcfffffff irq 16 at device 0.0 on pci3
miibus0: <MII bus> on bce0
brgphy0: <BCM5708C 1000BASE-T media interface> PHY 1 on miibus0
brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bce0: Ethernet address: 00:1a:64:c6:a8:7c
bce0: ASIC (0x57081020); Rev (B2); Bus (PCI-X, 64-bit, 133MHz); B/C (4.6.1); Bufs (RX:2;TX:2;PG:0); Flags (MFW); MFW (ipms 1.6.0)
Coal (RX:6,6,18,18; TX:20,20,80,80)
pcib13: <ACPI PCI-PCI bridge> at device 28.1 on pci0
pci5: <ACPI PCI bus> on pcib13
pcib14: <PCI-PCI bridge> at device 0.0 on pci5
pci6: <PCI bus> on pcib14
bce1: <Broadcom NetXtreme II BCM5708 1000Base-T (B2)> mem 0xca000000-0xcbffffff irq 17 at device 0.0 on pci6
miibus1: <MII bus> on bce1
brgphy1: <BCM5708C 1000BASE-T media interface> PHY 1 on miibus1
brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bce1: Ethernet address: 00:1a:64:c6:a8:7e
bce1: ASIC (0x57081020); Rev (B2); Bus (PCI-X, 64-bit, 133MHz); B/C (4.6.1); Bufs (RX:2;TX:2;PG:0); Flags (MFW); MFW (ipms 1.6.0)
Coal (RX:6,6,18,18; TX:20,20,80,80)
uhci0: <Intel 631XESB/632XESB/3100 USB controller USB-1> port 0x2200-0x221f irq 23 at device 29.0 on pci0
usbus0 on uhci0
uhci1: <Intel 631XESB/632XESB/3100 USB controller USB-2> port 0x2600-0x261f irq 22 at device 29.1 on pci0
usbus1 on uhci1
uhci2: <Intel 631XESB/632XESB/3100 USB controller USB-3> port 0x2a00-0x2a1f irq 23 at device 29.2 on pci0
usbus2 on uhci2
uhci3: <Intel 631XESB/632XESB/3100 USB controller USB-4> port 0x2e00-0x2e1f irq 22 at device 29.3 on pci0
usbus3 on uhci3
ehci0: <Intel 63XXESB USB 2.0 controller> mem 0xf9000000-0xf90003ff irq 23 at device 29.7 on pci0
usbus4: EHCI version 1.0
usbus4 on ehci0
pcib15: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci1: <ACPI PCI bus> on pcib15
vgapci0: <VGA-compatible display> port 0x4000-0x40ff mem 0xd0000000-0xd7ffffff,0xdfff0000-0xdfffffff irq 22 at device 6.0 on pci1
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel 63XXESB2 SATA300 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x480-0x48f at device 31.2 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ata1: <ATA channel> at channel 1 on atapci0
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: console (19200,n,8,1)
ipmi1: <IPMI System Interface> on isa0
device_attach: ipmi1 attach returned 16
ipmi1: <IPMI System Interface> on isa0
device_attach: ipmi1 attach returned 16
orm0: <ISA Option ROMs> at iomem 0xc0000-0xcafff,0xcb000-0xd07ff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x100>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
ppc0: cannot reserve I/O port range
p4tcc0: <CPU Frequency Thermal Control> on cpu0
p4tcc1: <CPU Frequency Thermal Control> on cpu1
p4tcc2: <CPU Frequency Thermal Control> on cpu2
p4tcc3: <CPU Frequency Thermal Control> on cpu3
ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is present;
to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf.
ZFS filesystem version 5
ZFS storage pool version 28
Timecounters tick every 1.000 msec
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 12Mbps Full Speed USB v1.0
ugen2.1: <Intel> at usbus2
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen3.1: <Intel> at usbus3
uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
usbus4: 480Mbps High Speed USB v2.0
ugen4.1: <Intel> at usbus4
uhub4: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus4
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
ipmi0: IPMI device rev. 0, firmware rev. 1.49, version 2.0
ipmi0: Number of channels 3
ipmi0: Attached watchdog
aacd0 on aac0
aacd0: 279800MB (573030400 sectors)
cd0 at ata1 bus 0 scbus1 target 0 lun 0
SMP: AP CPU #3 Launched!
SMP: AP CPU #1 Launched!
cd0: <HL-DT-ST RW/DVD GCC-T10N 1.00> Removable CD-ROM SCSI-0 device
SMP: AP CPU #2 Launched!
cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
cd0: cd present [1512048 x 2048 byte records]
Timecounter "TSC-low" frequency 15586274 Hz quality 1000
WARNING: WITNESS option enabled, expect reduced performance.
Root mount waiting for: usbus4
uhub4: 8 ports with 8 removable, self powered
Root mount waiting for: usbus4
Trying to mount root from zfs:zfsroot []...
ugen2.2: <IBM> at usbus2
ukbd0: <HID KB> on usbus2
kbd2 at ukbd0
uhid0: <HID MS> on usbus2
bce0: Gigabit link up!
WARNING: attempt to domain_add(netgraph) after domainfinalize()
lock order reversal:
1st 0xffffffff80b34620 pf task mtx (pf task mtx) @ /usr/src/sys/contrib/pf/net/pf_ioctl.c:1581
2nd 0xffffffff80d4ac00 ifnet_rw (ifnet_rw) @ /usr/src/sys/net/if.c:2022
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
kdb_backtrace() at kdb_backtrace+0x37
_witness_debugger() at _witness_debugger+0x2c
witness_checkorder() at witness_checkorder+0x853
_rw_rlock() at _rw_rlock+0x81
ifunit() at ifunit+0x22
pfioctl() at pfioctl+0x129b
devfs_ioctl_f() at devfs_ioctl_f+0x7a
kern_ioctl() at kern_ioctl+0xcd
sys_ioctl() at sys_ioctl+0xfd
amd64_syscall() at amd64_syscall+0x2fa
Xfast_syscall() at Xfast_syscall+0xf7
--- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x800d81cbc, rsp = 0x7fffffffa3d8, rbp = 0x7fffffffa530 ---
altq: emulate 256000000Hz cpu clock
lock order reversal:
1st 0xfffffe00079aabd8 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:849
2nd 0xfffffe0007465bd8 devfs (devfs) @ /usr/src/sys/kern/vfs_subr.c:2158
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
kdb_backtrace() at kdb_backtrace+0x37
_witness_debugger() at _witness_debugger+0x2c
witness_checkorder() at witness_checkorder+0x853
__lockmgr_args() at __lockmgr_args+0x1184
vop_stdlock() at vop_stdlock+0x39
VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b
_vn_lock() at _vn_lock+0x47
vget() at vget+0x7b
devfs_allocv() at devfs_allocv+0x13f
devfs_root() at devfs_root+0x4d
vfs_donmount() at vfs_donmount+0xb02
sys_nmount() at sys_nmount+0x66
amd64_syscall() at amd64_syscall+0x2fa
Xfast_syscall() at Xfast_syscall+0xf7
--- syscall (378, FreeBSD ELF64, sys_nmount), rip = 0x800a8d6fc, rsp = 0x7fffffffccc8, rbp = 0x801009048 ---
lock order reversal:
1st 0xffffffff80d50d48 PFil hook read/write mutex (PFil hook read/write mutex) @ /usr/src/sys/net/pfil.c:77
2nd 0xffffffff80d4e4f8 pcbinfohash (pcbinfohash) @ /usr/src/sys/netinet6/in6_pcb.c:1110
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
kdb_backtrace(
) at kdb_backtrace+0x37
_witness_debugger() at _witness_debugger+0x2c
witness_checkorder() at witness_checkorder+0x853
_rw_rlock() at _rw_rlock+0x81
in6_pcblookup_hash() at in6_pcblookup_hash+0x47
pf_socket_lookup() at pf_socket_lookup+0x191
pf_test_rule() at pf_test_rul
e+0x9fb
pf_test6() at pf_test6+0xf8a
pf_check6_in() at
pf_check6_in+0x35
pfil_run_hooks() at
pfil_run_hooks+0xd2
ip6_input() at ip6_input+0x7f9
swi_net() at swi_net+0x161
intr_eve
nt_execute_handlers() at intr_event_execute_handlers+0x6a
ithread_loop() at ithread_loop+0xb4
fork_exit() at fork_exit+0x135
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff800024ccf0, rbp = 0 ---
lock order reversal:
1st 0xffffffff80d50d48 PFil hook read/write mutex (PFil hook read/write mutex) @ /usr/src/sys/net/pfil.c:77
2nd 0xfffffe00074ed168 udpinp (udpinp) @ /usr/src/sys/netinet6/in6_pcb.c:1121
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
kdb_backtrace() at kdb_backtrace+0x37
_witness_debugger() at _witness_debugger+0x2c
witness
_
che
ckor
d
e
r
(
) at witness_checkorder+0x853
_rw_r
lock
() at _rw_rlo
c
k+0x81
in6_pcblookup_hash() at in6_pcblookup_hash+0xb7
pf_socket_l
o
okup() at pf_socket_lookup+0x24e
pf_test_rule() at pf_test_rule+0x9fb
pf_test6() at pf_test6+0xf8a
pf_check6_in() at pf_check6_in+0x35
pfil_run_hooks() at pfil_run_hooks+0xd2
ip6_input() at ip6_input+0x7f9
swi_net() at swi_net+0x161
intr_event_execute_handlers() at intr_event_execute_handlers+0x6a
ithread_loop() at ithread_loop+0xb4
fork_exit() at fork_exit+0x135
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff800024ccf0, rbp = 0 ---
lock order reversal:
1st 0xffffffff80d50d48 PFil hook read/write mutex (PFil hook read/write mutex) @ /usr/src/sys/net/pfil.c:77
2nd 0xfffffe0007b6af30 tcpinp (tcpinp) @ /usr/src/sys/netinet6/in6_pcb.c:1121
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
kdb_backtrace() at kdb_backtrace+0x37
_witness_debugger() at _witness_debugger+0x2c
witness_checkorder() at witness_checkorder+0x853
_rw_rlock() at _rw_rlock+0x81
in6_pcblookup_hash() at in6_pcblookup_hash+0xb7
pf_socket_lookup() at pf_socket_lookup+0x191
pf_test_rule() at pf_test_rule+0x9fb
pf_test6() at pf_test6+0xf8a
pf_check6_in() at pf_check6_in+0x35
pfil_run_hooks() at pfil_run_hooks+0xd2
ip6_input() at ip6_input+0x7f9
swi_net() at swi_net+0x161
intr_event_execute_handlers() at intr_event_execute_handlers+0x6a
ithread_loop() at ithread_loop+0xb4
fork_exit() at fork_exit+0x135
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff800024ccf0, rbp = 0 ---
lock order reversal:
1st 0xffffffff80d4cde0 in_ifaddr_lock (in_ifaddr_lock) @ /usr/src/sys/netinet/if_ether.c:569
2nd 0xfffffe0005acb720 carp_if (carp_if) @ /usr/src/sys/netinet/ip_carp.c:1195
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
kdb_backtrace() at kdb_backtrace+0x37
_witness_debugger() at _witness_debugger+0x2c
witness_checkorder() at witness_checkorder+0x853
_mtx_lock_flags() at _mtx_lock_flags+0x85
carp_iamatch() at carp_iamatch+0x39
arpintr() at arpintr+0x368
netisr_dispatch_src() at netisr_dispatch_src+0x152
ether_demux() at ether_demux+0x17d
ether_nh_input() at ether_nh_input+0x20e
netisr_dispatch_src() at netisr_dispatch_src+0x152
ether_demux() at ether_demux+0x86
ether_nh_input() at ether_nh_input+0x20e
netisr_dispatch_src() at netisr_dispatch_src+0x152
bce_intr() at bce_intr+0x46a
intr_event_execute_handlers() at intr_event_execute_handlers+0x6a
ithread_loop() at ithread_loop+0xb4
fork_exit() at fork_exit+0x135
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff811a901cf0, rbp = 0 ---
vlan1: promiscuous mode enabled
carp0: INIT -> BACKUP
carp0: link state changed to DOWN
carp0: link state changed to DOWN
carp0: INIT -> BACKUP
carp0: link state changed to DOWN
carp0: link state changed to DOWN
carp0: INIT -> BACKUP
carp0: link state changed to DOWN
carp0: BACKUP -> MASTER (preempting a slower master)
carp0: link state changed to UP
vlan15: promiscuous mode enabled
carp2: INIT -> BACKUP
carp2: link state changed to DOWN
carp2: INIT -> BACKUP
carp2: link state changed to DOWN
carp2: link state changed to DOWN
carp2: BACKUP -> MASTER (preempting a slower master)
carp2: link state changed to UP
lock order reversal:
1st 0xfffffe00a835d458 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:849
2nd 0xfffffe00096febd8 newnfs (newnfs) @ /usr/src/sys/kern/vfs_subr.c:2158
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
kdb_backtrace() at kdb_backtrace+0x37
_witness_debugger() at _witness_debugger+0x2c
witness_checkorder() at witness_checkorder+0x853
__lockmgr_args() at __lockmgr_args+0x1184
nfs_lock1() at nfs_lock1+0x4a
VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b
_vn_lock() at _vn_lock+0x47
vget() at vget+0x7b
vfs_hash_get() at vfs_hash_get+0xd5
ncl_nget() at ncl_nget+0xd5
nfs_root() at nfs_root+0x3d
vfs_donmount() at vfs_donmount+0xb02
sys_nmount() at sys_nmount+0x66
amd64_syscall() at amd64_syscall+0x2fa
Xfast_syscall() at Xfast_syscall+0xf7
--- syscall (378, FreeBSD ELF64, sys_nmount), rip = 0x80087c6fc, rsp = 0x7fffffffd1b8, rbp = 0x7fffffffd26c ---
arp: 192.168.6.57 moved from 14:8f:c6:bc:e0:b7 to 00:08:ca:65:7a:c5 on vlan10
arp: 192.168.6.40 moved from 40:30:04:4b:ed:ed to 14:8f:c6:bc:e0:b7 on vlan10
arp: 192.168.6.44 moved from 40:30:04:4b:ed:ed to b8:ff:61:31:1e:d4 on vlan10
cannot forward src fe80:b::9cb9:2755:f0a0:488b, dst fec0:0:0:ffff::3, nxt 17, rcvif vlan1, outif gif0
cannot forward src fe80:12::617a:1076:42b2:68a2, dst fec0:0:0:ffff::3, nxt 17, rcvif vlan15, outif gif0
cannot forward src fe80:12::617a:1076:42b2:68a2, dst fec0:0:0:ffff::2, nxt 17, rcvif vlan15, outif gif0
cannot forward src fe80:12::617a:1076:42b2:68a2, dst fec0:0:0:ffff::1, nxt 17, rcvif vlan15, outif gif0
cannot forward src fe80:12::95b6:d17:e8fc:4e39, dst fec0:0:0:ffff::3, nxt 17, rcvif vlan15, outif gif0
cannot forward src fe80:b::f10a:ed10:819f:59d6, dst fec0:0:0:ffff::3, nxt 17, rcvif vlan1, outif gif0
cannot forward src fe80:12::1455:1c89:e03d:ac90, dst fec0:0:0:ffff::3, nxt 17, rcvif vlan15, outif gif0
cannot forward src fe80:12::fd85:e73d:10b9:b225, dst fec0:0:0:ffff::3, nxt 17, rcvif vlan15, outif gif0
cannot forward src fe80:12::4cae:74c6:6a20:eb7, dst fec0:0:0:ffff::3, nxt 17, rcvif vlan15, outif gif0
cannot forward src fe80:12::f552:7bc3:5cd2:e591, dst fec0:0:0:ffff::3, nxt 17, rcvif vlan15, outif gif0
arp: 192.168.8.235 moved from 5c:b5:24:13:ee:ba to 94:63:d1:c6:02:9d on vlan11
cannot forward src fe80:12::b576:9a8a:fbe2:17e8, dst fec0:0:0:ffff::3, nxt 17, rcvif vlan15, outif gif0
cannot forward src fe80:12::4cae:74c6:6a20:eb7, dst fec0:0:0:ffff::3, nxt 17, rcvif vlan15, outif gif0
arp: 192.168.8.212 moved from 00:23:6c:00:e8:b9 to 38:16:d1:c6:e6:eb on vlan11
lock order reversal:
1st 0xffffffff80d50d48 PFil hook read/write mutex (PFil hook read/write mutex) @ /usr/src/sys/net/pfil.c:77
2nd 0xfffffe0007467e08 gif softc (gif softc) @ /usr/src/sys/net/if_gif.c:430
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
kdb_backtrace() at kdb_backtrace+0x37
_witness_debugger() at _witness_debugger+0x2c
witness_checkorder() at witness_checkorder+0x853
_mtx_lock_flags() at _mtx_lock_flags+0x85
gif_output() at gif_output+0xcb
nd6_output_lle() at nd6_output_lle+0x457
nd6_output() at nd6_output+0x18
ip6_output() at ip6_output+0x122f
pf_send_tcp() at pf_send_tcp+0x3ff
pf_tcp_track_full() at pf_tcp_track_full+0xb15
pf_test_state_tcp() at pf_test_state_tcp+0x167
pf_test6() at pf_test6+0xda5
pf_check6_in() at pf_check6_in+0x35
pfil_run_hooks() at pfil_run_hooks+0xd2
ip6_input() at ip6_input+0x7f9
netisr_dispatch_src() at netisr_dispatch_src+0x152
gif_input() at gif_input+0xf5
in_gif_input() at in_gif_input+0x13e
encap4_input() at encap4_input+0x139
ip_input() at ip_input+0xbd
netisr_dispatch_src() at netisr_dispatch_src+0x152
ether_demux() at ether_demux+0x17d
ng_ether_rcv_upper() at ng_ether_rcv_upper+0xa4
ng_apply_item() at ng_apply_item+0x281
ng_snd_item() at ng_snd_item+0x2af
ng_apply_item() at ng_apply_item+0x281
ng_snd_item() at ng_snd_item+0x2af
ng_apply_item() at ng_apply_item+0x281
ng_snd_item() at ng_snd_item+0x2af
ng_ether_input() at ng_ether_input+0x7a
ether_nh_input() at ether_nh_input+0x17c
netisr_dispatch_src() at netisr_dispatch_src+0x152
ether_demux() at ether_demux+0x86
ether_nh_input() at ether_nh_input+0x20e
netisr_dispatch_src() at netisr_dispatch_src+0x152
bce_intr() at bce_intr+0x46a
intr_event_execute_handlers() at intr_event_execute_handlers+0x6a
ithread_loop() at ithread_loop+0xb4
fork_exit() at fork_exit+0x135
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff811a901cf0, rbp = 0 ---
>How-To-Repeat:
>Fix:
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201208311121.q7VBL6LJ027166>