Date: Sun, 21 Feb 2010 19:01:14 GMT From: Dmitriy Demidov <dima_bsd@inbox.lv> To: freebsd-gnats-submit@FreeBSD.org Subject: kern/144187: deadlock using multiple ipfw nat and multiple limit statements Message-ID: <201002211901.o1LJ1EZW047220@www.freebsd.org> Resent-Message-ID: <201002211910.o1LJA1aC075142@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 144187
>Category: kern
>Synopsis: deadlock using multiple ipfw nat and multiple limit statements
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Sun Feb 21 19:10:01 UTC 2010
>Closed-Date:
>Last-Modified:
>Originator: Dmitriy Demidov
>Release: 7.3-PRERELEASE
>Organization:
>Environment:
FreeBSD evo.local.home 7.3-PRERELEASE FreeBSD 7.3-PRERELEASE #0: Sat Feb 20 22:57:09 EET 2010 root@evo.local.home:/usr/obj/usr/src/sys/STABLE i386
>Description:
I met one problem with multiple ipfw nat instances and multiple limit rules. If I configure my ipfw as showed below and make a try to connect to 192.168.1.2 from 192.168.1.1 using ssh, then system just die without any error messages or panics. It do not reboots but just stay in deadlock forever - I am unable to use console or ssh/ping it anymore, and only way to get it back is a reset or power buttons.
Problem disappears only then I remove "limit src-addr 80" from first, or from second nat instances.
System is a nat router what have 3 network interfaces - fxp0 is connected to inner LAN (172.16.1.0/24) and each of rl adapters is connected to different upstreem ISP. There is no any setfib at this moment. Default gateway (192.168.1.1) is on rl0 side.
Customized GENERIC kernel:
===
#cpu I486_CPU
#cpu I586_CPU
cpu I686_CPU
ident STABLE
options IPFIREWALL
options IPFIREWALL_NAT
options LIBALIAS
options ROUTETABLES=2
options DUMMYNET
options HZ=1000
makeoptions DEBUG=-g
options KDB
options DDB
options GDB
options INVARIANTS
options INVARIANT_SUPPORT
options WITNESS
options DEBUG_LOCKS
options DEBUG_VFS_LOCKS
options DIAGNOSTIC
##
===
sysctl:
===
sysctl -a | grep one_pass
net.inet.ip.fw.one_pass: 0
===
ipfw configuration:
===
add 1040 allow ip from any to any via fxp0
nat 1 config if rl0 reset same_ports deny_in redirect_port tcp 192.168.1.2:22 22
nat 2 config if rl1 reset same_ports deny_in
add 10130 nat 1 tcp from any to any out xmit rl0 limit src-addr 80
add 10131 allow tcp from any to any out xmit rl0
add 10140 nat 1 ip from any to any out xmit rl0
add 10141 allow ip from any to any out xmit rl0
add 20130 nat 2 tcp from any to any out xmit rl1 limit src-addr 80
add 20131 allow tcp from any to any out xmit rl1
add 20140 nat 2 ip from any to any out xmit rl1
add 20141 allow ip from any to any out xmit rl1
add 20150 nat 1 ip from any to any in recv rl0
add 20151 nat 2 ip from any to any in recv rl1
add 30160 allow ip from any to any
add 65534 deny ip from any to any
===
ipfw show:
===
00100 0 0 allow ip from any to any via lo0
00200 0 0 deny ip from any to 127.0.0.0/8
00300 0 0 deny ip from 127.0.0.0/8 to any
01040 0 0 allow ip from any to any via fxp0
10130 1 40 nat 1 tcp from any to any out xmit rl0 limit src-addr 80
10131 1 40 allow tcp from any to any out xmit rl0
10140 0 0 nat 1 ip from any to any out xmit rl0
10141 0 0 allow ip from any to any out xmit rl0
20130 0 0 nat 2 tcp from any to any out xmit rl1 limit src-addr 80
20131 0 0 allow tcp from any to any out xmit rl1
20140 0 0 nat 2 ip from any to any out xmit rl1
20141 0 0 allow ip from any to any out xmit rl1
20150 1 52 nat 1 ip from any to any in recv rl0
20151 0 0 nat 2 ip from any to any in recv rl1
30160 1 52 allow ip from any to any
65534 0 0 deny ip from any to any
65535 0 0 deny ip from any to any
===
ifconfig:
===
evo# ifconfig
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 00:30:4f:11:11:11
inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=2009<RXCSUM,VLAN_MTU,WOL_MAGIC>
ether 00:0b:cd:22:22:22
inet 172.16.1.1 netmask 0xffffff00 broadcast 172.16.1.255
media: Ethernet autoselect (none)
status: no carrier
rl1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 00:0c:46:33:33:33
inet 192.168.2.2 netmask 0xffffff00 broadcast 192.168.2.255
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
plip0: flags=108810<POINTOPOINT,SIMPLEX,MULTICAST,NEEDSGIANT> metric 0 mtu 1500
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
inet6 ::1 prefixlen 128
inet 127.0.0.1 netmask 0xff000000
===
===
evo# setfib 0 netstat -rn
Routing tables
Internet:
Destination Gateway Flags Refs Use Netif Expire
default 192.168.1.1 UGS 0 0 rl0
127.0.0.1 127.0.0.1 UH 0 0 lo0
172.16.1.0/24 link#2 UC 0 0 fxp0
192.168.1.0/24 link#1 UC 0 0 rl0
192.168.1.1 00:90:27:2c:12:12 UHLW 1 20 rl0 1193
192.168.2.0/24 link#3 UC 0 0 rl1
Internet6:
Destination Gateway Flags Netif Expire
::1 ::1 UHL lo0
fe80::%lo0/64 fe80::1%lo0 U lo0
fe80::1%lo0 link#5 UHL lo0
ff01:5::/32 fe80::1%lo0 UC lo0
ff02::%lo0/32 fe80::1%lo0 UC lo0
evo# setfib 1 netstat -rn
Routing tables
Internet:
Destination Gateway Flags Refs Use Netif Expire
127.0.0.1 127.0.0.1 UH 0 0 lo0
172.16.1.0/24 link#2 UC 0 0 fxp0
192.168.1.0/24 link#1 UC 0 0 rl0
192.168.1.1 00:90:27:2c:12:12 UHLW 1 0 rl0 1190
192.168.2.0/24 link#3 UC 0 0 rl1
Internet6:
Destination Gateway Flags Netif Expire
::1 ::1 UHL lo0
fe80::%lo0/64 fe80::1%lo0 U lo0
fe80::1%lo0 link#5 UHL lo0
ff01:5::/32 fe80::1%lo0 UC lo0
ff02::%lo0/32 fe80::1%lo0 UC lo0
===
>How-To-Repeat:
Configure kernel options and ipfw rules as it mentioned in Description and try to connect to system via ssh (or just telnet to port 22).
>Fix:
Do not use multiple limit statements.
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201002211901.o1LJ1EZW047220>