From owner-freebsd-security Sun Sep 15 15:42: 7 2002 Delivered-To: freebsd-security@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 43EE837B400 for ; Sun, 15 Sep 2002 15:42:03 -0700 (PDT) Received: from alcanet.com.au (mail3.alcanet.com.au [208.178.117.11]) by mx1.FreeBSD.org (Postfix) with ESMTP id A98C743E75 for ; Sun, 15 Sep 2002 15:42:01 -0700 (PDT) (envelope-from peter.jeremy@alcatel.com.au) Received: from sydsmtp01.alcatel.com.au (IDENT:root@localhost.localdomain [127.0.0.1]) by alcanet.com.au (8.12.4/8.12.4/Alcanet1.3) with ESMTP id g8FMfuRS019840; Mon, 16 Sep 2002 08:41:56 +1000 Received: from gsmx07.alcatel.com.au ([139.188.20.247]) by sydsmtp01.alcatel.com.au (Lotus Domino Release 5.0.10) with ESMTP id 2002091608415501:41570 ; Mon, 16 Sep 2002 08:41:55 +1000 Received: from gsmx07.alcatel.com.au (localhost [127.0.0.1]) by gsmx07.alcatel.com.au (8.12.5/8.12.5) with ESMTP id g8FMfs2t012597; Mon, 16 Sep 2002 08:41:54 +1000 (EST) (envelope-from peter.jeremy@alcatel.com.au) Received: (from jeremyp@localhost) by gsmx07.alcatel.com.au (8.12.5/8.12.5/Submit) id g8FMfs5E012596; Mon, 16 Sep 2002 08:41:54 +1000 (EST) (envelope-from peter.jeremy@alcatel.com.au) Date: Mon, 16 Sep 2002 08:41:54 +1000 From: Peter Jeremy To: freebsd-security@freebsd.org Subject: Unexpected keep state behaviour in ipfw Message-ID: <20020915224154.GD495@gsmx07.alcatel.com.au> Mail-Followup-To: freebsd-security@freebsd.org Mime-Version: 1.0 User-Agent: Mutt/1.4i X-MIMETrack: Itemize by SMTP Server on SYDSMTP01/AlcatelAustralia(Release 5.0.10 |March 22, 2002) at 16/09/2002 08:41:55 AM, Serialize by Router on SYDSMTP01/AlcatelAustralia(Release 5.0.10 |March 22, 2002) at 16/09/2002 08:41:56 AM, Serialize complete at 16/09/2002 08:41:56 AM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-freebsd-security@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org I've been putting together a firewall and bumped into an oddity in the ipfw keep-state behaviour. In particular, the TCP state information for connections through the firewall appears to be being lost so that the connection disappears. I've used keep-state in the past and I'm sure it has worked as expected. Also, my ssh connection from an internal host to the firewall host remains up even after extended periods of inactivity (like overnight). Either ipfw doesn't like a back-to-back keep-state configuration or I've done something silly but I can't see what. I was initially using ipfw1 in -STABLE from about a week ago. When I switched to ipfw2 in -STABLE as of last Friday, the problem remained. (In both cases, I did a complete build and installworld). Configuration: +-------+ +-------+ +-------+ | | internal | | Internet | | | hosta |------------| fwall |------------| hostb | | | ed0| |ed1 | | +-------+ +-------+ +-------+ fwall is using ipnat for address translation and was initially using both IPFilter and ipfw rules, but the problem remained after I changed IPFilter to do nothing (pass {in,out} all). [I wanted to use ipfw because I am more familiar with it and I wanted to use ipnat because the firewall box is relatively underpowered and I didn't want all the traffic to have to go through the kernel/userland interface to natd(8)]. My ipfw rules were initially: 00100 allow ip from any to any via lo0 00200 deny ip from any to 127.0.0.0/8 00300 deny ip from 127.0.0.0/8 to any 00600 check-state 00650 deny log tcp from any to any established 01200 allow ip from internal-net/24 to any in recv ed0 keep-state 01300 skipto 30000 ip from any to internal-net/24 out xmit ed0 11000 allow tcp from me to any out xmit ed1 keep-state 11005 allow udp from me to any dst-port 53 out xmit ed1 keep-state 11005 allow udp from me to any dst-port 123 out xmit ed1 11005 allow udp from me 68 to any dst-port 67 out xmit ed1 keep-state 11005 allow icmp from me to any out xmit ed1 icmptypes 0,3,4,8,11,12 11005 allow icmp from any to me in recv ed1 icmptypes 0,3,4,8,11,12 11010 allow udp from any 123 to me in recv ed1 13000 deny log ip from any to any 30020 allow udp from me to any dst-port 123 30030 allow icmp from me to any icmptypes 0,3,4,8,11,12 31000 deny log ip from any to any 65535 deny ip from any to any I initially thought there might be a problem with duplicate FIN packets and added the following rule but it made no difference: 00610 allow tcp from any to any tcpflags fin The following is an example of the problem showing up on an FTP control connection. 'fwall-ed1' is the internet-visible address. /var/log/security: 07:08:02 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 07:08:03 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 07:08:05 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 07:08:10 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 07:08:20 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 07:08:40 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 07:09:19 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 07:10:19 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 07:11:19 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 07:12:19 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 07:13:20 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 07:14:14 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 07:14:20 fwall /kernel: ipfw: 650 Deny TCP hostb:21 hosta:1112 in via ed1 fwall# tcpdump -i ed0 07:04:01.863247 hosta.1112 > hostb.ftp: P 147:157(10) ack 662 win 57964 (DF) [tos 0x10] 07:04:02.104378 hostb.ftp > hosta.1112: P 662:715(53) ack 157 win 17524 07:04:02.198438 hosta.1112 > hostb.ftp: . ack 715 win 57964 (DF) [tos 0x10] 07:04:02.428996 hostb.ftp > hosta.1112: P 715:745(30) ack 157 win 17524 07:04:02.528507 hosta.1112 > hostb.ftp: . ack 745 win 57964 (DF) [tos 0x10] 07:08:02.190741 hostb.ftp > hosta.1112: P 745:782(37) ack 157 win 17524 07:08:02.291580 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] 07:08:03.417151 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 07:08:03.417888 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] 07:08:05.877990 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 07:08:05.878703 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] 07:08:10.798902 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 07:08:10.799621 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] 07:08:20.651400 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 07:08:20.652125 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] 07:08:40.366279 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 07:08:40.367120 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] 07:09:19.796673 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 07:09:19.797487 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] 07:10:19.808705 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 07:10:19.809477 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] 07:11:19.821934 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 07:11:19.822731 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] 07:12:19.835383 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 07:12:19.836158 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] 07:13:19.849184 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 07:13:19.850012 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] 07:14:13.879800 hosta.1112 > hostb.ftp: P 157:163(6) ack 782 win 57927 (DF) [tos 0x10] 07:14:13.879993 hosta.1112 > hostb.ftp: F 163:163(0) ack 782 win 57964 (DF) [tos 0x10] 07:14:14.113211 hostb.ftp > hosta.1112: R 2194364317:2194364317(0) win 0 (DF) fwall# tcpdump -i ed1 07:04:02.103450 hostb.ftp > fwall-ed1.1112: P 662:715(53) ack 157 win 17524 07:04:02.122875 hostb.59915 > fwall-ed1.1115: P 1:223(222) ack 1 win 17524 07:04:02.122972 hostb.59915 > fwall-ed1.1115: F 223:223(0) ack 1 win 17524 07:04:02.125752 fwall-ed1.1115 > hostb.59915: . ack 224 win 57964 (DF) [tos 0x8] 07:04:02.140270 fwall-ed1.1115 > hostb.59915: F 1:1(0) ack 224 win 57964 (DF) [tos 0x8] 07:04:02.199236 fwall-ed1.1112 > hostb.ftp: . ack 715 win 57964 (DF) [tos 0x10] 07:04:02.369724 hostb.59915 > fwall-ed1.1115: . ack 2 win 17524 07:04:02.428304 hostb.ftp > fwall-ed1.1112: P 715:745(30) ack 157 win 17524 07:04:02.529223 fwall-ed1.1112 > hostb.ftp: . ack 745 win 57964 (DF) [tos 0x10] 07:08:02.189388 hostb.ftp > fwall-ed1.1112: P 745:782(37) ack 157 win 17524 07:08:03.416245 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 07:08:05.876702 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 07:08:10.797991 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 07:08:20.650392 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 07:08:40.364907 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 07:09:19.795268 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 07:10:19.807132 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 07:11:19.820499 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 07:12:19.834029 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 07:13:19.847717 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 07:14:13.882752 fwall-ed1.1112 > hostb.ftp: F 163:163(0) ack 782 win 57964 (DF) [tos 0x10] 07:14:14.112270 hostb.ftp > fwall-ed1.1112: R 2194364317:2194364317(0) win 0 (DF) 07:14:19.861397 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 07:15:19.875126 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 Why does ipfw close the outgoing path from hosta to hostb after 4 minutes of inactivity? According to the ipfw man page, ipfw2 will generate keep-alive packets when a keep-state rule is about to expire. Why don't I see any in the tcpdump traces? Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-security" in the body of the message