From owner-freebsd-bugs@FreeBSD.ORG Mon Sep 11 13:50:22 2006 Return-Path: X-Original-To: freebsd-bugs@hub.freebsd.org Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5E8BF16A417 for ; Mon, 11 Sep 2006 13:50:22 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id BABCC43D5A for ; Mon, 11 Sep 2006 13:50:17 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k8BDoHqG086859 for ; Mon, 11 Sep 2006 13:50:17 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k8BDoHFP086858; Mon, 11 Sep 2006 13:50:17 GMT (envelope-from gnats) Resent-Date: Mon, 11 Sep 2006 13:50:17 GMT Resent-Message-Id: <200609111350.k8BDoHFP086858@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Eugene Grosbein Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 91A8016A49E; Mon, 11 Sep 2006 13:41:54 +0000 (UTC) (envelope-from sa@nkz.delikates-nk.ru) Received: from nkz.delikates-nk.ru (nkz.delikates-nk.ru [81.16.143.102]) by mx1.FreeBSD.org (Postfix) with ESMTP id 92B7B43D49; Mon, 11 Sep 2006 13:41:53 +0000 (GMT) (envelope-from sa@nkz.delikates-nk.ru) Received: from nkz.delikates-nk.ru (localhost [127.0.0.1]) by nkz.delikates-nk.ru (8.13.8/8.13.8) with ESMTP id k8BDfogL020222; Mon, 11 Sep 2006 21:41:50 +0800 (KRAST) (envelope-from sa@nkz.delikates-nk.ru) Received: (from sa@localhost) by nkz.delikates-nk.ru (8.13.8/8.13.8/Submit) id k8BDfneZ020221; Mon, 11 Sep 2006 21:41:49 +0800 (KRAST) (envelope-from sa) Message-Id: <200609111341.k8BDfneZ020221@nkz.delikates-nk.ru> Date: Mon, 11 Sep 2006 21:41:49 +0800 (KRAST) From: Eugene Grosbein To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Cc: net@FreeBSD.org Subject: kern/103135: ipsec with ipfw divert (not NAT) encodes a packet twice breaking PMTUD X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Sep 2006 13:50:22 -0000 >Number: 103135 >Category: kern >Synopsis: ipsec with ipfw divert (not NAT) encodes a packet twice breaking PMTUD >Confidential: no >Severity: serious >Priority: high >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Mon Sep 11 13:50:16 GMT 2006 >Closed-Date: >Last-Modified: >Originator: Eugene Grosbein >Release: FreeBSD 6.1-STABLE i386 >Organization: Svyaz Service JSC >Environment: System: FreeBSD nkz.delikates-nk.ru 6.1-STABLE FreeBSD 6.1-STABLE #1: Thu Sep 7 13:31:53 KRAST 2006 root@nkz.delikates-nk.ru:/home/obj/home/src/sys/NKZ i386 options IPDIVERT options IPSEC options IPSEC_ESP >Description: When outgoing packet encoded due to corresponding IPSEC policy is passed to divert socket (f.e. to ipacctd for accounting), it is encoded second time with IPSEC then. Besides obvious logic error, this also results in broken Path MTU Discovery. >How-To-Repeat: Use a kernel with options IPDIVERT, IPSEC, IPSEC_ESP (my kernel also contains IPSEC_FILTERGIF, but this should not matter). Suppose there are two local nets numbered 192.168.1.0/24 and 192.168.2.0/24, each has a FreeBSD router (192.168.1.1 and 192.168.2.1). Routers make gif(4) tunnel between and use IPSEC transport mode to encrypt its contents. Their external IP addresses are 1.1.1.1 and 2.2.2.2 Here is /etc/ipsec.conf: add 1.1.1.1 81.16.143.102 esp 1007 -m transport -E blowfish-cbc "xxx"; add 2.2.2.2 1.1.1.1 esp 2007 -m transport -E blowfish-cbc "yyy"; spdadd 1.1.1.1/32 2.2.2.2/32 any -P in ipsec esp/transport/1.1.1.1-2.2.2.2/require; spdadd 2.2.2.2/32 1.1.1.1/32 any -P out ipsec esp/transport/2.2.2.2-1.1.1.1/require; Another router has similar /etc/ipsec.conf. Use this script to prepare IPSEC keys for tcpdump: #!/bin/sh setkey -D | awk ' /^[1-9]/ { ip=$2; } $1=="esp" { sub(/spi=[^(]+\(/, "", $3); sub(/\)/, "", $3); printf"%s@%s ",$3,ip; } $1=="E:" { printf "%s:0x%s%s%s%s\n", $2, $3, $4, $5, $6; } ' > /tmp/keys.txt When we use one of the routers to run "tcpdump -s0 -n -p -i $iface -E /tmp/keys.txt esp", we see that traffic is encoded with ESP and decoded properly, still good. Now install ports/net-mgmt/ipacctd, run it manually with "ipacctd -p 4000", then command: "ipfw add 10 divert 4000 ip from any to any out" and rerun tcpdump command shown above. You'll see that outgoing packets are encapsulated with ESP twice. The same you'll see at the other side for incoming packets. >Fix: Unknown. The only known workaround is to avoid diverting ESP. This workaroung is not always acceptable. Eugene Grosbein >Release-Note: >Audit-Trail: >Unformatted: