From owner-freebsd-net@freebsd.org Fri Jan 20 22:22:42 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4596CB9709 for ; Fri, 20 Jan 2017 22:22:42 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: from mail-it0-x22b.google.com (mail-it0-x22b.google.com [IPv6:2607:f8b0:4001:c0b::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7A6731658; Fri, 20 Jan 2017 22:22:42 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: by mail-it0-x22b.google.com with SMTP id r185so30770246ita.0; Fri, 20 Jan 2017 14:22:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=FuGKN+P+jILfDb2/Xfx5WXDi43zaje23eRTu+UHummA=; b=Lc1rr5CJ2CO2q+kQn1k4nJFr2taCSxQEya8jCSpfahplme/d0eve68SF1tzLaSZbrj wrmDN6nV9k2ZB060OmBnY8PXqaOqUMOBPiEI830dLkGTUPNc7VYq95M3SuhT9A31WaqR dcKAn1wNoKyib/WeGCPP1kafTzlX0xCe24E9DHZh57jgZl2YQnFLbAClz5YZwb5SBfCx uxSiZeOFbSC6JHDHmqOaiJGph/9ENzhGMRVaF6YmZFnP+YDl8102VKeLJYs+LAiOPN/U 1VsVx12Hq9FaUJWSB9o6OpZOpt8WbJbllfWV+LArrdSGy6BpEF9mi1LG2KX1pSCRRHH6 Z4ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=FuGKN+P+jILfDb2/Xfx5WXDi43zaje23eRTu+UHummA=; b=fdq28a7x/9HIxzdrrVpkowcJvJR682qFrw6+cYW4tbT00/6QuhKuLKDuYEIZdKxI3T F4fh/4Saq1q5XMt52A1EMEbO1CWrIc6RIbLrNt0ayXOLwZXXIHwFZd9S4tPu5ps43hpX 7Z5zvVSBFvnAkGCNUvZddebFIDlHtqxzlawZ62EtS1t8rQOiqwqx1/vRRvR4I9C09p1W qmOwKLXo+Y3HEwyoZjuZ2MjeiuAsGXMh/SPlzvroLEtNx6VMk/wp+9RGD9FnL2jjubzW dLYS4+xI78q/c9X5Q+3P/EfdD6axfXzOt1q1yamTqPEBEjW6st1xcrlg7fMmiQMw9uk/ 7GqA== X-Gm-Message-State: AIkVDXLLFaVxJKDAK+PbaZVfjWoiibYKyKqNtHi/zLMl4bn1QIcu1cvSieEuAmLTS+8PbUR/30SgsMU5l6OuFQ== X-Received: by 10.36.65.4 with SMTP id x4mr5721328ita.69.1484950961947; Fri, 20 Jan 2017 14:22:41 -0800 (PST) MIME-Version: 1.0 Sender: ermal.luci@gmail.com Received: by 10.107.129.99 with HTTP; Fri, 20 Jan 2017 14:22:41 -0800 (PST) In-Reply-To: <20170120211734.488D8124AEA5@mail.bitblocks.com> References: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> <7C29D00C-94C0-4550-B1B2-CE307482B544@FreeBSD.org> <20170120203106.CD2C8124AEA4@mail.bitblocks.com> <20170120205933.8948A124AEA3@mail.bitblocks.com> <20170120211734.488D8124AEA5@mail.bitblocks.com> From: =?UTF-8?Q?Ermal_Lu=C3=A7i?= Date: Fri, 20 Jan 2017 14:22:41 -0800 X-Google-Sender-Auth: TwWQqlXyidb1gjgCjgfD7I_oZdc Message-ID: Subject: Re: pf & NAT issue To: Bakul Shah Cc: FreeBSD Net , Alan Somers Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 22:22:42 -0000 On Fri, Jan 20, 2017 at 1:17 PM, Bakul Shah wrote: > On Fri, 20 Jan 2017 13:12:07 PST =?UTF-8?Q?Ermal_Lu=C3=A7i?= < > eri@freebsd.org> wrote: > > --001a1148cecc40685805468d1ad2 > > Content-Type: text/plain; charset=UTF-8 > > > > On Fri, Jan 20, 2017 at 12:59 PM, Bakul Shah > wrote: > > > > > On Fri, 20 Jan 2017 21:43:33 +0100 "Kristof Provost" > > > wrote: > > > > On 20 Jan 2017, at 21:31, Bakul Shah wrote: > > > > >> 11:56:28.168693 IP 192.168.125.7.65042 > 149.20.1.200.21: Flags > [P.], > > > > >> seq 1:10, ack 55, win 1026, options [nop,nop,TS val 198426 ecr > > > > >> 1468113725], length 9 > > > > > < 11:56:28.168712 IP 173.228.5.8.52015 > 149.20.1.200.21: Flags > [P.], > > > > > seq 3080825147:3080825156, ack 3912707414, win 1026, options > > > > > [nop,nop,TS val 198426 ecr 1468113725], length 9 > > > > > > > > > > Right here we see the problem. NAT mapping for the > > > > > port changed from 63716 to 52015. > > > > > > > > > Changing source ports is an entirely normal NAT behaviour. > > > > > > > > The best explanation is this: imagine that you have two clients A > and B, > > > > both connect to X on port 80 via the NAT gateway G. > > > > Both use port 1000 as their source port. > > > > A connects, and the gateway maps A:1000 -> X:80 to G:1000 -> X:80. > > > > B connects, and now the gateway has to map B:1000 -> X:80 onto > G:1000 -> > > > > X:80, but then it wouldn't be able to tell the two connections apart. > > > > That't can remap it onto G:1001 -> X:80 instead. > > > > > > It is the same connection! As a tcp connection is identified > > > by , If the port number > > > changes on the same connection, the remote side would see this > > > as a separate connection. > > > > > > > Most probably your timeouts are aggressive on states garbage collection. > > Give a look to those state limit teardown it might improve things. > > $ pfctl -s timeout > tcp.first 120s > tcp.opening 30s > tcp.established 86400s > tcp.closing 900s > tcp.finwait 45s > tcp.closed 90s > tcp.tsdiff 30s > udp.first 60s > udp.single 30s > udp.multiple 60s > icmp.first 20s > icmp.error 10s > other.first 60s > other.single 30s > other.multiple 60s > frag 30s > interval 10s > adaptive.start 6000 states > adaptive.end 12000 states > src.track 30s > > local port num changed after 23 seconds. All the tcp.* > timeouts seem ok. IIRC internal is used for IP frags. > Well if you do not overcome those limits than it means you most probably are receiving, out of order traffic pf does not like that much. That would be my guess if the limits and GC are not triggered. One other thing is those ip-option stats you have are weird and you probably should drop ip-option traffic. > Thanks > Bakul > -- Ermal