From owner-freebsd-pf@freebsd.org Wed Oct 31 11:45:39 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7814610D481D for ; Wed, 31 Oct 2018 11:45:39 +0000 (UTC) (envelope-from jjasen@gmail.com) Received: from mail-qt1-x82d.google.com (mail-qt1-x82d.google.com [IPv6:2607:f8b0:4864:20::82d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0C4A270DD4 for ; Wed, 31 Oct 2018 11:45:39 +0000 (UTC) (envelope-from jjasen@gmail.com) Received: by mail-qt1-x82d.google.com with SMTP id v1-v6so13013102qtq.5 for ; Wed, 31 Oct 2018 04:45:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=to:from:openpgp:autocrypt:subject:message-id:date:user-agent :mime-version:content-language; bh=YTILJoAjTW8qNEAUpE/loCaw83SXZB607TjYsHiDm9A=; b=XeH/S0QPfgrkhuz1ZS80gkUH2ERKHg8+yM2HSznn7C4k8TdTHQpLtpWKxSXR3XzK+p VgOcvv3Tu+tSbT4EbLxaHnzXqj2uPLuzmseCNa+tp9Vjz9wwIG0r6A5tz/WZYLm7YAD/ gUHGiBXbeOpVQSRxEedVpJXTSIfI9xEXwikjLS8etTr2h99+6Jqny0rL07B4+2uoScaX w76xjHxbchh3QLjHkTQg/ohbNlrdnnOVpfSyY+h9BxM5DCRrQWWMsLlHLxBUW2SF5zBn PBHh/ayZ0vlmNLBqsTKNJY5xShg0SC8+v9mZNYqQh5Jl+SnP6EdSrg9U7eQyeMfMZ50v Okuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:openpgp:autocrypt:subject:message-id :date:user-agent:mime-version:content-language; bh=YTILJoAjTW8qNEAUpE/loCaw83SXZB607TjYsHiDm9A=; b=WAF3kTy3alm3WEIeZYnoGBhDOtynRKnljM+KDh1U7DVTE5dEKJ2igFvQQs2/YLiTb6 O5sXqMmfEVb7i+8uG5R3GYsxDZorcgaom1TFvhlmzCKn/NjcnB+oqxo5yu8ieJ5M86cy Zod8OvHpAfgleqqDEWVuq4L7Q69nZbFHR24w+S2+NZaO6z+u/gL+YpT8sivPR+kfnEjy 8ExsP48pTYumxW/ZU/QvAjZ+VInRvPnnSHCRD3uhPhzHBXFhUWFbasBO+aZ4rf5MVvsn DyerZDfvjKEFMkun6me1seqrGKOHfgLr5h44ybr63aPwHF3GQ8+DpxnQErc/Is/ZS4+y IWGQ== X-Gm-Message-State: AGRZ1gLcIMlBWyvNiXi3ee2osYu4Fl1nFtW9NorbfGCx7FiHOgqN76hc /qYrR29i3Q5tPDoPK5ISmjo9kQ2f X-Google-Smtp-Source: AJdET5emQVGv9vOsOrqdHKEWVzL+Q1BPO2jVB+n0y+qI4iG3dZwL9QhqkhUIz0nNb8BoTcU5GUN8VA== X-Received: by 2002:aed:3445:: with SMTP id w63-v6mr2241165qtd.346.1540986338096; Wed, 31 Oct 2018 04:45:38 -0700 (PDT) Received: from [192.168.1.7] (pool-71-166-47-163.bltmmd.fios.verizon.net. [71.166.47.163]) by smtp.googlemail.com with ESMTPSA id k188-v6sm14252042qkb.3.2018.10.31.04.45.37 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 31 Oct 2018 04:45:37 -0700 (PDT) To: FreeBSD PF From: John Jasen Openpgp: preference=signencrypt Autocrypt: addr=jjasen@gmail.com; keydata= xsDiBD6teiwRBADgIwslBO2Zcvj4fSulZLBxkGozRwkw75Sp7flp2I7f3Xv2SJNnNdDR2fSP WkJDZKR6897HgTO+ymFUOAbashYC3HWShPwskhpf7IE7kjaCacZ/9J47Hr/ZXpYPfRZXz1rP oX64I3rZRJmZe7mYGwXHrjth5aGGQz8JjOR9hFrBxwCgzeWoGVo6oG+f7U4vqT3WidCNIjME AI2ATIcgh9k6faiY+PQHl/NmzDojhs11jWN+z8cgAxfDtEzEkY+516kaBuG5Z8BKj6RlxH4/ crwx9GB2Uxwx5sS1tjahPVBq1toB7UDVKXq5Azeh9AY1EiAMDJyyUOmYgPvbcWTYZ4h7Khrs x1H2fz2H82btjnVaihwu8/djqpqzBAC3cjQ3mYovFIqHIIf/TWY/XS1c9V1zuYjgd6vDW9qV DFB9u6ZOw7sNfESjC9KlPbSfMigKd3RgYmIDhuNA2iE2evjYj8LioKXzx/rcUgbUD5pikKfg 6KexF88CuLMJYxc4YJUat/OEIYp0tXJyu3E714tfqzzBcmtgbWk2bmgWN80zSm9obiBKYXNl biAobWFpbiBlbWFpbCkgPGpqYXNlbkByZWFsaXR5ZmFpbHVyZS5vcmc+wloEExECABoFCwcK AwQDFQMCAxYCAQIXgAUCTJNrlQIZAQAKCRB8EKIWAeDGRBQpAKCTtcJ4jz50JWsjCOrMG69K 3Fs3VwCgo3kr8SqPP24Xw9W2kM1m4tdZzljOwE0EPq16QhAEAPgfFATG5kmX4yjcOj2bilTD 9lTdvJUCaQ5FdLycGx9sseMMwaWUlsiTv25LyIQUZ3z8ifmtsylmYefEun/bpJw5gCGMfXKm ZuXbA3AqbI3U0SthZmbn2P9CPfuMVDAut+f3FdZzLH1NlAS1kY1u2rxzK1R1SLgb2KnTD2DN BxGDAAMFBACsP6W5kxawClUBQnQgvN46gxLlt0eNM4tVmH5wR2I7WTnh45Dy5jqnC8WOYbRn yDrySnA7ZKStiJBSxNHDKBXniRpRNmWWXoLrITaIyPo8NukKObHKKIP+FupSdg5Uo1C1iJkQ 6iarV4uO3fgCttnAp7/mhs1YswvKHWA3orELFcJGBBgRAgAGBQI+rXpCAAoJEHwQohYB4MZE TG8An1eJgR0d60NGrYUKVvnccUefUaS3AJ919cZDBUSA/t2Da6D0pUG+OCU43w== Subject: NFSv4 connections and pf: BAD state stalling issues? Message-ID: Date: Wed, 31 Oct 2018 07:45:36 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 Content-Language: en-US Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Oct 2018 11:45:39 -0000 We run pf-based firewalls between linux-based servers and linux-clients over NFSv4. Periodically, events we've not pinned down cause the connection to be blocked at the firewall, manifesting as stale NFS mounts on the clients. These blocks were not logged at normal levels in pflog. I need to double check to see if enabling verbose logging has helped. The only way we've found to unblock them is to manually flush the state between the offending clients and the server with pfctl -k server-ip -k client-ip Before flushing the state table, pfctl -x loud will show: kernel: pf: BAD state: TCP in wire: client-ip:priv-port server-ip:2049 stack: - [lo=3D1342594619 high=3D1342782267 win=3D38400 modulator=3D0 wsc= ale=3D11] [lo=3D905052699 high=3D982817819 win=3D733 modulator=3D0 wscale=3D8] 4:4 = S seq=3D4197460108 (4197460108) ack=3D905052699 len=3D0 ackskew=3D0 pkts=3D290647578:883730744 dir=3Din,fwd So, it looks to me like the client lost contact initially, and is attempting to re-establish the connection. Given its recycling the same source port and destination and its a new SYN, this drives pf to declare the state bad and drop it. Any ideas on how to address this? Or where to look for issues? Thanks in advance! -- John Jasen