From owner-freebsd-pf@freebsd.org Thu Apr 1 11:37:15 2021 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9B9585C9B46 for ; Thu, 1 Apr 2021 11:37:15 +0000 (UTC) (envelope-from f0x0ff@gmail.com) Received: from mail-ot1-x335.google.com (mail-ot1-x335.google.com [IPv6:2607:f8b0:4864:20::335]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FB1PQ5ZWXz3QX8 for ; Thu, 1 Apr 2021 11:37:14 +0000 (UTC) (envelope-from f0x0ff@gmail.com) Received: by mail-ot1-x335.google.com with SMTP id w31-20020a9d36220000b02901f2cbfc9743so1802522otb.7 for ; Thu, 01 Apr 2021 04:37:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=TgkVzRDaS/RHAg8oFvWl8vgG1DrkxXjEUONPqeYe+Ck=; b=QzMcSo3nF221Y5hv/Yr+yh/ZE418rfXiYfCsdAsnqVRUeh020OmOyoaHNp+++FmsTF yb9mUFcCFW0vQmSeICRXUnqfaRsBtc4zR52M5pmujyut1BFzDcVUSjn1eNg3CKDr47v6 NTns0O5BMmVo49Nj7d7KNsM3DA92sVU0+CAcbCYneVL91qESJr+zEgmk1x6986Tgjau2 GyKPbS5Qxq9R5LwtBZVyfGwuHDJYjs/sc08FiP4kwzO/qN7KVoPNpZOTr7Eb3CgQ7LUU IbLP54Bx3XFDQ7inoZltWMy171jHEpcipHWoMl9VfY5LgJFqPcdR30mErN8TN2OfzvNp D1rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=TgkVzRDaS/RHAg8oFvWl8vgG1DrkxXjEUONPqeYe+Ck=; b=f7WCfKM1xAzBIRbFUvQdmV+bWpwCa3achi7c6S7YqQO8qHKqtpxC4ltCPfI7SYz8ea O2bYI881s8z48AEB3ZGKEO7mi2hC5KKPNCdjLM4ozvIMxM3sFc4slNGaEBa4RtAMcUjT yl4tgW3t71+O+aygtJZR2j0qyxOAkq0yJSoryTUeU3ug+zaQ5AbNiiztsIebVmPzqnU3 Mq+VwNGXrSfPb5U8YHpxrjf++DzQQEjaOM5A/8kfd6DuzJakgMkMky4PZkCNU/rimVTC zB3M269ywDha1PZBYWImxzvzi5Se2FSLvceK6Pd6sEfF+oc5ReWNBPpmpjQOipehHLpa PwQw== X-Gm-Message-State: AOAM530D/d74fakeRpL/KEMfzBOcgmBvIvVxf1Xa9UeTg/Ka6GgwYn8T wQINzN2wR24wrR5ZYW8CrPtcTY1uTURVB/Cof5zQO0CAx6vLTQ== X-Google-Smtp-Source: ABdhPJx+r1z2d67FADryI/IRY8lzBpzHPfevVEwspwYTxg1FD6vXRzXhV/J1Xg4SvlUXO+qU8dqWlm5xeDZJSoyW83k= X-Received: by 2002:a9d:550b:: with SMTP id l11mr6366678oth.218.1617277033839; Thu, 01 Apr 2021 04:37:13 -0700 (PDT) MIME-Version: 1.0 From: Plamen Mladenov Date: Thu, 1 Apr 2021 14:37:03 +0300 Message-ID: Subject: pfsync - Active/Active + defer on To: freebsd-pf@freebsd.org X-Rspamd-Queue-Id: 4FB1PQ5ZWXz3QX8 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=QzMcSo3n; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of f0x0ff@gmail.com designates 2607:f8b0:4864:20::335 as permitted sender) smtp.mailfrom=f0x0ff@gmail.com X-Spamd-Result: default: False [-2.02 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::335:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-pf@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::335:from:127.0.2.255]; NEURAL_SPAM_SHORT(0.98)[0.984]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::335:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-pf] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Apr 2021 11:37:15 -0000 Hello, I'm trying to setup an active-active PF cluster (consists of 2 freebsd hosts:FW1 and FW2) using pfsync and dynamic routing protocol (CARP is not used at this deployment) All works as expected when IN/OUT traffic for a single session is symmetrical (uses either FW1 or FW2). The problem I'm facing is when the traffic is asymmetrical For example: (1) Client ----------TCP SYN ---------> FW1 -------------------------------------> Server | pfsync | (2) Client <--------------------------------- FW2 <------- TCP SYN+ACK------- Server Client is sending TCP segment with SYN flag set which is received and allowed by FW1 and send to the Server. Server is replying with TCP segment with SYC and ACK flag sets (just as per TCP 3 way handshake), but this TCP segment is routed to FW2. At that time FW2 haven't received the SYNC-SENT session (from FW1) yet and therefore it denies that TCP segment. Few miliseconds after that FW2 gets the session from FW1, but the SYN+ACK is already dropped and a TCP re-transmission occurs. I've found that this behavior can be fixed with pfsync "defer" option, however based on my lab and prod tests - this option is not changing anything. As per my understandings, the initial packet should be delayed until session is replicated between both firewalls, but that's not the case. My other concern is that although the "defer" option is there (I can successfully turn it on/off and see it with ifconfig pfsync0) I can't find a word about it in man 4 pfsync on FreeBSD (unlike in OpenBSD documentation) which it makes me think - there is a reason why it's not in the man page. Can someone confirm - is pfsync "defer" option working on FreeBSD? Regard, Plamen Mladenov