From owner-freebsd-pf@freebsd.org Fri Oct 2 12:59:55 2020 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4350242C61E for ; Fri, 2 Oct 2020 12:59:55 +0000 (UTC) (envelope-from SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C2qpL21hZz43l0 for ; Fri, 2 Oct 2020 12:59:54 +0000 (UTC) (envelope-from SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id D9FF728434 for ; Fri, 2 Oct 2020 14:59:45 +0200 (CEST) Received: from illbsd.quip.test (ip-94-112-144-235.net.upcbroadband.cz [94.112.144.235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 064AE2842E for ; Fri, 2 Oct 2020 14:59:45 +0200 (CEST) To: The Doctor via freebsd-pf From: Miroslav Lachman <000.fbsd@quip.cz> Subject: PF states limit reached Message-ID: Date: Fri, 2 Oct 2020 14:59:44 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4C2qpL21hZz43l0 X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz has no SPF policy when checking 94.124.105.4) smtp.mailfrom=SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz X-Spamd-Result: default: False [2.60 / 15.00]; RCVD_TLS_LAST(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(0.14)[0.136]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-pf@freebsd.org]; ARC_NA(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; DMARC_NA(0.00)[quip.cz]; TO_DN_ALL(0.00)[]; NEURAL_SPAM_MEDIUM(0.63)[0.626]; NEURAL_SPAM_LONG(0.64)[0.640]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz]; RECEIVED_SPAMHAUS_PBL(0.00)[94.112.144.235:received]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz]; MAILMAN_DEST(0.00)[freebsd-pf] X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Oct 2020 12:59:55 -0000 I have many machines (physical and virtual) with PF running for years. Few days back I started observing problem on one machine running in headless VirtualBox (if it matters) kernel: [zone: pf states] PF states limit reached The problem is there are states inserts but states are never removed (pfctl -s info shows 0 removals) If I run "pfctl -s state | wc -l" the count is the same as shown by "pfctl -s info | grep inserts". There are thousands of states after 30 minutes. "netstat -an" show only about 90 connections in WAIT or CLOSED or ESTABLISHED state. Why PF does not remove all states? What can be wrong on this machine in question? My current workaround is to restart PF many times a day (or use pfctl -F states) pf.conf if relatively simple, just a basic rules to allow incomming traffic for TCP services, allowing all outgoing traffic and some "set" options: set limit { states 200000, frags 5000 } set limit table-entries 900000 set optimization aggressive set block-policy drop set loginterface $ext_if set skip on $unfiltered scrub in on $ext_if scrub out on $ext_if no-df random-id And the last question - is there any way to use PF as stateless firewall? PF automatically add "keep state" to all rules, how can I change this behavior to not add "keep state" on all or some rules? Kind regards Miroslav Lachman From owner-freebsd-pf@freebsd.org Fri Oct 2 14:42:42 2020 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D5DB042EE39 for ; Fri, 2 Oct 2020 14:42:42 +0000 (UTC) (envelope-from kisscoolandthegangbang@hotmail.fr) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01olkn0801.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe1f::801]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C2t4x66Kjz4HV7 for ; Fri, 2 Oct 2020 14:42:41 +0000 (UTC) (envelope-from kisscoolandthegangbang@hotmail.fr) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=N0kXipCu50bNt/IoKfblwEdTjyIILmgcpIX6wzinR5rVYhY8p+vTtDJCT6KC1W/NEeYPH2Pd2g42DgDDPUveME+8fjYI55IO6AeI1kEw8uFGuWGkU9t93hdJbDc9ZpFkRCyaQoXJjTeTAc91I9KmOlHjPMsLqtz+mjGXFh5PiI5rnkeLL07/zoYZ28t4hNitYLEzIfD2eIuRjTVUBwyEIEnZ7pwY7hNIzdw6l7rcC9jqRYjlfe5P2VG0tGoqBaHLK7wXfiKVMSRUpIeqiojpAMKsT+cisuPj5HV6q5jhHjlokSL4/JqSL+WnkWyMJzo+DD/7lBeKT9i8CwYJzY0rmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1bEUSxul+6WEW1XLgvsj01WusPxasBilMWgbIybolFY=; b=HLOcJQXt4r9WU/GghWQI5/MaBODx1Iq8WwpNb6SIkSJ8vBt52i38/Z3Nz3nhoWAY/pE5rKIpOhAuy5vm2zC0VVrT7xJEYNUBVBZPKZQNT4oxPWROqtLMHglcKwrKGoJJBNa7z8gW89AsiOeIS4Zfgac0XLM323Z387YEK+YlL418wVWDoLsdmEyrxFKA7N4SpAJqzHuQBN0mI+6JHIHY1cmdQVFArO7OG1NFhB/ezlZu0PYSG7o2JTkNkFO2QNJZNiPBs0RAbR5RJGqhQikDoQp8d9edUC3cZD+cxrxed4zdqSk5+nLGx7xzKwE0+f5U3G678p3XRwgYiE8OW0U60Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none Received: from VE1EUR01FT016.eop-EUR01.prod.protection.outlook.com (2a01:111:e400:7e19::4a) by VE1EUR01HT136.eop-EUR01.prod.protection.outlook.com (2a01:111:e400:7e19::357) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.21; Fri, 2 Oct 2020 14:42:39 +0000 Received: from VE1PR03MB5629.eurprd03.prod.outlook.com (2a01:111:e400:7e19::50) by VE1EUR01FT016.mail.protection.outlook.com (2a01:111:e400:7e19::227) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3433.34 via Frontend Transport; Fri, 2 Oct 2020 14:42:39 +0000 X-IncomingTopHeaderMarker: OriginalChecksum:828F0D250E95AA5CF1A642AD40F58411A11F0C059411995FECE320A35C30699C; UpperCasedChecksum:4772FEF82098FF3367C4FCD9ADD3CE965B8257CB107CA5B09ED4607BB578868B; SizeAsReceived:7898; Count:47 Received: from VE1PR03MB5629.eurprd03.prod.outlook.com ([fe80::3440:3970:7a3a:b48f]) by VE1PR03MB5629.eurprd03.prod.outlook.com ([fe80::3440:3970:7a3a:b48f%7]) with mapi id 15.20.3433.038; Fri, 2 Oct 2020 14:42:39 +0000 Date: Fri, 2 Oct 2020 16:44:03 +0200 From: kaycee gb To: freebsd-pf@freebsd.org Subject: Re: PF states limit reached Message-ID: In-Reply-To: References: X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.31; x86_64-slackware-linux-gnu) Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-TMN: [EGcX97AUN8CIZpdRB6c4LQsMtPrPOzw6] X-ClientProxiedBy: AM4PR0902CA0008.eurprd09.prod.outlook.com (2603:10a6:200:9b::18) To VE1PR03MB5629.eurprd03.prod.outlook.com (2603:10a6:803:11e::30) X-Microsoft-Original-Message-ID: <20201002164403.2c8ad5dd@slackstro.home.lan> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mail.lacabanedeladmin.trickip.net (93.1.37.139) by AM4PR0902CA0008.eurprd09.prod.outlook.com (2603:10a6:200:9b::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3433.35 via Frontend Transport; Fri, 2 Oct 2020 14:42:38 +0000 Received: from slackstro.home.lan ([172.16.93.19]) (authenticated bits=0) by mail.lacabanedeladmin.trickip.net (8.15.2/8.15.2) with ESMTPSA id 092EgZFZ010949 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NO) for ; Fri, 2 Oct 2020 16:42:36 +0200 (CEST) (envelope-from kisscoolandthegangbang@hotmail.fr) X-MS-PublicTrafficType: Email X-IncomingHeaderCount: 47 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-Correlation-Id: d2a53c51-ac84-40cf-eb47-08d866e16917 X-MS-TrafficTypeDiagnostic: VE1EUR01HT136: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3kh/9rhpO3HIzlIkSuRnLkbhJSP9yDuKdRIYYuDYiIiPoW+1gYaSYXiJc4lenhADJA3U4uvx/ew7ysfKDrwSwg40G+rPuuAfzU/xoDcUQ++0sOuBfRU56OzrwTJZxS+XJeKL/mPQOOzY0XqE/P1de8eMA5woY7iAwFQjt+zL8OsTcTcbfMM3BDpSSREeosT7FzA/wQZcVPndKpXl9KethSn/HwgbeHN6vQI+PKgqbnWgk6i2SGPaqcR2bzNmUmwo X-MS-Exchange-AntiSpam-MessageData: VGBYEc5EasAVJfLiMabgmzfOGpH2P94VPhcUS9A85SH+3KI3Uroau7rzaVZ6bUst+fA62e3Wq9uvDMS8LFXwtwC43PVm9En+N31x3+VnVWK5JhBRWVR+98I1jM0REuXFHirwu96ndMSXHk6KtRhQsw== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: d2a53c51-ac84-40cf-eb47-08d866e16917 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Oct 2020 14:42:39.6161 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-AuthSource: VE1EUR01FT016.eop-EUR01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1EUR01HT136 X-Rspamd-Queue-Id: 4C2t4x66Kjz4HV7 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=hotmail.fr; spf=pass (mx1.freebsd.org: domain of kisscoolandthegangbang@hotmail.fr designates 2a01:111:f400:fe1f::801 as permitted sender) smtp.mailfrom=kisscoolandthegangbang@hotmail.fr X-Spamd-Result: default: False [-4.45 / 15.00]; RCVD_TLS_LAST(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[6]; RECEIVED_SPAMHAUS_PBL(0.00)[93.1.37.139:received]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[hotmail.fr]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-pf@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.97)[-0.969]; NEURAL_HAM_MEDIUM(-0.97)[-0.969]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f400::/48]; NEURAL_HAM_SHORT(-0.71)[-0.711]; DMARC_POLICY_ALLOW(-0.50)[hotmail.fr,none]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[hotmail.fr]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; MIME_TRACE(0.00)[0:+]; MAILMAN_DEST(0.00)[freebsd-pf]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1] X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Oct 2020 14:42:42 -0000 Le Fri, 2 Oct 2020 14:59:44 +0200, Miroslav Lachman <000.fbsd@quip.cz> a =E9crit : > I have many machines (physical and virtual) with PF running for years.=20 > Few days back I started observing problem on one machine running in=20 > headless VirtualBox (if it matters) >=20 > kernel: [zone: pf states] PF states limit reached >=20 > The problem is there are states inserts but states are never removed=20 > (pfctl -s info shows 0 removals) >=20 > If I run "pfctl -s state | wc -l" the count is the same as shown by=20 > "pfctl -s info | grep inserts". There are thousands of states after 30=20 > minutes. >=20 > "netstat -an" show only about 90 connections in WAIT or CLOSED or=20 > ESTABLISHED state. >=20 > Why PF does not remove all states? What can be wrong on this machine in=20 > question? >=20 > My current workaround is to restart PF many times a day (or use pfctl -F= =20 > states) >=20 > pf.conf if relatively simple, just a basic rules to allow incomming=20 > traffic for TCP services, allowing all outgoing traffic and some "set"=20 > options: >=20 > set limit { states 200000, frags 5000 } > set limit table-entries 900000 > set optimization aggressive > set block-policy drop > set loginterface $ext_if > set skip on $unfiltered >=20 > scrub in on $ext_if > scrub out on $ext_if no-df random-id >=20 >=20 > And the last question - is there any way to use PF as stateless=20 > firewall? PF automatically add "keep state" to all rules, how can I=20 > change this behavior to not add "keep state" on all or some rules? >=20 If you have a little set of rules, you can add a "no state" or "no-state" t= o the rule, check in man page, I am not sure about the syntax right now.=20 There may be also an option to change the default behaviour to not add "kee= p state" automatically. Once again looking in man page may help.=20 And that is strange, I agree, maybe some optimisation/option is the culprit= . But I don't know where to look. What version of FreeBSD are you using ? Tha= t may help others =20 > Kind regards > Miroslav Lachman > _______________________________________________ > freebsd-pf@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-pf > To unsubscribe, send any mail to "freebsd-pf-unsubscribe@freebsd.org" K. From owner-freebsd-pf@freebsd.org Fri Oct 2 15:54:19 2020 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 04EA8430A8C for ; Fri, 2 Oct 2020 15:54:19 +0000 (UTC) (envelope-from SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C2vgZ0sGdz4M7g for ; Fri, 2 Oct 2020 15:54:17 +0000 (UTC) (envelope-from SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 0690328431 for ; Fri, 2 Oct 2020 17:54:16 +0200 (CEST) Received: from illbsd.quip.test (ip-94-112-144-235.net.upcbroadband.cz [94.112.144.235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 1126A2842E for ; Fri, 2 Oct 2020 17:54:15 +0200 (CEST) Subject: Re: PF states limit reached To: freebsd-pf@freebsd.org References: From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <489adbd3-4400-0cf8-31f1-45509af31925@quip.cz> Date: Fri, 2 Oct 2020 17:54:13 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4C2vgZ0sGdz4M7g X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz has no SPF policy when checking 94.124.105.4) smtp.mailfrom=SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz X-Spamd-Result: default: False [1.34 / 15.00]; RCVD_TLS_LAST(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-pf@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; ARC_NA(0.00)[]; NEURAL_SPAM_MEDIUM(0.41)[0.407]; NEURAL_SPAM_LONG(0.60)[0.603]; DMARC_NA(0.00)[quip.cz]; NEURAL_HAM_SHORT(-0.87)[-0.869]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz]; RECEIVED_SPAMHAUS_PBL(0.00)[94.112.144.235:received]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz]; MAILMAN_DEST(0.00)[freebsd-pf] X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Oct 2020 15:54:19 -0000 On 02/10/2020 16:44, kaycee gb wrote: > Le Fri, 2 Oct 2020 14:59:44 +0200, > Miroslav Lachman <000.fbsd@quip.cz> a écrit : > >> I have many machines (physical and virtual) with PF running for years. >> Few days back I started observing problem on one machine running in >> headless VirtualBox (if it matters) >> >> kernel: [zone: pf states] PF states limit reached >> >> The problem is there are states inserts but states are never removed >> (pfctl -s info shows 0 removals) >> >> If I run "pfctl -s state | wc -l" the count is the same as shown by >> "pfctl -s info | grep inserts". There are thousands of states after 30 >> minutes. >> >> "netstat -an" show only about 90 connections in WAIT or CLOSED or >> ESTABLISHED state. >> >> Why PF does not remove all states? What can be wrong on this machine in >> question? >> >> My current workaround is to restart PF many times a day (or use pfctl -F >> states) >> >> pf.conf if relatively simple, just a basic rules to allow incomming >> traffic for TCP services, allowing all outgoing traffic and some "set" >> options: >> [...] >> >> >> And the last question - is there any way to use PF as stateless >> firewall? PF automatically add "keep state" to all rules, how can I >> change this behavior to not add "keep state" on all or some rules? >> > If you have a little set of rules, you can add a "no state" or "no-state" to > the rule, check in man page, I am not sure about the syntax right now. > > There may be also an option to change the default behaviour to not add "keep > state" automatically. Once again looking in man page may help. > > And that is strange, I agree, maybe some optimisation/option is the culprit. > But I don't know where to look. What version of FreeBSD are you using ? That > may help others I am sorry, it is on FreeBSD 11.4-p4 amd64. I tried to read man page, maybe not so carefully, but didn't found how to turn automatic keep state off. I also tried to search on the net without any luck. Thank you Miroslav Lachman From owner-freebsd-pf@freebsd.org Fri Oct 2 16:17:12 2020 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1A99D430D96 for ; Fri, 2 Oct 2020 16:17:12 +0000 (UTC) (envelope-from kisscoolandthegangbang@hotmail.fr) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-oln040092075076.outbound.protection.outlook.com [40.92.75.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C2w9y419Xz4N77 for ; Fri, 2 Oct 2020 16:17:10 +0000 (UTC) (envelope-from kisscoolandthegangbang@hotmail.fr) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cdG/p7Rk5tdzHaUC5v1/9W1aza9Xj34iVdcVKWe4Hi7gVCqBfuF/cMTl0XYjwfv54jYyldlElPiB3N1a9YEYbYSY3zCXXRnqeHo/hM9GQuDP16gQNVIZ0G0K+2u/xJnsPEItUFV1BvI54lOnKzcbZYgnHe/s9pNCeLrr+CMSaSmrqkKv0D4CflkSZHHcrkcj9f5JQIJInPEeLOLPkFt1GII9+YaK82BzTbL9ooL0OpoGWIvVdz7Oh+0tCT4UsiLNN5I2V55oHyHDdFseOg6bL7WxUxqsIqti5AbNfnhbah6pIOIMhnoTYmjdpMJRjOlI1IHX9zouG4L0n88AKTrkhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rZN6w0oF+lDmhr1P5ZsfCntzd8KvzzD2/r55il0wz2o=; b=YKH5Gq4OF6enM52JVo/7s1yNW/KguoCUOFFPoNjwAv+l0esXAhdKo2+ypx/ezxoiGLwFSLi2Cw5phi6LIrBMpuHOK9x6witOj1sDqJstRvIK/K2+CPybtUkLlNmIrmkv6xH71LmKNGcKP5xk3OL8U/Q339dKZGcryP8Trj+SxHxrxIrtqA/BAy8PuqfSU3TTU1kqDPlX6Ay2KNueuV8sSRhpH+/abiiKfYUwuThzT+jgWsO256Hs3HtnS5/VUkkQjKHMzPwvazkqrGPcJv7dCFCxVj4A+AdEeu6zRwyUjOCJNJzMiiUXwJz0RZr0jnLVRGl86H5tpsc+Dnj0jARBug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none Received: from HE1EUR04FT035.eop-eur04.prod.protection.outlook.com (2a01:111:e400:7e0d::50) by HE1EUR04HT004.eop-eur04.prod.protection.outlook.com (2a01:111:e400:7e0d::107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3433.34; Fri, 2 Oct 2020 16:17:08 +0000 Received: from VE1PR03MB5629.eurprd03.prod.outlook.com (2a01:111:e400:7e0d::49) by HE1EUR04FT035.mail.protection.outlook.com (2a01:111:e400:7e0d::294) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3433.34 via Frontend Transport; Fri, 2 Oct 2020 16:17:08 +0000 X-IncomingTopHeaderMarker: OriginalChecksum:95B426C7B3A301E86ABF934A0D0B6AD23617293347E9FF0C6418B1194EC3A054; UpperCasedChecksum:5E61DCBFCDBB8F8D965D813A2CAC74A8A5993DD625F44C44D2471465F05B9A4E; SizeAsReceived:8026; Count:47 Received: from VE1PR03MB5629.eurprd03.prod.outlook.com ([fe80::3440:3970:7a3a:b48f]) by VE1PR03MB5629.eurprd03.prod.outlook.com ([fe80::3440:3970:7a3a:b48f%7]) with mapi id 15.20.3433.038; Fri, 2 Oct 2020 16:17:08 +0000 Date: Fri, 2 Oct 2020 18:18:34 +0200 From: kaycee gb To: freebsd-pf@freebsd.org Subject: Re: PF states limit reached Message-ID: In-Reply-To: <489adbd3-4400-0cf8-31f1-45509af31925@quip.cz> References: <489adbd3-4400-0cf8-31f1-45509af31925@quip.cz> X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.31; x86_64-slackware-linux-gnu) Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-TMN: [J+r5c3vvZfWSm68/vMnxiSwIyC5sp/3q] X-ClientProxiedBy: AM4P190CA0015.EURP190.PROD.OUTLOOK.COM (2603:10a6:200:56::25) To VE1PR03MB5629.eurprd03.prod.outlook.com (2603:10a6:803:11e::30) X-Microsoft-Original-Message-ID: <20201002181834.0079e7db@slackstro.home.lan> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mail.lacabanedeladmin.trickip.net (93.1.37.139) by AM4P190CA0015.EURP190.PROD.OUTLOOK.COM (2603:10a6:200:56::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3433.32 via Frontend Transport; Fri, 2 Oct 2020 16:17:08 +0000 Received: from slackstro.home.lan ([172.16.93.19]) (authenticated bits=0) by mail.lacabanedeladmin.trickip.net (8.15.2/8.15.2) with ESMTPSA id 092GH5P7031654 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NO) for ; Fri, 2 Oct 2020 18:17:06 +0200 (CEST) (envelope-from kisscoolandthegangbang@hotmail.fr) X-MS-PublicTrafficType: Email X-IncomingHeaderCount: 47 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-Correlation-Id: 14b1bdf3-3805-4b93-7d74-08d866ee9c34 X-MS-TrafficTypeDiagnostic: HE1EUR04HT004: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sH204k9Xx5EGow117jaencL/DdSl84y3QBUBajyRwvkRWFwIMZnMmbj1syQdcHF1+4+7MDsk5SUlcznFm+CEvZUDwsl6yyyQJNIkKFH+UedFVKj5T/HU5fryji3uLb56wfMum3XyY90Z2g1wKDrcP6Tj9r/iVxn9DikdH+Rd1XUBiSwvnWruzAflyBYHu9RcVlD+iu2cDNCzZ492N2hkv3YgPnYKZfml86QGJwDEX+GsffYO+kWFDhLEOaviJqm5 X-MS-Exchange-AntiSpam-MessageData: mPfjrebBloq6fcEbbT88uEd8so8Fay8ZJRBt6aLytnTjX3jpm7kThVyD4DTazvTjqTJIiAqNvIR1CCjHTcRi+bBtP3xGaQMVodNR2g1U1CnfzPNVTCSZQBWa4DxsS1z7aMoad+0LI6uKm6yRP+2rAw== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 14b1bdf3-3805-4b93-7d74-08d866ee9c34 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Oct 2020 16:17:08.4698 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-AuthSource: HE1EUR04FT035.eop-eur04.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1EUR04HT004 X-Rspamd-Queue-Id: 4C2w9y419Xz4N77 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=hotmail.fr; spf=pass (mx1.freebsd.org: domain of kisscoolandthegangbang@hotmail.fr designates 40.92.75.76 as permitted sender) smtp.mailfrom=kisscoolandthegangbang@hotmail.fr X-Spamd-Result: default: False [-4.36 / 15.00]; RCVD_TLS_LAST(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[6]; RECEIVED_SPAMHAUS_PBL(0.00)[93.1.37.139:received]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[hotmail.fr]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-pf@freebsd.org]; TO_DN_NONE(0.00)[]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.98)[-0.981]; NEURAL_HAM_MEDIUM(-0.98)[-0.976]; R_SPF_ALLOW(-0.20)[+ip4:40.92.0.0/15]; NEURAL_HAM_SHORT(-0.60)[-0.600]; DMARC_POLICY_ALLOW(-0.50)[hotmail.fr,none]; RCVD_IN_DNSWL_NONE(0.00)[40.92.75.76:from]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[hotmail.fr]; ASN(0.00)[asn:8075, ipnet:40.80.0.0/12, country:US]; MIME_TRACE(0.00)[0:+]; MAILMAN_DEST(0.00)[freebsd-pf]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.92.75.76:from] X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Oct 2020 16:17:12 -0000 Le Fri, 2 Oct 2020 17:54:13 +0200, Miroslav Lachman <000.fbsd@quip.cz> a =E9crit : > On 02/10/2020 16:44, kaycee gb wrote: > > Le Fri, 2 Oct 2020 14:59:44 +0200, > > Miroslav Lachman <000.fbsd@quip.cz> a =E9crit : > > =20 > [...] =20 >=20 > [...] >=20 > [...] =20 > > If you have a little set of rules, you can add a "no state" or "no-stat= e" to > > the rule, check in man page, I am not sure about the syntax right now. > >=20 > > There may be also an option to change the default behaviour to not add = "keep > > state" automatically. Once again looking in man page may help. > >=20 > > And that is strange, I agree, maybe some optimisation/option is the cul= prit. > > But I don't know where to look. What version of FreeBSD are you using ?= That > > may help others =20 >=20 > I am sorry, it is on FreeBSD 11.4-p4 amd64. >=20 > I tried to read man page, maybe not so carefully, but didn't found how=20 > to turn automatic keep state off. I also tried to search on the net=20 > without any luck. >=20 Looking quickly, can't find too. Maybe I was thinking about "set state-defaults".=20 I'm afraid you'll have to use "no state" manually for each rule.=20 > Thank you >=20 > Miroslav Lachman > _______________________________________________ > freebsd-pf@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-pf > To unsubscribe, send any mail to "freebsd-pf-unsubscribe@freebsd.org" >=20 From owner-freebsd-pf@freebsd.org Fri Oct 2 17:35:40 2020 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7B71F432998 for ; Fri, 2 Oct 2020 17:35:40 +0000 (UTC) (envelope-from jdavidlists@gmail.com) Received: from mail-lf1-x141.google.com (mail-lf1-x141.google.com [IPv6:2a00:1450:4864:20::141]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C2xwW3hJSz4RYs for ; Fri, 2 Oct 2020 17:35:39 +0000 (UTC) (envelope-from jdavidlists@gmail.com) Received: by mail-lf1-x141.google.com with SMTP id 77so2848707lfj.0 for ; Fri, 02 Oct 2020 10:35:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=VT3lXANOtM5gKp6EUBNU5rZRmDpqC4GFom33G1MVjms=; b=qwIRwj1HOQQdFwOtmx0Bw3w6FNHhjlpL3RSxs9j9dGC7xwBR4jfhS/xss8oknEC6as hVSZ49w7xX/XQhpnEH+xkdyKfFdK3Y98btjJMp3hZ8EBPOoqnApfjg3l/+SdlqNwLuW2 5x9QlsxTd7SADtmeXgU5iYdsCr0QJXOwy90haQ/huOd0QgmQqJLXKkb9Jr4BSXcXttUX ypglhRJi6U1/MkjQHz61vd0zyL4klqPwyE5ck7FNvAEsUVD/Seha2OJVkB0GlR29mq1H ywuPIgdWyhYjggLc4RBkgiscZUItSFFSfUCb05WgDIKTpwI/j13lnnNRrLeXVXBoeap4 lAcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=VT3lXANOtM5gKp6EUBNU5rZRmDpqC4GFom33G1MVjms=; b=qLllYzREOsU9AdmY714CG6nQqJLq+I2YqhzT/bZ0AVeVNe+uKb1Tc4hZUwsIQCaPqE B/QP8Bh2KGdNpasZ+BoEV7tGsa1yqEusMiclnAsUa1qTKweSYBNWNu23bfKgl5qeDmd6 /YV3CF6ZEVxRvTkTkvIU3i3EcpqH1LCuMDBwcGQ6UcV3idcwwxm2y5fIwq7cVPbaoMNu 3SygZbprkJwp9xqp1WhpQIUmLgTNRKuxgoAERTMmodDVuNHS5cY4XQpHXS+zdHUIS7HV g9QjiYWKkIwHyIcOX886HZ4cTKUKnSgnGTr+5jxhLJhv+mfNlmZFpyjGVWOXa6xPJfJI trzw== X-Gm-Message-State: AOAM5319OrdSkqUoYhnEfs3GAlxBcTdZQvX+1Bn5NY17ncGdsFX9UZHk K042f4ERRIvc6HZ1n4MhhZoHef37CbixOfk/vIrL2FOg2yQ= X-Google-Smtp-Source: ABdhPJyJ+WMy+vMr7uINchEc8eCHSK+Qil4/YunaI5JZruavl9xf0iAVmaudLzGvXLyGR9Y2llGaNhNLG020xSRfVcA= X-Received: by 2002:ac2:548d:: with SMTP id t13mr1336319lfk.602.1601660137173; Fri, 02 Oct 2020 10:35:37 -0700 (PDT) MIME-Version: 1.0 From: J David Date: Fri, 2 Oct 2020 13:35:26 -0400 Message-ID: Subject: Packets passed by pf don't make it out? To: freebsd-pf@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4C2xwW3hJSz4RYs X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=qwIRwj1H; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of jdavidlists@gmail.com designates 2a00:1450:4864:20::141 as permitted sender) smtp.mailfrom=jdavidlists@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.04)[-0.040]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.960]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-pf@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::141:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-pf] X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Oct 2020 17:35:40 -0000 We have pf running on a FreeBSD 11.4 system acting as a load balancer, mapping a set of 8 external DNS service IP addresses to a set of internal DNS servers, any of which can handle those requests. When UDP packets from one source IP/port arrive for multiple external IPs in a short period of time, pf claims they all pass, but only the ones for the first IP actually make it out the outbound interface. Redirect rule: rdr inet proto udp from any to { 172.17.53.1, 172.17.53.2, 172.17.53.3, 172.17.53.4, 172.17.53.5, 172.17.53.6, 172.17.53.7, 172.17.53.8 } port 53 -> { 10.53.0.1, 10.53.0.2, 10.53.0.3 } round-robin sticky-address Pass rule: pass in log quick proto udp to { 10.53.0.1, 10.53.0.2, 10.53.0.3 } port 53 (The pass rule isn't technically necessary, it's only there to log the packets to debug this issue.) With tcpdumps running simultaneously on ix1, all packets show up the inbound interface: 16:32:39.183168 IP 149.20.1.48.56246 > 172.17.53.1.53: 3215 SOA? example.com. (29) 16:32:39.183761 IP 149.20.1.48.56246 > 172.17.53.2.53: 2934 SOA? example.com. (29) 16:32:39.184368 IP 149.20.1.48.56246 > 172.17.53.3.53: 52875 SOA? example.com. (29) 16:32:39.185618 IP 149.20.1.48.56246 > 172.17.53.4.53: 36289 SOA? example.com. (29) 16:32:39.186067 IP 149.20.1.48.56246 > 172.17.53.5.53: 44049 SOA? example.com. (29) 16:32:39.186422 IP 149.20.1.48.56246 > 172.17.53.6.53: 34410 SOA? example.com. (29) 16:32:39.186494 IP 149.20.1.48.56246 > 172.17.53.7.53: 30923 SOA? example.com. (29) 16:32:39.188541 IP 149.20.1.48.56246 > 172.17.53.8.53: 48814 SOA? example.com. (29) and on pflog0: 16:32:39.183189 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 > 10.53.0.1.53: 3215 SOA? example.com. (29) 16:32:39.183780 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 > 10.53.0.1.53: 2934 SOA? example.com. (29) 16:32:39.184375 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 > 10.53.0.1.53: 52875 SOA? example.com. (29) 16:32:39.185625 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 > 10.53.0.1.53: 36289 SOA? example.com. (29) 16:32:39.186074 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 > 10.53.0.1.53: 44049 SOA? example.com. (29) 16:32:39.186425 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 > 10.53.0.1.53: 34410 SOA? example.com. (29) 16:32:39.186499 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 > 10.53.0.1.53: 30923 SOA? example.com. (29) 16:32:39.188548 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 > 10.53.0.1.53: 48814 SOA? example.com. (29) but only the first one appears on ix0, the outbound interface: 16:32:39.183211 IP 149.20.1.48.56246 > 10.53.0.1.53: 3215 SOA? example.com. (29) The actual query order is random, so if the test is repeated a minute later, then 172.17.53.3 might be hit first, and then that one will make it through and the rest will disappear. So it is not specific to any destination IP. It also only appears to occur when the UDP source port is the same across the connections. (This is probably why TCP is not affected.) It does not appear related to state entries ("no state") doesn't help. If "sticky-address" is removed from the rdr, then one packet will make it through for each backend IP, instead of one total. What could be causing this? It seems somehow related to 5-tuple non-uniqueness after the rdr, but that shouldn't be an issue for UDP; it should be treated as two connectionless packets from the same source for the same destination. (The query test source, 149.20.1.48 is an EDNS checker found at https://dnsflagday.net/2020/ .) Thanks for any advice! From owner-freebsd-pf@freebsd.org Fri Oct 2 22:28:13 2020 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0F3163F0F9F for ; Fri, 2 Oct 2020 22:28:13 +0000 (UTC) (envelope-from SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C34Q340Ypz4kNW for ; Fri, 2 Oct 2020 22:28:11 +0000 (UTC) (envelope-from SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 88ECE28417 for ; Sat, 3 Oct 2020 00:28:08 +0200 (CEST) Received: from illbsd.quip.test (ip-94-112-144-235.net.upcbroadband.cz [94.112.144.235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 95E7A2840C for ; Sat, 3 Oct 2020 00:28:07 +0200 (CEST) Subject: Re: PF states limit reached To: freebsd-pf@freebsd.org References: <489adbd3-4400-0cf8-31f1-45509af31925@quip.cz> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <9c2bc3f6-0420-fe79-ae36-8a62511f71b2@quip.cz> Date: Sat, 3 Oct 2020 00:28:05 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4C34Q340Ypz4kNW X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz has no SPF policy when checking 94.124.105.4) smtp.mailfrom=SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz X-Spamd-Result: default: False [2.13 / 15.00]; RCVD_TLS_LAST(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-pf@freebsd.org]; TO_DN_NONE(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; ARC_NA(0.00)[]; NEURAL_SPAM_MEDIUM(0.58)[0.578]; NEURAL_SPAM_LONG(0.59)[0.592]; DMARC_NA(0.00)[quip.cz]; NEURAL_HAM_SHORT(-0.24)[-0.237]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz]; RECEIVED_SPAMHAUS_PBL(0.00)[94.112.144.235:received]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=YstE=DJ=quip.cz=000.fbsd@elsa.codelab.cz]; MAILMAN_DEST(0.00)[freebsd-pf] X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Oct 2020 22:28:13 -0000 On 02/10/2020 18:18, kaycee gb wrote: > Le Fri, 2 Oct 2020 17:54:13 +0200, > Miroslav Lachman <000.fbsd@quip.cz> a écrit : > >> On 02/10/2020 16:44, kaycee gb wrote: >>> If you have a little set of rules, you can add a "no state" or "no-state" to >>> the rule, check in man page, I am not sure about the syntax right now. >>> >>> There may be also an option to change the default behaviour to not add "keep >>> state" automatically. Once again looking in man page may help. >>> >>> And that is strange, I agree, maybe some optimisation/option is the culprit. >>> But I don't know where to look. What version of FreeBSD are you using ? That >>> may help others >> >> I am sorry, it is on FreeBSD 11.4-p4 amd64. >> >> I tried to read man page, maybe not so carefully, but didn't found how >> to turn automatic keep state off. I also tried to search on the net >> without any luck. >> > Looking quickly, can't find too. Maybe I was thinking about "set > state-defaults". > > I'm afraid you'll have to use "no state" manually for each rule. I will try to add "no state" to each rule. This is how stats looks after few hours: # pfctl -s info Status: Enabled for 0 days 09:39:07 Debug: Urgent Interface Stats for em0 IPv4 IPv6 Bytes In 829122714 0 Bytes Out 3363291237 0 Packets In Passed 2039822 0 Blocked 4248 0 Packets Out Passed 3047245 0 Blocked 321 0 State Table Total Rate current entries 164 searches 5091731 146.5/s inserts 83739 2.4/s removals 9886 0.3/s Counters match 88304 2.5/s bad-offset 0 0.0/s fragment 0 0.0/s short 0 0.0/s normalize 0 0.0/s memory 0 0.0/s bad-timestamp 0 0.0/s congestion 0 0.0/s ip-option 0 0.0/s proto-cksum 0 0.0/s state-mismatch 4 0.0/s state-insert 0 0.0/s state-limit 0 0.0/s src-limit 0 0.0/s synproxy 0 0.0/s map-failed 0 0.0/s About 8000 of removals was caused by one "pfctl -F states" after 1 hour of run. There are more than 74 000 thousands of states at this time. # pfctl -s state | wc -l 74294 Miroslav Lachman