From owner-freebsd-pf@FreeBSD.ORG Mon May 16 11:07:11 2011 Return-Path: Delivered-To: freebsd-pf@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E3DF3106566B for ; Mon, 16 May 2011 11:07:10 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D16988FC2A for ; Mon, 16 May 2011 11:07:10 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p4GB7AJp071289 for ; Mon, 16 May 2011 11:07:10 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p4GB7A8L071287 for freebsd-pf@FreeBSD.org; Mon, 16 May 2011 11:07:10 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 16 May 2011 11:07:10 GMT Message-Id: <201105161107.p4GB7A8L071287@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-pf@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-pf@FreeBSD.org X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 May 2011 11:07:11 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/155736 pf [pf] [altq] borrow from parent queue does not work wit o kern/153307 pf [pf] Bug with PF firewall o kern/148290 pf [pf] "sticky-address" option of Packet Filter (PF) blo o kern/148260 pf [pf] [patch] pf rdr incompatible with dummynet o kern/147789 pf [pf] Firewall PF no longer drops connections by sendin o kern/146832 pf [pf] "(self)" not always matching all local IPv6 addre o kern/143543 pf [pf] [panic] PF route-to causes kernel panic o bin/143504 pf [patch] outgoing states are not killed by authpf(8) o conf/142961 pf [pf] No way to adjust pidfile in pflogd o conf/142817 pf [patch] etc/rc.d/pf: silence pfctl o kern/141905 pf [pf] [panic] pf kernel panic on 7.2-RELEASE with empty o kern/140697 pf [pf] pf behaviour changes - must be documented o kern/137982 pf [pf] when pf can hit state limits, random IP failures o kern/136781 pf [pf] Packets appear to drop with pf scrub and if_bridg o kern/135948 pf [pf] [gre] pf not natting gre protocol o kern/135162 pf [pfsync] pfsync(4) not usable with GENERIC kernel o kern/134996 pf [pf] Anchor tables not included when pfctl(8) is run w o kern/133732 pf [pf] max-src-conn issue o kern/132769 pf [pf] [lor] 2 LOR's with pf task mtx / ifnet and rtent f kern/132176 pf [pf] pf stalls connection when using route-to [regress o conf/130381 pf [rc.d] [pf] [ip6] ipv6 not fully configured when pf st o kern/129861 pf [pf] [patch] Argument names reversed in pf_table.c:_co o kern/127920 pf [pf] ipv6 and synproxy don't play well together o conf/127814 pf [pf] The flush in pf_reload in /etc/rc.d/pf does not w o kern/127439 pf [pf] deadlock in pf f kern/127345 pf [pf] Problem with PF on FreeBSD7.0 [regression] o kern/127121 pf [pf] [patch] pf incorrect log priority o kern/127042 pf [pf] [patch] pf recursion panic if interface group is o kern/125467 pf [pf] pf keep state bug while handling sessions between s kern/124933 pf [pf] [ip6] pf does not support (drops) IPv6 fragmented o kern/124364 pf [pf] [panic] Kernel panic with pf + bridge o kern/122773 pf [pf] pf doesn't log uid or pid when configured to o kern/122014 pf [pf] [panic] FreeBSD 6.2 panic in pf o kern/120281 pf [pf] [request] lost returning packets to PF for a rdr o kern/120057 pf [pf] [patch] Allow proper settings of ALTQ_HFSC. The c o bin/118355 pf [pf] [patch] pfctl(8) help message options order false o kern/114567 pf [pf] [lor] pf_ioctl.c + if.c o kern/114095 pf [carp] carp+pf delay with high state limit s conf/110838 pf [pf] tagged parameter on nat not working on FreeBSD 5. o kern/103283 pf pfsync fails to sucessfully transfer some sessions o kern/103281 pf pfsync reports bulk update failures o kern/93825 pf [pf] pf reply-to doesn't work o sparc/93530 pf [pf] Incorrect checksums when using pf's route-to on s o kern/92949 pf [pf] PF + ALTQ problems with latency o bin/86635 pf [patch] pfctl(8): allow new page character (^L) in pf. o kern/82271 pf [pf] cbq scheduler cause bad latency 46 problems total. From owner-freebsd-pf@FreeBSD.ORG Wed May 18 11:30:05 2011 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79FDE106566B for ; Wed, 18 May 2011 11:30:05 +0000 (UTC) (envelope-from quentin.narvor@ensi-bourges.fr) Received: from ensi-bourges.fr (mail.ensi-bourges.fr [195.221.38.25]) by mx1.freebsd.org (Postfix) with ESMTP id 001B68FC08 for ; Wed, 18 May 2011 11:30:04 +0000 (UTC) Received: (qmail 20135 invoked from network); 18 May 2011 11:01:53 -0000 Received: from unknown (HELO webmail.ensi-bourges.fr) (Authenticated_user:quentin.narvor@[195.221.38.6]) (envelope-sender ) by mail.ensi-bourges.fr (qmail-ldap-1.03) with SMTP for ; 18 May 2011 11:01:53 -0000 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Wed, 18 May 2011 13:03:14 +0200 From: "quentin.narvor" To: Message-ID: <390946c3b25ae3d887574555a494cb42@ensi-bourges.fr> X-Sender: quentin.narvor@ensi-bourges.fr User-Agent: Roundcube Webmail/0.5.1 Subject: Large table issue X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 11:30:05 -0000 I am trying to detect problems on hosts in my network : I want to detect when a communication occurs with a compromised host. I have built a blacklist which holds near 2 millions ip (spam, malware.... hosts). But I can't load it into pf, I get this when I try : /etc/pf.conf:6: cannot define table bl: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded I suspect there is a memory limitation somewhere (in the kernel ??) which prevent me from loading the table but I am not very comfortable with kernel variables. I have already try modifying kern.maxssiz and kern.dflsiz without success. Any idea? From owner-freebsd-pf@FreeBSD.ORG Wed May 18 12:37:56 2011 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 189AE1065674 for ; Wed, 18 May 2011 12:37:56 +0000 (UTC) (envelope-from artemrts@ukr.net) Received: from ffe2.ukr.net (ffe2.ukr.net [195.214.192.44]) by mx1.freebsd.org (Postfix) with ESMTP id C22CE8FC12 for ; Wed, 18 May 2011 12:37:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=fsm; h=Date:Message-Id:From:To:References:In-Reply-To:Subject:Cc:Content-Type:Content-Transfer-Encoding:MIME-Version; bh=w/NHZkToT4MQx7OYcRjX901gKjr/qIxvP4hMkF8An5g=; b=Jy2OlZeJsgSCGG2B7/cvNYQtNMgOlrVadth4UOjhfOnSmgt9SKBCEQOuT+C1D6Z+StG/6JdjRR12Fs7rIdjOU5kN8mMmysEa4ssCH9Ci5ZyyMKWBmv0nNLDNSphfYAUHbqhAScIPpOB2y1q1VXXIgBS6wHgELwGlwFCxfhr2Nmk=; Received: from mail by ffe2.ukr.net with local ID 1QMg0f-0006p0-Iy ; Wed, 18 May 2011 15:37:53 +0300 MIME-Version: 1.0 Content-Disposition: inline Content-Transfer-Encoding: binary Content-Type: text/plain; charset="windows-1251" In-Reply-To: <390946c3b25ae3d887574555a494cb42@ensi-bourges.fr> References: <390946c3b25ae3d887574555a494cb42@ensi-bourges.fr> To: "quentin.narvor" From: =?WINDOWS-1251?B?wujy4Ovo6SDC6+Dk6Ozo8O7i6Pc=?= X-Mailer: freemail.ukr.net 4.0 X-Originating-Ip: [195.200.251.92] X-Browser: Mozilla/5.0 (Unix) Message-Id: Date: Wed, 18 May 2011 15:37:53 +0300 X-UkrNet-Flag: 1 Cc: freebsd-pf@freebsd.org Subject: Re: Large table issue X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 12:37:56 -0000 --- Original Message --- From: "quentin.narvor" To: freebsd-pf@freebsd.org Date: 18 May 2011, 14:03:14 Subject: Re: Large table issue > I am trying to detect problems on hosts in my network : I want to > detect when a communication occurs with a compromised host. > I have built a blacklist which holds near 2 millions ip (spam, > malware.... hosts). > > But I can't load it into pf, I get this when I try : > > /etc/pf.conf:6: cannot define table bl: Cannot allocate memory > pfctl: Syntax error in config file: pf rules not loaded > > I suspect there is a memory limitation somewhere (in the kernel ??) > which prevent me from loading the table but I am not very comfortable > with kernel variables. > I have already try modifying kern.maxssiz and kern.dflsiz without > success. > > Any idea? May be you should set this: set limit table-entries 2000000 From owner-freebsd-pf@FreeBSD.ORG Wed May 18 13:01:04 2011 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0CE8E106564A for ; Wed, 18 May 2011 13:01:04 +0000 (UTC) (envelope-from neamtu@gmail.com) Received: from mail-yi0-f54.google.com (mail-yi0-f54.google.com [209.85.218.54]) by mx1.freebsd.org (Postfix) with ESMTP id BE16C8FC08 for ; Wed, 18 May 2011 13:01:03 +0000 (UTC) Received: by yie12 with SMTP id 12so662935yie.13 for ; Wed, 18 May 2011 06:01:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=o+hq/WLbVs3Rjm4q507gKN9ijbFVZ1dB9r5e1umqgB0=; b=ZW9sFOqxL1Y8TznNyI6bK9ZMg+/d2t+JhNq8FF2TYsCx0Aa3JMZwr0lc3fSDm0EYl8 K3s/pvhTBlkgHFZcdkKBbv7F6oTZSM10r2bBZUnxtwiV36I/3j4dHP74bqBvOTQAyxuT xKY/o66uRY9M7U5CqJdAca0yYcVNoF1Xq3etk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=stfoqX3swLSrfrNtjoGpfHf/zpyoAoAwCSeTdXodTi66aT9elSu4wg9DnYFsXpBQAm vAC/CMlKaWiAmJB2x9goSikcTzP8Q5WKv7vHJkBnRWky0xKjNQbfdzJuI/T6Jf4vOAHy 3jxGQngvJI9kZQmaNQpXGHaKfEMWn0h4n6gnc= MIME-Version: 1.0 Received: by 10.91.92.8 with SMTP id u8mr1455288agl.15.1305722089361; Wed, 18 May 2011 05:34:49 -0700 (PDT) Received: by 10.90.70.18 with HTTP; Wed, 18 May 2011 05:34:49 -0700 (PDT) In-Reply-To: <390946c3b25ae3d887574555a494cb42@ensi-bourges.fr> References: <390946c3b25ae3d887574555a494cb42@ensi-bourges.fr> Date: Wed, 18 May 2011 15:34:49 +0300 Message-ID: From: =?ISO-8859-1?Q?Richard_Brend=F6rfer?= To: "quentin.narvor" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-pf@freebsd.org Subject: Re: Large table issue X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 13:01:04 -0000 Hi, try with *set limit table-entries number* in pf.vonf or split you table in 2 or 3 tables. On Wed, May 18, 2011 at 2:03 PM, quentin.narvor < quentin.narvor@ensi-bourges.fr> wrote: > I am trying to detect problems on hosts in my network : I want to detect > when a communication occurs with a compromised host. > I have built a blacklist which holds near 2 millions ip (spam, malware.... > hosts). > > But I can't load it into pf, I get this when I try : > > /etc/pf.conf:6: cannot define table bl: Cannot allocate memory > pfctl: Syntax error in config file: pf rules not loaded > > I suspect there is a memory limitation somewhere (in the kernel ??) which > prevent me from loading the table but I am not very comfortable with kernel > variables. > I have already try modifying kern.maxssiz and kern.dflsiz without success. > > Any idea? > _______________________________________________ > freebsd-pf@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-pf > To unsubscribe, send any mail to "freebsd-pf-unsubscribe@freebsd.org" > From owner-freebsd-pf@FreeBSD.ORG Wed May 18 13:01:08 2011 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B5C0C106564A for ; Wed, 18 May 2011 13:01:08 +0000 (UTC) (envelope-from quentin.narvor@ensi-bourges.fr) Received: from ensi-bourges.fr (mail.ensi-bourges.fr [195.221.38.25]) by mx1.freebsd.org (Postfix) with ESMTP id E31E38FC12 for ; Wed, 18 May 2011 13:01:07 +0000 (UTC) Received: (qmail 32496 invoked from network); 18 May 2011 12:59:36 -0000 Received: from unknown (HELO webmail.ensi-bourges.fr) (Authenticated_user:quentin.narvor@[195.221.38.6]) (envelope-sender ) by mail.ensi-bourges.fr (qmail-ldap-1.03) with SMTP for ; 18 May 2011 12:59:36 -0000 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Wed, 18 May 2011 15:00:57 +0200 From: "quentin.narvor" To: =?UTF-8?Q?Richard_Brend=C3=B6rfer?= In-Reply-To: References: <390946c3b25ae3d887574555a494cb42@ensi-bourges.fr> Message-ID: X-Sender: quentin.narvor@ensi-bourges.fr User-Agent: Roundcube Webmail/0.5.1 Cc: freebsd-pf@freebsd.org Subject: Re: Large table issue X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 13:01:08 -0000 On Wed, 18 May 2011 15:34:49 +0300, Richard Brendörfer wrote: > Hi, > try with _set limit table-entries number_ in pf.vonf or split you > table in 2 or 3 tables. > Hi, I forgot to say that I have already set this option to 3000000 in my pf.conf. I have tried to split the table in smaller pieces (~450000 entries in each table) but the command "pfctl -f /etc/pf.conf" gives me the same memory issue when loading the third table. I don't know the precise number but it seems that there is a limit near 1000000 entries for the sum of all tables, even with the limit table-entries set to 3000000. > On Wed, May 18, 2011 at 2:03 PM, quentin.narvor wrote: > >> I am trying to detect problems on hosts in my network : I want to >> detect when a communication occurs with a compromised host. >> I have built a blacklist which holds near 2 millions ip (spam, >> malware.... hosts). >> >> But I can't load it into pf, I get this when I try : >> >>     /etc/pf.conf:6: cannot define table bl: Cannot allocate >> memory >>     pfctl: Syntax error in config file: pf rules not loaded >> >> I suspect there is a memory limitation somewhere (in the kernel ??) >> which prevent me from loading the table but I am not very >> comfortable with kernel variables. >> I have already try modifying kern.maxssiz and kern.dflsiz without >> success. >> >> Any idea? >> _______________________________________________ >> freebsd-pf@freebsd.org [1] mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-pf [2] >> To unsubscribe, send any mail to >> "freebsd-pf-unsubscribe@freebsd.org [3]" > > > > Links: > ------ > [1] mailto:freebsd-pf@freebsd.org > [2] http://lists.freebsd.org/mailman/listinfo/freebsd-pf > [3] mailto:freebsd-pf-unsubscribe@freebsd.org > [4] mailto:quentin.narvor@ensi-bourges.fr From owner-freebsd-pf@FreeBSD.ORG Wed May 18 13:12:18 2011 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2BB35106564A for ; Wed, 18 May 2011 13:12:18 +0000 (UTC) (envelope-from neamtu@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id D82768FC12 for ; Wed, 18 May 2011 13:12:17 +0000 (UTC) Received: by gxk28 with SMTP id 28so666892gxk.13 for ; Wed, 18 May 2011 06:12:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=mW0z8zW3muvTGdPKP3EDJnGGR6rNHBcrtxElyHvK/Fg=; b=PAzdJULh0ve/p7+pVIHEb9yWxixkqxlr90HLa4v6/o3PA7eyW/jvjwJYrtraevYEjd fs4vmscmRUp4Lx2EorC3pB2Z2wsf+gm0q0bWzolhUSImxM8AS38zJ6FQ/cpX0KoH/fNe tPJG5x1DLNByuvz5hyJaaFJrQQ5qXFTK4tr3A= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=DyKjnsiHBO++seognXh7ugBTeI3f2kmfcBe4kGh+h80PiPNroVXUuVsVGrT1e50mWK QZLJ0hAW4D6FlkY5jJ4UbvhdJn3GioP4qc/Lt61C/GHYNrdS7vtdSIqXGn55tCNlp3Gh DqZ7MaY2y/HGrxHddqTqUCZfmPUHL0MfjQY7A= MIME-Version: 1.0 Received: by 10.91.163.4 with SMTP id q4mr1449076ago.96.1305724336963; Wed, 18 May 2011 06:12:16 -0700 (PDT) Received: by 10.90.70.18 with HTTP; Wed, 18 May 2011 06:12:16 -0700 (PDT) In-Reply-To: References: <390946c3b25ae3d887574555a494cb42@ensi-bourges.fr> Date: Wed, 18 May 2011 16:12:16 +0300 Message-ID: From: =?ISO-8859-1?Q?Richard_Brend=F6rfer?= To: "quentin.narvor" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-pf@freebsd.org Subject: Re: Large table issue X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 13:12:18 -0000 Look what I found: One side note: It might occur that PF states: "Cannot allocate memory", in which case the table is too big to fit in memory. The problem is that FreeBSD has a limit on the maximum size a program may b= e in memory. This is by default 524288 kB. One can obtain this and other limits by entering (values in kB): In sh: *# ulimit -a* In tcsh: *# limits* To resolv the problem, simply edit /boot/defaults/loader.conf. Find the following line under the "Kernel tunables" section: #kern.maxdsiz=3D"" # Set the max data size And change it into (values in bytes): kern.maxdsiz=3D"1073741824" # Set the max data size (IN BYTES) Changes made to /boot/defaults/loader.conf will be effective after rebooting. On Wed, May 18, 2011 at 4:00 PM, quentin.narvor < quentin.narvor@ensi-bourges.fr> wrote: > On Wed, 18 May 2011 15:34:49 +0300, Richard Brend=F6rfer wrote: > >> Hi, >> try with _set limit table-entries number_ in pf.vonf or split you >> table in 2 or 3 tables. >> >> Hi, > > I forgot to say that I have already set this option to 3000000 in my > pf.conf. > I have tried to split the table in smaller pieces (~450000 entries in eac= h > table) but the command "pfctl -f /etc/pf.conf" gives me the same memory > issue when loading the third table. > I don't know the precise number but it seems that there is a limit near > 1000000 entries for the sum of all tables, even with the limit table-entr= ies > set to 3000000. > > On Wed, May 18, 2011 at 2:03 PM, quentin.narvor wrote: >> >> I am trying to detect problems on hosts in my network : I want to >>> detect when a communication occurs with a compromised host. >>> I have built a blacklist which holds near 2 millions ip (spam, >>> malware.... hosts). >>> >>> But I can't load it into pf, I get this when I try : >>> >>> /etc/pf.conf:6: cannot define table bl: Cannot allocate >>> memory >>> pfctl: Syntax error in config file: pf rules not loaded >>> >>> I suspect there is a memory limitation somewhere (in the kernel ??) >>> which prevent me from loading the table but I am not very >>> comfortable with kernel variables. >>> I have already try modifying kern.maxssiz and kern.dflsiz without >>> success. >>> >>> Any idea? >>> _______________________________________________ >>> freebsd-pf@freebsd.org [1] mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-pf [2] >>> >>> To unsubscribe, send any mail to >>> "freebsd-pf-unsubscribe@freebsd.org [3]" >>> >> >> >> >> Links: >> ------ >> [1] mailto:freebsd-pf@freebsd.org >> [2] http://lists.freebsd.org/mailman/listinfo/freebsd-pf >> [3] mailto:freebsd-pf-unsubscribe@freebsd.org >> [4] mailto:quentin.narvor@ensi-bourges.fr >> > > From owner-freebsd-pf@FreeBSD.ORG Wed May 18 20:13:32 2011 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D624E106566B for ; Wed, 18 May 2011 20:13:32 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 80B098FC08 for ; Wed, 18 May 2011 20:13:32 +0000 (UTC) Received: by ywf7 with SMTP id 7so860248ywf.13 for ; Wed, 18 May 2011 13:13:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:sender:date:from:to:cc:subject:message-id :references:mime-version:content-type:content-disposition :in-reply-to:x-openpgp-key-id:x-openpgp-key-fingerprint :x-openpgp-key-url; bh=CIaOdOf+WG7Er6PlssInuU2kVVbpkzhW0TOgl4SOX6Q=; b=pWF06aYT1lFfe6W/YO2EGM0JVgXYYL7hqXmA9VyM2fmy9RzNjvQCEq47L5nQPK6Ag/ oJl5Lke0LxPIDYOXucm2VVibiDk+aNWKsiA2uS7cZoNW4aH1Le5nJFNN+etkMDIzB3xg pXZXZ4W8pgDUS+kpUoNfQtxYHAuLQ1RariZi8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:x-openpgp-key-id :x-openpgp-key-fingerprint:x-openpgp-key-url; b=xvPKXmgou0/vEbZeDHhC3qlCJCHcEAsxI9bsukBxseRZc7VkjPeUfUES5DuwYpm3kN 4Lw/lpA5FTSte9pMI5+x5ktDZwDs2X+d/ao9Jl67leUFT3znTEmym513S1BAPSQpZ9Ll RCtkmrsOfRk3M/428/G8fOFBG/WLkCIfucfIU= Received: by 10.150.9.29 with SMTP id 29mr1814707ybi.148.1305749611572; Wed, 18 May 2011 13:13:31 -0700 (PDT) Received: from DataIX.net (adsl-99-181-146-200.dsl.klmzmi.sbcglobal.net [99.181.146.200]) by mx.google.com with ESMTPS id f13sm878696ybi.18.2011.05.18.13.13.28 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 18 May 2011 13:13:29 -0700 (PDT) Sender: "J. Hellenthal" Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.4/8.14.4) with ESMTP id p4IKDPcp035922 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 18 May 2011 16:13:26 -0400 (EDT) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.4/8.14.4/Submit) id p4IKDOo5035921; Wed, 18 May 2011 16:13:25 -0400 (EDT) (envelope-from jhell@DataIX.net) Date: Wed, 18 May 2011 16:13:24 -0400 From: Jason Hellenthal To: "quentin.narvor" Message-ID: <20110518201324.GA35466@DataIX.net> References: <390946c3b25ae3d887574555a494cb42@ensi-bourges.fr> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="u3/rZRmxL6MmkK24" Content-Disposition: inline In-Reply-To: X-OpenPGP-Key-Id: 0x89D8547E X-OpenPGP-Key-Fingerprint: 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E X-OpenPGP-Key-URL: http://bit.ly/0x89D8547E Cc: freebsd-pf@freebsd.org Subject: Re: Large table issue X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 20:13:32 -0000 --u3/rZRmxL6MmkK24 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable quentin.narvor, On Wed, May 18, 2011 at 03:00:57PM +0200, quentin.narvor wrote: > On Wed, 18 May 2011 15:34:49 +0300, Richard Brend=F6rfer wrote: > > Hi, > > try with=A0_set limit table-entries number_ in pf.vonf=A0or split you > > table in 2 or 3 tables. > > > Hi, >=20 > I forgot to say that I have already set this option to 3000000 in my=20 > pf.conf. > I have tried to split the table in smaller pieces (~450000 entries in=20 > each table) but the command "pfctl -f /etc/pf.conf" gives me the same=20 > memory issue when loading the third table. > I don't know the precise number but it seems that there is a limit near= =20 > 1000000 entries for the sum of all tables, even with the limit=20 > table-entries set to 3000000. >=20 > > On Wed, May 18, 2011 at 2:03 PM, quentin.narvor wrote: > > > >> I am trying to detect problems on hosts in my network : I want to > >> detect when a communication occurs with a compromised host. > >> I have built a blacklist which holds near 2 millions ip (spam, > >> malware.... hosts). > >> > >> But I can't load it into pf, I get this when I try : > >> > >> =A0 =A0 /etc/pf.conf:6: cannot define table bl: Cannot allocate > >> memory > >> =A0 =A0 pfctl: Syntax error in config file: pf rules not loaded > >> > >> I suspect there is a memory limitation somewhere (in the kernel ??) > >> which prevent me from loading the table but I am not very > >> comfortable with kernel variables. > >> I have already try modifying kern.maxssiz and kern.dflsiz without > >> success. > >> > >> Any idea? If you are going to be dealing with tables this size it might be wise to write a filter to run your table file through and output the end result of multiple CIDR ranges that are going to take up a considerable less amount of space than what you have there. And if you hit a range where you dont want certain ip's blocked you can also use a !127.0.0.1/29 to cover a specfic range for example. Ive seen someone on the lists once post something about a script but don't remember off hand what that was so youll have to do some searching. Have fun! --=20 Regards, (jhell) Jason Hellenthal --u3/rZRmxL6MmkK24 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) Comment: http://bit.ly/0x89D8547E iQEcBAEBAgAGBQJN1ChkAAoJEJBXh4mJ2FR+yOcIAIovM0vfxolx4N+2jHByeOvq PtBXLHX/qK2QWwGQ3/ygVT4PN1zXbwISaNPq4zreMahckaWjrBn9cMozI46+Kvpm t0ig+Fn1zlRPd7xW4qO2qBNycQQ3ev0J5PS1bDnBzmxseM8FaY7wnKKOjLxdt61G xInK0HevMi7whwnzdV4XpG+gg6hLYhYN2Oo626Gp7VcESDL4qNn5JEoKdFu8NjeO gJiNFjNZxGBIGbVecZtLgkfUk0o0alpxts2P4QPhYHfG5w4Q/ahkwOTc3L5DCJpZ RYkUO2+zb2T68VEfDUn8vf1BzUzOEGLUuxkhcSJkMDO77jLIbCWFAsmQaN0ufos= =G3qO -----END PGP SIGNATURE----- --u3/rZRmxL6MmkK24--