Date: Mon, 05 Jan 2004 08:56:13 +0800 From: Ganbold <ganbold@micom.mng.net> To: Don Bowman <don@sandvine.com> Cc: freebsd-hackers@freebsd.org Subject: RE: ipfw2 problem Message-ID: <6.0.1.1.2.20040105085202.029b8820@202.179.0.80> In-Reply-To: <FE045D4D9F7AED4CBFF1B3B813C85337035E43C0@mail.sandvine.com > References: <FE045D4D9F7AED4CBFF1B3B813C85337035E43C0@mail.sandvine.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi,
How much memory does your machine have? I have never tried ipfw with -d option.
I'll try next time. Actually one_pass is already turned off in sysctl.conf
Any other recommendations? One suggested me to remove keep-state from
http filtering rules. Will it solve the problem?
Ganbold
At 01:41 AM 05.01.2004, you wrote:
>i have:
>
>sysctl net.inet.ip.fw.dyn_buckets=16384
>sysctl net.inet.ip.fw.dyn_syn_lifetime=5
>sysctl net.inet.ip.fw.dyn_max=32000
>sysctl net.inet.ip.fw.debug=0
>sysctl net.inet.ip.dummynet.max_chain_len=256
>sysctl net.inet.ip.dummynet.hash_size=1024
>sysctl net.inet.ip.fw.verbose_limit=1
>
>and am running ~3000 users with ~2 sessions each, stateful, with shaping.
>
>i wonder what you get if you run ipfw -d show when your error happens?
>
>i wonder if your shaper is getting full and droping the syn packets that
>setup the flow? maybe if you put the shaper rules @ the end and turned off
>one-pass?
>
> > -----Original Message-----
> > From: Ganbold [mailto:ganbold@micom.mng.net]
> > Sent: January 4, 2004 4:32 AM
> > To: freebsd-ipfw@freebsd.org
> > Cc: freebsd-hackers@freebsd.org
> > Subject: ipfw2 problem
> >
> >
> > Hi,
> >
> > I'm using FreeBSD 5.2-current machine for firewall. It is
> > configured as a
> > bridged ipfw2 firewall.
> > Also this machine works a a traffic shaper using ip dummynet features.
> > The machine has 2GHz Pentium 4 CPU and 128MB RAM and 3 Intel
> > Pro 100MB
> > cards. 2 cards are used
> > for bridging.
> > Everything works fine, except sometimes it seems to be
> > dropping some packets.
> > When I try to browse the web, sometimes it just shows error
> > page. This
> > situation happens during peak hours
> > So my guess is firewall drops packets and maybe the machine
> > needs more RAM.
> > Another guess is I'm using stateful features of ipfw2 and
> > when dynamic rule
> > count reaches maximum
> > it just drops packets waiting to be deleted some dynamic
> > rules. Am I right?
> > Can somebody explain
> > me what will happen when net.inet.ip.fw.dyn_count reaches
> > net.inet.ip.fw.dyn_max value?
> >
> > Also I tried to increase the maximum value up to 8192 but it
> > seems no result.
> >
> > # Added in sysctl.conf
> > net.inet.ip.fw.dyn_max=8192
> >
> > I attached my /etc/rc.firewall and /etc/sysctl.conf files.
> > Can somebody tell me where I did wrong in config files?
> > Should I increase
> > the RAM?
> > Or should I set smaller life time for dynamic rules?
> >
> > I hope somebody in this list point me to the right direction.
> >
> > Part of the /etc/rc.firewall
> > --------------------------------------------------------------
> > ---------------------------------------------------------------
> > ...
> > [Cc][Uu][Ss][Tt][Oo][Mm])
> >
> > ${fwcmd} -f flush
> > ${fwcmd} -f pipe flush
> >
> > # Things that we have kept state on before get to go through
> > in a hurry
> > ${fwcmd} add 10 check-state
> >
> > ${fwcmd} add 21 deny all from 10.0.0.0/8 to any via fxp0
> > ${fwcmd} add 23 deny all from 172.16.0.0/12 to any via fxp0
> > ${fwcmd} add 25 deny all from 192.168.0.0/16 to any via fxp0
> >
> > ${fwcmd} add 34 deny all from 127.0.0.0/8 to any in via fxp0
> >
> > ################### stop Welcia/Nachi ###########################
> > ${fwcmd} add 35 deny icmp from any to any iplen 92
> >
> > ####################### DUMMYNET config #########################
> >
> > ##################### 64KB #######################################
> > #
> > # selenge
> > ${fwcmd} pipe 41 config bw 64kbit/s
> > ${fwcmd} pipe 42 config bw 64kbit/s
> > ${fwcmd} add 62 pipe 41 all from 202.179.x.x/30 to any in via fxp1
> > ${fwcmd} add 63 pipe 42 all from any to 202.179.x.x/30 in via fxp0
> >
> > # khentii
> > ${fwcmd} pipe 43 config bw 64kbit/s
> > ${fwcmd} pipe 44 config bw 64kbit/s
> > ${fwcmd} add 64 pipe 43 all from 202.179.x.x/30 to any in via fxp1
> > ${fwcmd} add 65 pipe 44 all from any to 202.179.x.x/30 in via fxp0
> >
> > # arkhangai
> > ${fwcmd} pipe 45 config bw 64kbit/s
> > ${fwcmd} pipe 46 config bw 64kbit/s
> > ${fwcmd} add 66 pipe 45 all from 202.179.x.x/30 to any in via fxp1
> > ${fwcmd} add 67 pipe 46 all from any to 202.179.x.x/30 in via fxp0
> >
> > # traffic police
> > ${fwcmd} pipe 47 config bw 64kbit/s
> > ${fwcmd} pipe 48 config bw 64kbit/s
> > ${fwcmd} add 68 pipe 47 all from
> > 202.179.x.x/30,202.179.x.x/28 to any in
> > via fxp1
> > ${fwcmd} add 69 pipe 48 all from any to
> > 202.179.x.x/30,202.179.x.x/28 in
> > via fxp0
> >
> > ##################### 128KB #######################################
> > #
> > # glencore
> > ${fwcmd} pipe 49 config bw 128kbit/s
> > ${fwcmd} pipe 50 config bw 128kbit/s
> > ${fwcmd} add 70 pipe 49 all from
> > 202.179.x.x/29,202.179.x.x/30 to any in
> > via fxp1
> > ${fwcmd} add 71 pipe 50 all from any to
> > 202.179.x.x/29,202.179.x.x/30 in
> > via fxp0
> >
> > # ikh tenger
> > ${fwcmd} pipe 51 config bw 128kbit/s
> > ${fwcmd} pipe 52 config bw 128kbit/s
> > ${fwcmd} add 72 pipe 51 all from 202.179.x.x/29 to any in via fxp1
> > ${fwcmd} add 73 pipe 52 all from any to 202.179.x.x/29 in via fxp0
> >
> > # xas
> > ${fwcmd} pipe 53 config bw 128kbit/s
> > ${fwcmd} pipe 54 config bw 128kbit/s
> > ${fwcmd} add 74 pipe 53 all from
> > 202.179.x.x/29,202.179.x.x/30 to any in
> > via fxp1
> > ${fwcmd} add 75 pipe 54 all from any to
> > 202.179.x.x/29,202.179.x.x/30 in
> > via fxp0
> >
> >
> > ##################### 256KB #######################################
> > #mtc
> > ${fwcmd} pipe 55 config bw 256kbit/s
> > ${fwcmd} pipe 56 config bw 256kbit/s
> >
> > ${fwcmd} add 76 pipe 55 all from
> > 202.179.x.x/30,202.179.x.x/29 to any in
> > via fxp1
> > ${fwcmd} add 77 pipe 56 all from any to
> > 202.179.x.x/30,202.179.x.x/29 in
> > via fxp0
> >
> > #gtz
> > ${fwcmd} pipe 57 config bw 256kbit/s
> > ${fwcmd} pipe 58 config bw 256kbit/s
> >
> > ${fwcmd} add 78 pipe 57 all from 202.179.x.x/28 to any in via fxp1
> > ${fwcmd} add 79 pipe 58 all from any to 202.179.x.x/28 in via fxp0
> >
> > ######################### STANDARDS #########################
> > # Allow TCP through if setup succeeded
> > ${fwcmd} add 100 pass tcp from any to any established
> >
> > # Allowing connections through localhost.
> > ${fwcmd} add 300 pass all from any to any via lo0
> >
> > # pass ARP
> > ${fwcmd} add 301 allow layer2 mac-type arp
> >
> > # Allow the inside hosts to say anything they want
> > ${fwcmd} add pass tcp from any to any in via fxp1 setup keep-state
> > ${fwcmd} add pass udp from any to any in via fxp1 keep-state
> > ${fwcmd} add pass ip from any to any in via fxp1
> >
> > # Allowing SSH,web connection and LOG all incoming connections.
> > ${fwcmd} add pass tcp from any to any 22 in via fxp0 setup keep-state
> > ${fwcmd} add pass tcp from any to any 80,443 in via fxp0
> > setup keep-state
> >
> > # Allowing and LOG all INCOMING, outgoing FTP, telnet, SMTP,
> > POP3, ident,
> > imap conections.
> > ${fwcmd} add pass tcp from any to any 20-21,23,25,110,113,143 in via
> > fxp0 setup keep-state
> > ${fwcmd} add pass udp from any to any 20-21,23,25,110,113,143
> > in via fxp0
> > keep-state
> >
> > # Pass the "quarantine" range
> > ${fwcmd} add pass tcp from any to any 18198,18211,40000-65535
> > in via fxp0
> > setup keep-state
> > ${fwcmd} add pass udp from any to any 18198,18211,40000-65535
> > in via fxp0
> > keep-state
> >
> > # MSN, Yahoo ports
> > ${fwcmd} add pass tcp from any to any
> > 1863,2001-2120,6801,6891-6901,7801-7825 in via fxp0 setup keep-state
> > ${fwcmd} add pass udp from any to any
> > 1863,2001-2120,6801,6891-6901,7801-7825 in via fxp0 keep-state
> >
> > # additional h323,yahoo,remote admin,vnc ports
> > ${fwcmd} add pass tcp from any to any
> > 1719-1725,2082,5000-6000,8010,8100 in
> > via fxp0 setup keep-state
> > ${fwcmd} add pass udp from any to any
> > 1719-1725,2082,5000-6000,8010,8100 in
> > via fxp0 keep-state
> >
> > # Allowing mysql,Jabber,IRC,chat.
> > ${fwcmd} add pass tcp from any to any
> > 3306,4899,6155,6502,6667,8000 in via
> > fxp0 setup keep-state
> > ${fwcmd} add pass udp from any to any
> > 3306,4899,6155,6502,6667,8000 in via
> > fxp0 keep-state
> >
> > # allow radius
> > ${fwcmd} add pass tcp from any to any
> > 1645,1646,1812,1813,9000-9002 in via
> > fxp0 setup keep-state
> > ${fwcmd} add pass udp from any to any
> > 1645,1646,1812,1813,9000-9002 in via
> > fxp0 keep-state
> >
> > # additional eMule ports
> > ${fwcmd} add pass tcp from any to any
> > 2323,4242,4243,4661-4672,7700-7800 in
> > via fxp0 setup keep-state
> > ${fwcmd} add pass udp from any to any
> > 2323,4242,4243,4661-4672,7700-7800 in
> > via fxp0 keep-state
> >
> > # Allowing DNS lookups.
> > ${fwcmd} add pass tcp from any to any 53 in via fxp0 setup keep-state
> > ${fwcmd} add pass udp from any to any 53 in via fxp0 keep-state
> > ${fwcmd} add pass udp from any 53 to any in via fxp0 keep-state
> >
> > ${fwcmd} add pass icmp from 202.179.x.x/19 to any icmptypes
> > 0,3,4,8,11,12
> > ${fwcmd} add pass icmp from not 202.179.x.x/19 to
> > 202.179.x.x/19 icmptypes
> > 0,3,4,11,12
> >
> > # Allowing SOCKS,HTTP proxy to outside only
> > ${fwcmd} add pass tcp from 202.179.x.x/19 to any 1080,8080 in via
> > fxp0 setup keep-state
> > ${fwcmd} add pass udp from 202.179.x.x/19 to any 1080,8080 in
> > via fxp0
> > keep-state
> >
> > # Allow the bridge machine to say anything it wants
> > ${fwcmd} add pass tcp from 202.179.x.x to any setup keep-state
> > ${fwcmd} add pass udp from 202.179.x.x to any keep-state
> > ${fwcmd} add pass ip from 202.179.x.x to any
> >
> > ${fwcmd} add pass tcp from any to any in via fxp2 setup keep-state
> > ${fwcmd} add pass udp from any to any in via fxp2 keep-state
> > ${fwcmd} add pass ip from any to any in via fxp2
> >
> > # Allow NTP queries out in the world
> > ${fwcmd} add pass udp from any to any 123 in via fxp0 keep-state
> >
> > # allow multicast
> > ${fwcmd} add pass all from 202.179.x.x/19 to 224.0.0.0/4 via fxp0
> > ${fwcmd} add pass all from 224.0.0.0/4 to 202.179.x.x/19 via fxp0
> >
> > # Allowing OSPF
> > ${fwcmd} add pass ospf from any to any
> >
> > # Allowing GRE
> > ${fwcmd} add pass gre from any to any
> >
> > # Allowing IP fragments to pass through.
> > ${fwcmd} add 65001 pass all from any to any frag
> >
> > # Everything else is suspect
> > ${fwcmd} add drop log ip from any to any
> > ...
> > --------------------------------------------------------------
> > ---------------------------------------------------------------
> >
> > /etc/sysctl.conf file.
> > --------------------------------------------------------------
> > ---------------------------------------------------------------
> > net.link.ether.bridge_cfg=fxp0:0,fxp1:0
> > net.link.ether.bridge_ipfw=1
> > net.link.ether.bridge.enable=1
> >
> > net.inet.ip.fw.one_pass=0
> > security.bsd.see_other_uids=0
> > net.link.ether.inet.max_age=1200
> > kern.ipc.somaxconn=1024
> > net.inet.tcp.sendspace=32768
> > net.inet.tcp.recvspace=32768
> >
> > net.inet.ip.sourceroute=0
> > net.inet.ip.accept_sourceroute=0
> >
> > # Stop broadcast ECHO response
> > net.inet.icmp.bmcastecho=0
> >
> > # Stop other broadcast probes
> > net.inet.icmp.maskrepl=0
> >
> > net.inet.tcp.blackhole=2
> > net.inet.udp.blackhole=1
> >
> > net.inet.ip.fw.dyn_max=8192
> > net.inet.ip.fw.dyn_ack_lifetime=3600
> > net.inet.ip.fw.dyn_udp_lifetime=10
> > net.inet.ip.fw.dyn_buckets=1024
> >
> > --------------------------------------------------------------
> > ---------------------------------------------------------------
> >
> > tia,
> >
> > Ganbold
> >
> > _______________________________________________
> > freebsd-hackers@freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> > To unsubscribe, send any mail to
> > "freebsd-hackers-unsubscribe@freebsd.org"
> >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6.0.1.1.2.20040105085202.029b8820>