Date: Mon, 02 Jan 2006 11:15:44 +0200 From: G Bryant <bsd@roamingsolutions.net> To: Dennis Olvany <dennisolvany@gmail.com>, freebsd-ipfw@freebsd.org Subject: Re: route selection and ipfw forwarding Message-ID: <43B8EF40.7020508@roamingsolutions.net> In-Reply-To: <43B8D510.2010908@gmail.com> References: <43B875FD.6000102@gmail.com> <43B8CC58.9020906@roamingsolutions.net> <43B8D510.2010908@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format.
--------------060604070009010508060108
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Content-Type: text/plain; charset="ISO-8859-1"
Dennis Olvany wrote:
It is possible to do load sharing, but not balancing in the true
sense of the word. You need to use ipfw's FWD rule to forward (push)
packets to the different gateways of the 4 WAN links, but it get's a
bit more complicated from there.
To do this using forwarding to four different gateways is simple enough,
but I'm not sure what results to expect by doing this to the same
gateway over four different links. I know of no way to influence the
decision and I am not sure how the routing algorithm would decide to
handle this situation.
You now also need to keep-state on each of the sessions, and will
have to check-state of all established outgoing packets to send them
out the same WAN link as the rest of that sessions packets.
I understand the importance of source-based routing with respect to
transit, but I'm a bit unclear on how dynamic rules relate to
forwarding. As long as all packets arrive at the destination and are
properly addressed, the host should not care how they got there. I'm not
sure if I know what you mean by session. I can't imagine an IPFW ruleset
that would provide stateful routing.
It's a dirty solution, but I have a working system with 2 WAN links.
You can also use IPFW set's to manage the percentage splits betweeen
the links. (or for failover).
Would you mind sharing your ipfw ruleset (ipfw list)?
My setup has 2 different external IP's with different gateway's. With
the different external IP's, the destination
host will reject packets coming from the "wrong" IP as they will not
match the current TCP session.
I also have web, ftp, email and vpn servers hosted on the inside which
complicates matters.
There's also a proxy server and bandwidth management for outgoing
connections.
What does your setup look like?
--------------060604070009010508060108
Content-Type: text/plain;
name="ipfw.ruleset"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="ipfw.ruleset"
#!/bin/sh
################ Start of IPFW rules file ###############################
# Flush out the list before we begin.
ipfw -q -f flush
ipfw -q add 10 allow tcp from any to me 22 in
ipfw -q add 12 allow tcp from me 22 to any out
# Set rules command prefix
cmd="ipfw -q add"
bwm="ipfw -q pipe"
skip="skipto 8000"
ext_if1="rl0" # public interface name of NIC
ext_if2="rl0"
int_if="vr0" # private interface name of NIC
ext_ip1="196.yy.yy.yy"
ext_ip2="196.xx.xx.xx"
ext_gw1="196.yy.yy.1"
ext_gw2="196.xx.xx.1"
lan="192.168.1.0/24"
# Setup the different Sets to be used for different connection options
ipfw -q set disable 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
# Initially only enable set 1 (and 2 and 12 when we have 2 WAN links)
ipfw -q set enable 1 2 5 10
# Specify which ip addresses get what bandwidth
# Can also tell this dhcp server to give certain addresses to selected mac
# addresses in file /usr/local/etc/dhcpd.conf
u512k="192.168.10.2" # Users given 512kb/s link
u256k="192.168.1.0/24{2-254}" # Users given 256kb/s link
u128k="192.168.10.2" # Users given 128kb/s link
u64k="192.168.10.2" # Users given 64kb/s link (default from dhcp)
#################################################################
# Check and drop packets that are appearing to come from
# the destination LAN i.e. a spoofed source ip address
$cmd 100 deny ip from any to any not antispoof in
#################################################################
# No restrictions on Loopback Interface
# Protect spoofing to localhost
$cmd allow ip from any to any via lo0
$cmd deny ip from any to 127.0.0.0/8
$cmd deny ip from 127.0.0.0/8 to any
################################################################
# Divert incoming http and ftp traffic to the proxy (squid and jftpgw)
$cmd set 5 fwd 192.168.1.1,3128 tcp from $lan to any 80 in via $int_if
#$cmd fwd 192.168.1.1,2370 tcp from $lan to any 21 via $int_if
#################################################################
# Allow the rest of the LAN traffic in and out
$cmd allow ip from any to any via $int_if
#################################################################
# Incoming traffic from internet - jump to incoming section
$cmd skipto 4000 ip from any to me in
################################################################
############## Bandwidth Management ############################
# Setup up pipes for each of the user groups
# Users with 512Kb / 256Kb access (out)
$cmd pipe 11 ip from $u512k to any out via $ext_if1
$bwm 11 config mask src-ip 0x000000ff bw 256Kbit/s queue 3KBytes
# Users with 256Kb / 128Kb access
$cmd pipe 21 ip from $u256k to any out via $ext_if1
$bwm 21 config mask src-ip 0x000000ff bw 128Kbit/s queue 3KBytes
# Users with 128Kb / 64Kb access
$cmd pipe 31 ip from $u128k to any out via $ext_if1
$bwm 31 config mask src-ip 0x000000ff bw 64Kbit/s queue 3KBytes
# Users with 64Kb / 56Kb access
$cmd pipe 41 ip from $u64k to any out via $ext_if1
$bwm 41 config mask src-ip 0x000000ff bw 56Kbit/s queue 12KBytes
#################################################################
# Interface facing Public Internet (Outbound Section)
#################################################################
#Deny outgoing windows connections
$cmd deny tcp from $lan to any 137,445 out
$cmd deny tcp from $lan 137,445 to any out
# Allow out access to my ISP's Domain name server.
# Get the IP addresses from /etc/resolv.conf file
#$cmd $skip UDP from any to { 196.7.0.138 or 196.28.86.2 or 196.28.86.3 or 196.25.1.1 } 53 out
$cmd $skip udp from any to any 53 out
# Allow this box out access to my ISP's DHCP server (or adsl router)
# to get it's network configuration settings (IP, DNS, etc).
$cmd $skip udp from me to any 67 out
# Allow skype connections out
# Allow ntp time server out
$cmd $skip UDP from any to any 80,443,123,1024-65535 out
$cmd $skip UDP from any 80,443,1024-65535 to any out
$cmd $skip tcp from any 1024-65535 to any 1024-65535 out
# Allow me to reach local routers from this box (tunneled via ssh)
$cmd allow tcp from me to 192.168.0.0/16 80 out
# Allow out www, ftp from me or from proxy out
$cmd $skip tcp from me to any 20,21,80 out
# Natting because of the 2 outgoing lines
# Allow out secure www function https over TLS SSL
# Allow out send & get email function (GMail uses ports 587, 995)
# Allow out MSN messenger
# Allow out Time, nntp news (i.e. news groups),
# SSH (secure FTP, Telnet, and SCP), whois
$cmd $skip tcp from any to any 443,25,110,587,995,1863,6891-6901,37,119,22,43 out
# Allow out regular ftp access (not via proxy)
$cmd $skip tcp from any to any 20,21,80 out
# Allow out ping
$cmd $skip icmp from $lan to any out icmptypes 3,8
$cmd allow icmp from me to any out icmptypes 3,8
$cmd allow icmp from me to 192.168.0.0/16 out
# Allow external ssh connection back out
$cmd $skip tcp from me 22 to any out
# Allow external web traffic from internal servers back out
$cmd $skip tcp from 192.168.1.8,192.168.1.2 80,443 to any out
# Allow external mail traffic from internal servers back out
$cmd $skip tcp from 192.168.1.3 25,110 to any out
# Allow external ftp, vpn connection traffic from internal servers back out
$cmd $skip tcp from 192.168.1.9 20,21,1723 to any out
$cmd $skip gre from 192.168.1.9 to any out
# Allow external remote desktop back out
$cmd $skip tcp from 192.168.1.2 3389 to any out
# temp allow 137
#$cmd $skip ip from any to any 137 out
#$cmd $skip ip from any 137 to any out
# Allow out FreeBSD (make install & CVSUP) functions
# Give user root "GOD" privileges.
$cmd allow ip from me to any out uid root
# Deny the rest out
$cmd deny log ip from any to any out
#################################################################
# Interface facing Public Internet (Inbound Section)
# Interrogate packets originating from the public Internet
# destine for this gateway server or the private network.
#################################################################
# check if packet is inbound and nat address if it is
# Double-nat traffic for incoming connections to determine which WAN connection
# it is coming from so as to send it out the same WAN link.
$cmd 4000 divert natd1 ip from any to $ext_ip1 80,443,20,21,25,110,3389 in
$cmd 4000 divert natd2 ip from any to $ext_ip2 80,443,20,21,25,110,3389 in
$cmd 4100 divert natd1 ip from any to $ext_ip1 in
$cmd 4100 divert natd2 ip from any to $ext_ip2 in
$cmd 4150 set 6 count log gre from any to any in
$cmd 4160 set 6 count log ip from any to any 1723 in
################################################################
############## Bandwidth Management ############################
# Users with 512Kb / 256Kb access (in)
$cmd 4200 pipe 10 ip from any to $u512k in via $ext_if1
$bwm 10 config mask dst-ip 0x000000ff bw 512Kbit/s queue 4KBytes
# Users with 256Kb / 128Kb access
$cmd pipe 20 ip from any to $u256k in via $ext_if1
$bwm 20 config mask dst-ip 0x000000ff bw 256Kbit/s queue 4KBytes
# Users with 128Kb / 64Kb access
$cmd pipe 30 ip from any to $u128k in via $ext_if1
$bwm 30 config mask dst-ip 0x000000ff bw 128Kbit/s queue 4KBytes
# Users with 64Kb / 56Kb access
$cmd pipe 40 ip from any to $u64k in via $ext_if1
$bwm 40 config mask dst-ip 0x000000ff bw 64Kbit/s queue 14KBytes
# Deny all inbound traffic from non-routable reserved address spaces
#$cmd 300 deny all from 192.168.0.0/16 to any in via $ext_if1 #RFC 1918 private IP
$cmd deny all from 172.16.0.0/12,10.0.0.0/8,0.0.0.0/8,169.254.0.0/16,192.0.2.0/24,204.152.64.0/23,224.0.0.0/3 to any in #RFC 1918 private IP
#RFC 1918 private IP #DHCP auto-config #reserved for docs #Sun cluster #Class D & E multicast
# Deny RIP (Router protocol) packets
$cmd deny udp from any to any 520 in
# Deny ident
# Deny all Netbios service. 137=name, 138=datagram, 139=session
# Netbios is MS/Windows sharing services.
# Block MS/Windows hosts2 name server requests 81
$cmd deny all from any to any 113,137,138,139,81 in
#$cmd deny all from any to any 113,138,139,81 in
#$cmd allow all from any to any 137 in
#$cmd allow all from any 137 to any in
# Allow traffic in from ISP's DHCP server. This rule must contain
# the IP address of your ISP's DHCP server as it's the only
# authorized source to send this packet type.
$cmd allow udp from $ext_gw1,$ext_gw2 to any 68,1900,5678 in
# Allow dns lookups back in
$cmd allow udp from any 53,67 to $lan in
$cmd allow udp from any 53,67 to me in
# Allow skype connections in
$cmd allow udp from any 80,123,443,1024-655353 to $lan in
$cmd allow udp from any to $lan 80,443,1024-655353 in
$cmd deny log udp from any to any in # Deny the rest
$cmd allow tcp from any 1024-65535 to $lan 1024-65535 in
# Allow in standard www function because I have Apache server - or is there an internal webserver?
# Allow Webmin connections from close-by
$cmd allow tcp from { 192.168.8.0/24 or 192.168.0.0/24 } to me 10000,80 in
#$cmd allow tcp from any to 192.168.1.8,192.168.1.9,192.168.1.2 80,443 in
# Allow outgoing ftp, web traffic (via proxy) back in
$cmd allow tcp from any 80 to me in
# Allow in regular ftp, http access if proxy is off
$cmd allow tcp from any 20,21,80 to $lan in
# Allow in secure www function https over TLS SSL
# Allow in send & get email function (GMail uses ports 587, 995)
# Allow in MSN messenger
# Allow in Time, nntp news (i.e. news groups),
# SSH (secure FTP, Telnet, and SCP), whois
$cmd allow tcp from any 443,25,110,587,995,1863,6891-6901,37,119,22,43 to any in
# Allow external web traffic in to internal servers
$cmd allow tcp from any to 192.168.1.8,192.168.1.2 80,443 in
# Allow external mail traffic in to internal servers
$cmd allow tcp from any to 192.168.1.3 25,110 in
# Allow external ftp, vpn connection traffic in to internal servers
$cmd allow tcp from any to 192.168.1.9 20,21,1723 in
$cmd allow gre from any to 192.168.1.9 in
# Allow in secure FTP, Telnet, and SCP from public Internet
$cmd allow tcp from { 192.168.0.0/24 or $lan or 192.168.8.0/24 } to me 22 in setup limit src-addr 3
$cmd allow tcp from any to me 22 in setup limit src-addr 2
# Deny the rest to me
$cmd deny log tcp from any to me in
#Allow in ICMP (ping) from public networks close by only.
$cmd allow icmp from 196.7.0.138,196.25.1.1,196.4.160.7 to me in icmptypes 0,3,11
$cmd allow icmp from any to $lan in icmptypes 0,3,11
$cmd allow icmp from any to me in icmptypes 0,3,11
$cmd allow icmp from 192.168.0.0/16 to me in
#Deny the rest icmp
$cmd deny icmp from any to any in
# Reject & Log all unauthorized incoming connections from the public Internet (/var/log/security)
$cmd deny log all from any to any in
### *********************************************************
# This is skipto location for outbound (stateful) rules
$cmd 8000 skipto 9000 tcp from any to any out setup
$cmd 8010 skipto 8900 tcp from 192.168.1.2,192.168.1.3,192.168.1.8,192.168.1.9 20,21,25,80,110,443,1723,3389 to any out
$cmd 8020 skipto 8900 udp from 192.168.1.9 1723 to any out
$cmd 8030 skipto 9000 udp from any to any out
$cmd 8040 skipto 9000 icmp from any to any out
$cmd 8050 skipto 8600 tcp from $ext_ip1,$ext_ip2 22 to any out
$cmd 8100 tee natd2 ip from any to any out
$cmd 8200 skipto 8400 ip from $ext_ip2 to any out
$cmd 8300 divert natd1 ip from any to any out
$cmd 8400 check-state
$cmd 8500 fwd $ext_gw1 gre from $ext_ip1 to any out
$cmd 8500 fwd $ext_gw2 gre from $ext_ip2 to any out
$cmd 8600 fwd $ext_gw1 tcp from $ext_ip1 22 to any out
$cmd 8600 fwd $ext_gw2 tcp from $ext_ip2 22 to any out
$cmd 8700 deny ip from any to any out
# Reply's from internal servers to external requests
$cmd 8900 tee natd1 ip from any 1723 to any out
$cmd 8902 skipto 8960 ip from $ext_ip1 1723 to any out
$cmd 8904 divert natd2 ip from any 1723 to any out
$cmd 8906 skipto 8990 ip from $ext_ip2 1723 to any out
$cmd 8910 divert natd1 ip from any to any out
$cmd 8920 skipto 8950 ip from $ext_ip1 8025,8011,8086,8087,8020,8021,3390 to any out
$cmd 8930 skipto 8980 ip from $ext_ip1 8026,8012,8088,8089,8022,8023,3391 to any out
$cmd 8940 deny log ip from any to any out
$cmd 8950 divert natd1 ip from any to any out
$cmd 8960 fwd $ext_gw1 ip from $ext_ip1 to any out
$cmd 8970 deny log ip from any to any out
$cmd 8980 divert natd2 ip from any to any out
$cmd 8990 fwd $ext_gw2 ip from $ext_ip2 to any out
$cmd 8999 deny log ip from any to any out
# skipto location for new outgoing connections
$cmd 9000 set 10 skipto 9500 tcp from 192.168.1.3 to any 25,110 out
$cmd 9000 set 10 skipto 9200 tcp from any to any 443,1863 out
$cmd 9000 set 10 skipto 9200 udp from any to any 80,443,1024-65535 out
$cmd 9000 set 10 skipto 9200 udp from any 80,443,1024-65535 to any out
$cmd 9000 set 10 skipto 9200 ip from any to any out
$cmd 9000 set 12 skipto 9500 tcp from 192.168.1.3 to any 25,110 out
$cmd 9000 set 12 skipto 9200 tcp from any to any 443,1863 out
$cmd 9000 set 12 skipto 9200 udp from any to any 80,443,1024-65535 out
$cmd 9000 set 12 skipto 9200 udp from any 80,443,1024-65535 to any out
$cmd 9000 set 12 prob 0.2 skipto 9500 ip from any to any out
$cmd 9000 set 13 skipto 9500 tcp from 192.168.1.3 to any 25,110 out
$cmd 9000 set 13 skipto 9200 tcp from any to any 443,1863 out
$cmd 9000 set 13 skipto 9200 udp from any to any 80,443,1024-65535 out
$cmd 9000 set 13 skipto 9200 udp from any 80,443,1024-65535 to any out
$cmd 9000 set 13 prob 0.3 skipto 9500 ip from any to any out
$cmd 9000 set 14 skipto 9500 tcp from 192.168.1.3 to any 25,110 out
$cmd 9000 set 14 skipto 9200 tcp from any to any 443,1863 out
$cmd 9000 set 14 skipto 9200 udp from any to any 80,443,1024-65535 out
$cmd 9000 set 14 skipto 9200 udp from any 80,443,1024-65535 to any out
$cmd 9000 set 14 prob 0.4 skipto 9500 ip from any to any out
$cmd 9000 set 15 skipto 9500 tcp from 192.168.1.3 to any 25,110 out
$cmd 9000 set 15 skipto 9200 tcp from any to any 443,1863 out
$cmd 9000 set 15 skipto 9200 udp from any to any 80,443,1024-65535 out
$cmd 9000 set 15 skipto 9200 udp from any 80,443,1024-65535 to any out
$cmd 9000 set 15 prob 0.5 skipto 9500 ip from any to any out
$cmd 9000 set 16 skipto 9500 tcp from 192.168.1.3 to any 25,110 out
$cmd 9000 set 16 skipto 9200 tcp from any to any 443,1863 out
$cmd 9000 set 16 skipto 9200 udp from any to any 80,443,1024-65535 out
$cmd 9000 set 16 skipto 9200 udp from any 80,443,1024-65535 to any out
$cmd 9000 set 16 prob 0.6 skipto 9500 ip from any to any out
$cmd 9000 set 17 skipto 9500 tcp from 192.168.1.3 to any 25,110 out
$cmd 9000 set 17 skipto 9200 tcp from any to any 443,1863 out
$cmd 9000 set 17 skipto 9200 udp from any to any 80,443,1024-65535 out
$cmd 9000 set 17 skipto 9200 udp from any 80,443,1024-65535 to any out
$cmd 9000 set 17 prob 0.7 skipto 9500 ip from any to any out
$cmd 9000 set 18 skipto 9500 tcp from 192.168.1.3 to any 25,110 out
$cmd 9000 set 18 skipto 9200 tcp from any to any 443,1863 out
$cmd 9000 set 18 skipto 9200 udp from any to any 80,443,1024-65535 out
$cmd 9000 set 18 skipto 9200 udp from any 80,443,1024-65535 to any out
$cmd 9000 set 18 prob 0.8 skipto 9500 ip from any to any out
$cmd 9000 set 20 skipto 9500 tcp from 192.168.1.3 to any 25,110 out
$cmd 9000 set 20 skipto 9200 tcp from any to any 443,1863 out
$cmd 9000 set 20 skipto 9200 udp from any to any 80,443,1024-65535 out
$cmd 9000 set 20 skipto 9200 udp from any 80,443,1024-65535 to any out
$cmd 9000 set 20 skipto 9500 ip from any to any out
# testing diginet line out
#$cmd 9000 set 25 skipto 9500 tcp from any to ongers.net out
$cmd 9000 set 25 skipto 9500 tcp from 192.168.1.3 to any 25,110 out
$cmd 9200 set 1 divert natd1 ip from any to any out
$cmd 9300 set 1 fwd $ext_gw1 ip from any to any out keep-state
$cmd 9500 set 2 divert natd2 ip from any to any out
$cmd 9600 set 2 fwd $ext_gw2 ip from any to any out keep-state
# Everything else is denied by default
# deny and log all packets that fell through to see what they are
$cmd 9999 deny log all from any to any
ipfw -q delete 10
ipfw -q delete 12
################ End of IPFW rules file ###############################
--------------060604070009010508060108--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?43B8EF40.7020508>