Date: Thu, 8 May 2008 17:44:59 +0200 From: "=?ISO-8859-1?Q?Ermal_Lu=E7i?=" <ermal.luci@gmail.com> To: freebsd-ipfw@freebsd.org Cc: 482254ac@razorfever.net Subject: Re: Dummynet, gif, and ipsec Message-ID: <9a542da30805080844t395c8c81sd313fc2fd1780fcb@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
> Date: Wed, 07 May 2008 21:41:23 -0400
> From: "Derek (freebsd-ipfw)" <482254ac@razorfever.net>
> Subject: Dummynet, gif, and ipsec
> To: freebsd-ipfw@freebsd.org
> Message-ID: <48225A43.3080909@razorfever.net>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Hello,
>
> I am trying to do some traffic shaping on my router using dummynet. The
> router is one endpoint in an ipsec tunnel (configured as the handbook
> suggests).
>
> I'm running voip inside ipsec, so the goal here is to minimize the delay
> in transmitting the encapsulated voip packets. I'd like to be able to
> classify other traffic inside ipsec as well.
>
> I currently have my outgoing pipe configured with 5 queues:
>
> pipe 10 config delay 0 #outgoing pipe send
> queue 101 config pipe 10 weight 100 #highest priority
> queue 102 config pipe 10 weight 20
> queue 103 config pipe 10 weight 5
> queue 104 config pipe 10 weight 1 #lowest priority
>
> I have all of my outgoing ipsec queued into queue 101, with most of my
> other traffic hitting the other queues.
>
> I have set up additional queues to run over top of my
> gif interfaces:
>
> pipe 20 config delay 0 #VPN outgoing pipe (gif*) send
> queue 201 config pipe 20 weight 100 #highest priority
> queue 202 config pipe 20 weight 20
> queue 203 config pipe 20 weight 5
> queue 204 config pipe 20 weight 1 #lowest priority
>
> As I'm running a tunnel, I have specific subnets that are communicating
> over the gif interfaces.
>
> If I simply allow the traffic, everything works fine:
>
> add 7200 allow ip from table(1) to table(1)
>
> If I try to queue traffic going out, packets disappear right after the
> queue:
>
> add 7020 queue 203 ip from table(1) to table(1) out via gif*
> add 7025 count ip from table(1) to table(1) out via gif*
> add 7200 allow ip from table(1) to table(1)
>
> I zero my stats, and let the traffic run for a little while. When I
> show my stats again, there is a gross discrepancy in the counts at 7020,
> and 7025. Packets that normally show up on the other end of the tunnel,
> don't. If I remove the count, the packets still don't show up.
>
> If anyone can recognize why this is happening, or provide suggestions on
> how to troubleshoot it, I'd greatly appreciate it.
>
> If your are doing a similar approach successfully, I'd like to know.
>
> If you have alternative suggestions on how to shape traffic based on
> what is inside an ipsec tunnel, that is also appreciated.
>
> I tried using IPFW's tag command, and I think that is the best way to do
> what I want to do, period. Tag the unencrypted data as it is going out
> the gif interface, and queue the IPSEC traffic based on the packet tag,
> but the tag disappears after IPSEC processing.
>
> Also, I'm aware of using the IP TOS flags for this, and I may have to go
> that route (based on the research I've done), but I want to know why
> what I'm currently is failing the way it is. The TOS flags don't seem
> as flexible, as there are only a few classes that I can use, and there
> isn't any built-in way to modify these flags. I'd have to use netgraph,
> or divert, and I'm not too keen on either of these approaches right now.
>
Well this is a patch to shape IPSec tunnels with ALTQ and FreeBSD 6.3
as you are running. It is another alternative to dummynet though it
have been tested with pf but should work with ipfw too since it knows
about ALTQ.
Hope it helps!
Ermal
Index: sys/net/if_enc.c
===================================================================
RCS file: /home/eri/FreeBSD/src/sys/net/if_enc.c,v
retrieving revision 1.5.2.2.2.1
diff -u -r1.5.2.2.2.1 if_enc.c
--- sys/net/if_enc.c 29 Dec 2007 17:29:11 -0000 1.5.2.2.2.1
+++ sys/net/if_enc.c 10 Feb 2008 01:19:41 -0000
@@ -48,6 +48,8 @@
#include <net/bpf.h>
#include <net/bpfdesc.h>
+#include <altq/if_altq.h>
+
#include <netinet/in.h>
#include <netinet/in_systm.h>
#include <netinet/ip.h>
@@ -194,10 +196,12 @@
}
int
-ipsec_filter(struct mbuf **mp, int dir)
+ipsec_filter(struct mbuf **mp, struct secasindex *saidx, int dir)
{
int error, i;
struct ip *ip;
+ struct m_tag *t;
+ struct altq_tag *atag;
KASSERT(encif != NULL, ("%s: encif is null", __func__));
@@ -267,6 +271,11 @@
if (error != 0)
goto bad;
+ if (saidx && (t = m_tag_find(*mp, PACKET_TAG_PF_QID, NULL)) != NULL) {
+ atag = (struct altq_tag *)(t + 1);
+ saidx->qid = atag->qid;
+ }
+
return (error);
bad:
Index: sys/netipsec/ipsec.h
===================================================================
RCS file: /home/eri/FreeBSD/src/sys/netipsec/ipsec.h,v
retrieving revision 1.8.2.2
diff -u -r1.8.2.2 ipsec.h
--- sys/netipsec/ipsec.h 24 Jul 2006 23:20:59 -0000 1.8.2.2
+++ sys/netipsec/ipsec.h 8 Feb 2008 17:13:03 -0000
@@ -413,7 +413,7 @@
extern struct mbuf *m_makespace(struct mbuf *m0, int skip, int hlen, int *off);
extern caddr_t m_pad(struct mbuf *m, int n);
extern int m_striphdr(struct mbuf *m, int skip, int hlen);
-extern int ipsec_filter(struct mbuf **, int);
+extern int ipsec_filter(struct mbuf **, struct secasindex *, int);
extern void ipsec_bpf(struct mbuf *, struct secasvar *, int);
#endif /* _KERNEL */
Index: sys/netipsec/ipsec_input.c
===================================================================
RCS file: /home/eri/FreeBSD/src/sys/netipsec/ipsec_input.c,v
retrieving revision 1.9.2.2
diff -u -r1.9.2.2 ipsec_input.c
--- sys/netipsec/ipsec_input.c 24 Jul 2006 23:20:59 -0000 1.9.2.2
+++ sys/netipsec/ipsec_input.c 8 Feb 2008 16:53:05 -0000
@@ -451,7 +451,7 @@
ipsec_bpf(m, sav, AF_INET);
if (prot != IPPROTO_IPIP)
- if ((error = ipsec_filter(&m, 1)) != 0)
+ if ((error = ipsec_filter(&m, &sav->sah->saidx, 1)) != 0)
return (error);
#endif
Index: sys/netipsec/ipsec_output.c
===================================================================
RCS file: /home/eri/FreeBSD/src/sys/netipsec/ipsec_output.c,v
retrieving revision 1.10.8.1
diff -u -r1.10.8.1 ipsec_output.c
--- sys/netipsec/ipsec_output.c 24 Jul 2006 23:20:59 -0000 1.10.8.1
+++ sys/netipsec/ipsec_output.c 9 Feb 2008 17:49:47 -0000
@@ -44,6 +44,9 @@
#include <sys/syslog.h>
#include <net/if.h>
+#ifdef DEV_ENC
+#include <altq/if_altq.h>
+#endif
#include <net/route.h>
#include <netinet/in.h>
@@ -160,6 +163,25 @@
}
key_sa_recordxfer(sav, m); /* record data transfer */
+#ifdef DEV_ENC
+ /*
+ * Restore previous queue selected by the classifier for the
+ * packet.
+ */
+ if (saidx->qid) {
+ mtag = m_tag_get(PACKET_TAG_PF_QID, sizeof(struct altq_tag),
+ M_NOWAIT);
+ if (mtag != NULL) { /* Safe to ignore */
+ struct altq_tag *atag;
+ atag = (struct altq_tag *)(mtag + 1);
+ atag->qid = saidx->qid;
+ /* add hints for ecn */
+ atag->af = saidx->dst.sa.sa_family;
+ atag->hdr = NULL; /* This should be safe! */
+ m_tag_prepend(m, mtag);
+ }
+ }
+#endif
/*
* We're done with IPsec processing, transmit the packet using the
* appropriate network protocol (IP or IPv6). SPD lookup will be
@@ -362,7 +384,7 @@
#ifdef DEV_ENC
/* pass the mbuf to enc0 for packet filtering */
- if ((error = ipsec_filter(&m, 2)) != 0)
+ if ((error = ipsec_filter(&m, &sav->sah->saidx, 2)) != 0)
goto bad;
#endif
Index: sys/netipsec/keydb.h
===================================================================
RCS file: /home/eri/FreeBSD/src/sys/netipsec/keydb.h,v
retrieving revision 1.5
diff -u -r1.5 keydb.h
--- sys/netipsec/keydb.h 7 Jan 2005 01:45:46 -0000 1.5
+++ sys/netipsec/keydb.h 8 Feb 2008 17:12:01 -0000
@@ -58,6 +58,9 @@
u_int8_t mode; /* mode of protocol, see ipsec.h */
u_int32_t reqid; /* reqid id who owned this SA */
/* see IPSEC_MANUAL_REQID_MAX. */
+ u_int32_t qid; /* used for ALTQ shaping inside */
+ /* tunnel */
+
};
/* Security Association Data Base */
Index: sys/netipsec/xform_ipip.c
===================================================================
RCS file: /home/eri/FreeBSD/src/sys/netipsec/xform_ipip.c,v
retrieving revision 1.11.2.2
diff -u -r1.11.2.2 xform_ipip.c
--- sys/netipsec/xform_ipip.c 24 Jul 2006 23:20:59 -0000 1.11.2.2
+++ sys/netipsec/xform_ipip.c 8 Feb 2008 16:54:41 -0000
@@ -348,7 +348,7 @@
#ifdef DEV_ENC
/* pass the mbuf to enc0 for packet filtering */
- if (ipsec_filter(&m, 1) != 0)
+ if (ipsec_filter(&m, NULL, 1) != 0)
return;
#endif
> Best regards, and thanks!
> -- Derek
>
>
>
> More details about the machine:
>
> Running 6.3-RELEASE
>
> /etc/sysctl.conf:
> net.inet.ip.fw.one_pass=0
>
> Kernel:
> include SMP
> ident GW2
>
> options MPTABLE_FORCE_HTT
> options NO_F00F_HACK
> options DEVICE_POLLING
> options AUTO_EOI_1
>
> options IPSEC
> options IPSEC_ESP
> options IPSEC_FILTERGIF
>
> device carp
> device disc
>
> options IPFIREWALL
> options IPFIREWALL_VERBOSE
> options IPFIREWALL_FORWARD
> options IPDIVERT
> options IPSTEALTH
> options TCP_DROP_SYNFIN
> options DUMMYNET
> options ZERO_COPY_SOCKETS
> options HZ=1000
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9a542da30805080844t395c8c81sd313fc2fd1780fcb>