Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 8 May 2008 17:44:59 +0200
From:      "=?ISO-8859-1?Q?Ermal_Lu=E7i?=" <ermal.luci@gmail.com>
To:        freebsd-ipfw@freebsd.org
Cc:        482254ac@razorfever.net
Subject:   Re: Dummynet, gif, and ipsec
Message-ID:  <9a542da30805080844t395c8c81sd313fc2fd1780fcb@mail.gmail.com>

Next in thread | Raw E-Mail | Index | Archive | Help
> Date: Wed, 07 May 2008 21:41:23 -0400
> From: "Derek (freebsd-ipfw)" <482254ac@razorfever.net>
> Subject: Dummynet, gif, and ipsec
> To: freebsd-ipfw@freebsd.org
> Message-ID: <48225A43.3080909@razorfever.net>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Hello,
>
> I am trying to do some traffic shaping on my router using dummynet.  The
> router is one endpoint in an ipsec tunnel (configured as the handbook
> suggests).
>
> I'm running voip inside ipsec, so the goal here is to minimize the delay
> in transmitting the encapsulated voip packets.  I'd like to be able to
> classify other traffic inside ipsec as well.
>
> I currently have my outgoing pipe configured with 5 queues:
>
> pipe 10 config delay 0                  #outgoing pipe send
> queue 101 config pipe 10 weight 100     #highest priority
> queue 102 config pipe 10 weight 20
> queue 103 config pipe 10 weight 5
> queue 104 config pipe 10 weight 1       #lowest priority
>
> I have all of my outgoing ipsec queued into queue 101, with most of my
> other traffic hitting the other queues.
>
> I have set up additional queues to run over top of my
> gif interfaces:
>
> pipe 20 config delay 0                  #VPN outgoing pipe (gif*) send
> queue 201 config pipe 20 weight 100     #highest priority
> queue 202 config pipe 20 weight 20
> queue 203 config pipe 20 weight 5
> queue 204 config pipe 20 weight 1       #lowest priority
>
> As I'm running a tunnel, I have specific subnets that are communicating
> over the gif interfaces.
>
> If I simply allow the traffic, everything works fine:
>
> add 7200 allow ip from table(1) to table(1)
>
> If I try to queue traffic going out, packets disappear right after the
> queue:
>
> add 7020 queue 203 ip from table(1) to table(1) out via gif*
> add 7025 count ip from table(1) to table(1) out via gif*
> add 7200 allow ip from table(1) to table(1)
>
> I zero my stats, and let the traffic run for a little while.  When I
> show my stats again, there is a gross discrepancy in the counts at 7020,
> and 7025.  Packets that normally show up on the other end of the tunnel,
> don't.  If I remove the count, the packets still don't show up.
>
> If anyone can recognize why this is happening, or provide suggestions on
> how to troubleshoot it, I'd greatly appreciate it.
>
> If your are doing a similar approach successfully, I'd like to know.
>
> If you have alternative suggestions on how to shape traffic based on
> what is inside an ipsec tunnel, that is also appreciated.
>
> I tried using IPFW's tag command, and I think that is the best way to do
> what I want to do, period.  Tag the unencrypted data as it is going out
> the gif interface, and queue the IPSEC traffic based on the packet tag,
> but the tag disappears after IPSEC processing.
>
> Also, I'm aware of using the IP TOS flags for this, and I may have to go
> that route (based on the research I've done), but I want to know why
> what I'm currently is failing the way it is.  The TOS flags don't seem
> as flexible, as there are only a few classes that I can use, and there
> isn't any built-in way to modify these flags.  I'd have to use netgraph,
> or divert, and I'm not too keen on either of these approaches right now.
>

Well this is a patch to shape IPSec tunnels with ALTQ and FreeBSD 6.3
as you are running. It is another alternative to dummynet though it
have been tested with pf but should work with ipfw too since it knows
about ALTQ.
Hope it helps!

Ermal

Index: sys/net/if_enc.c
===================================================================
RCS file: /home/eri/FreeBSD/src/sys/net/if_enc.c,v
retrieving revision 1.5.2.2.2.1
diff -u -r1.5.2.2.2.1 if_enc.c
--- sys/net/if_enc.c	29 Dec 2007 17:29:11 -0000	1.5.2.2.2.1
+++ sys/net/if_enc.c	10 Feb 2008 01:19:41 -0000
@@ -48,6 +48,8 @@
 #include <net/bpf.h>
 #include <net/bpfdesc.h>

+#include <altq/if_altq.h>
+
 #include <netinet/in.h>
 #include <netinet/in_systm.h>
 #include <netinet/ip.h>
@@ -194,10 +196,12 @@
 }

 int
-ipsec_filter(struct mbuf **mp, int dir)
+ipsec_filter(struct mbuf **mp, struct secasindex *saidx, int dir)
 {
 	int error, i;
 	struct ip *ip;
+	struct m_tag *t;
+	struct altq_tag *atag;

 	KASSERT(encif != NULL, ("%s: encif is null", __func__));

@@ -267,6 +271,11 @@
 	if (error != 0)
 		goto bad;

+	if (saidx && (t = m_tag_find(*mp, PACKET_TAG_PF_QID, NULL)) != NULL) {
+		atag = (struct altq_tag *)(t + 1);
+		saidx->qid = atag->qid;
+	}
+
 	return (error);

 bad:
Index: sys/netipsec/ipsec.h
===================================================================
RCS file: /home/eri/FreeBSD/src/sys/netipsec/ipsec.h,v
retrieving revision 1.8.2.2
diff -u -r1.8.2.2 ipsec.h
--- sys/netipsec/ipsec.h	24 Jul 2006 23:20:59 -0000	1.8.2.2
+++ sys/netipsec/ipsec.h	8 Feb 2008 17:13:03 -0000
@@ -413,7 +413,7 @@
 extern	struct mbuf *m_makespace(struct mbuf *m0, int skip, int hlen, int *off);
 extern	caddr_t m_pad(struct mbuf *m, int n);
 extern	int m_striphdr(struct mbuf *m, int skip, int hlen);
-extern	int ipsec_filter(struct mbuf **, int);
+extern	int ipsec_filter(struct mbuf **, struct secasindex *, int);
 extern	void ipsec_bpf(struct mbuf *, struct secasvar *, int);
 #endif /* _KERNEL */

Index: sys/netipsec/ipsec_input.c
===================================================================
RCS file: /home/eri/FreeBSD/src/sys/netipsec/ipsec_input.c,v
retrieving revision 1.9.2.2
diff -u -r1.9.2.2 ipsec_input.c
--- sys/netipsec/ipsec_input.c	24 Jul 2006 23:20:59 -0000	1.9.2.2
+++ sys/netipsec/ipsec_input.c	8 Feb 2008 16:53:05 -0000
@@ -451,7 +451,7 @@
 	ipsec_bpf(m, sav, AF_INET);

 	if (prot != IPPROTO_IPIP)
-		if ((error = ipsec_filter(&m, 1)) != 0)
+		if ((error = ipsec_filter(&m, &sav->sah->saidx, 1)) != 0)
 			return (error);
 #endif

Index: sys/netipsec/ipsec_output.c
===================================================================
RCS file: /home/eri/FreeBSD/src/sys/netipsec/ipsec_output.c,v
retrieving revision 1.10.8.1
diff -u -r1.10.8.1 ipsec_output.c
--- sys/netipsec/ipsec_output.c	24 Jul 2006 23:20:59 -0000	1.10.8.1
+++ sys/netipsec/ipsec_output.c	9 Feb 2008 17:49:47 -0000
@@ -44,6 +44,9 @@
 #include <sys/syslog.h>

 #include <net/if.h>
+#ifdef DEV_ENC
+#include <altq/if_altq.h>
+#endif
 #include <net/route.h>

 #include <netinet/in.h>
@@ -160,6 +163,25 @@
 	}
 	key_sa_recordxfer(sav, m);		/* record data transfer */

+#ifdef DEV_ENC
+	/*
+         * Restore previous queue selected by the classifier for the
+         * packet.
+         */
+	if (saidx->qid) {
+		mtag = m_tag_get(PACKET_TAG_PF_QID, sizeof(struct altq_tag),
+			M_NOWAIT);
+        	if (mtag != NULL) { /* Safe to ignore */
+			struct altq_tag *atag;
+		        atag = (struct altq_tag *)(mtag + 1);
+	                atag->qid = saidx->qid;
+                	/* add hints for ecn */
+        	        atag->af = saidx->dst.sa.sa_family;
+	                atag->hdr = NULL; /* This should be safe! */
+                	m_tag_prepend(m, mtag);
+        	}
+	}
+#endif
 	/*
 	 * We're done with IPsec processing, transmit the packet using the
 	 * appropriate network protocol (IP or IPv6). SPD lookup will be
@@ -362,7 +384,7 @@

 #ifdef DEV_ENC
 	/* pass the mbuf to enc0 for packet filtering */
-	if ((error = ipsec_filter(&m, 2)) != 0)
+	if ((error = ipsec_filter(&m, &sav->sah->saidx, 2)) != 0)
 		goto bad;
 #endif

Index: sys/netipsec/keydb.h
===================================================================
RCS file: /home/eri/FreeBSD/src/sys/netipsec/keydb.h,v
retrieving revision 1.5
diff -u -r1.5 keydb.h
--- sys/netipsec/keydb.h	7 Jan 2005 01:45:46 -0000	1.5
+++ sys/netipsec/keydb.h	8 Feb 2008 17:12:01 -0000
@@ -58,6 +58,9 @@
 	u_int8_t mode;			/* mode of protocol, see ipsec.h */
 	u_int32_t reqid;		/* reqid id who owned this SA */
 					/* see IPSEC_MANUAL_REQID_MAX. */
+	u_int32_t	qid;		/* used for ALTQ shaping inside */
+					/* tunnel */
+					
 };

 /* Security Association Data Base */
Index: sys/netipsec/xform_ipip.c
===================================================================
RCS file: /home/eri/FreeBSD/src/sys/netipsec/xform_ipip.c,v
retrieving revision 1.11.2.2
diff -u -r1.11.2.2 xform_ipip.c
--- sys/netipsec/xform_ipip.c	24 Jul 2006 23:20:59 -0000	1.11.2.2
+++ sys/netipsec/xform_ipip.c	8 Feb 2008 16:54:41 -0000
@@ -348,7 +348,7 @@

 #ifdef DEV_ENC
 	/* pass the mbuf to enc0 for packet filtering */
-	if (ipsec_filter(&m, 1) != 0)
+	if (ipsec_filter(&m, NULL, 1) != 0)
 		return;
 #endif




> Best regards, and thanks!
> -- Derek
>
>
>
> More details about the machine:
>
> Running 6.3-RELEASE
>
> /etc/sysctl.conf:
> net.inet.ip.fw.one_pass=0
>
> Kernel:
> include SMP
> ident           GW2
>
> options         MPTABLE_FORCE_HTT
> options         NO_F00F_HACK
> options         DEVICE_POLLING
> options         AUTO_EOI_1
>
> options         IPSEC
> options         IPSEC_ESP
> options         IPSEC_FILTERGIF
>
> device          carp
> device          disc
>
> options         IPFIREWALL
> options         IPFIREWALL_VERBOSE
> options         IPFIREWALL_FORWARD
> options         IPDIVERT
> options         IPSTEALTH
> options         TCP_DROP_SYNFIN
> options         DUMMYNET
> options         ZERO_COPY_SOCKETS
> options         HZ=1000



Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?9a542da30805080844t395c8c81sd313fc2fd1780fcb>