Date: Sat, 16 Jul 2005 18:02:19 +0300 From: "Chris Dionissopoulos" <dionch@freemail.gr> To: <freebsd-ipfw@freebsd.org>, <freebsd-net@freebsd.org> Subject: Traffic quota features in IPFW Message-ID: <001c01c58a17$5dbe4a40$0100000a@R3B>
next in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format. ------=_NextPart_000_0019_01C58A30.81E63C20 Content-Type: text/plain; format=flowed; charset="windows-1253"; reply-type=original Content-Transfer-Encoding: 7bit Hi ppl, ( and sorry for cross posting) I review Andrey's Elsukov patch for adding "bound" support in ipfw, and i decide to push a little forward this feature. You can see the whole picture in there: http://www.freebsd.org/cgi/query-pr.cgi?pr=80642 and there: http://butcher.heavennet.ru/ In my patch, 3 new options are added: 1. "below <VALUE>" (which is the same option as Andrey's "bound" option, I just rename it) 2. "above <VALUE>" which is the oposite option of "below". Match rules when the counter is above <value> 3. "check-quota" (which is the same option as Andrey's "check-bound" , but now applies to both "above" and "below" options). Notes: 1. Patch is against releng_6. 2. I also include a more compicated example which is (IMHO) a complete traffic quota+shaping solution for a small (or not so small) ISP. 3. For installation, follow the instructions Adrey publish in his webspace: http://butcher.heavennet.ru/ 4. Patch doesn't breaks ipfw ABI (today) , because adds new options at the end of list. If you apply this patch in a month or so, I cannot guarantee success. 5. Please test, and send me your feedbacks. I 'll be happy if you find usefull these features and if any developer commits this patch in current or releng_6 branch. Chris. ____________________________________________________________________ http://www.freemail.gr - δωρεάν υπηρεσία ηλεκτρονικού ταχυδρομείου. http://www.freemail.gr - free email service for the Greek-speaking. ------=_NextPart_000_0019_01C58A30.81E63C20 Content-Type: application/octet-stream;name="releng6_ipfw_quota.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment;filename="releng6_ipfw_quota.patch" --- sys/netinet/ip_fw.h.orig Sat Jul 16 14:55:58 2005=0A= +++ sys/netinet/ip_fw.h Sat Jul 16 15:08:37 2005=0A= @@ -154,6 +154,13 @@=0A= O_NGTEE, /* copy to ng_ipfw */=0A= =0A= O_IP4,=0A= + =0A= + /*=0A= + * Traffic quota options=0A= + */=0A= + O_QBELOW, /* u64 =3D uplimit in bytes */=0A= + O_QABOVE, /* u64 =3D downlimit in bytes */=0A= + O_CHECK_QUOTA, /* u16 =3D rule number */=0A= =0A= O_LAST_OPCODE /* not an opcode! */=0A= };=0A= @@ -230,6 +237,14 @@=0A= } ipfw_insn_u32;=0A= =0A= /*=0A= + * This is used to store 64-bit quota value.=0A= + */=0A= +typedef struct _ipfw_insn_u64 {=0A= + ipfw_insn o;=0A= + u_int64_t quota;=0A= +} ipfw_insn_u64;=0A= +=0A= +/*=0A= * This is used to store IP addr-mask pairs.=0A= */=0A= typedef struct _ipfw_insn_ip {=0A= @@ -351,12 +366,17 @@=0A= *=0A= * When assembling instruction, remember the following:=0A= *=0A= + * + if a rule has a "quota" option, then the first instruction=0A= + * (at r->cmd) MUST BE an O_QBELOW|O_QABOVE=0A= * + if a rule has a "keep-state" (or "limit") option, then the=0A= * first instruction (at r->cmd) MUST BE an O_PROBE_STATE=0A= * + if a rule has a "log" option, then the first action=0A= * (at ACTION_PTR(r)) MUST be O_LOG=0A= * + if a rule has an "altq" option, it comes after "log"=0A= *=0A= + *=0A= + * NOTE: actually, O_PROB instruction may be first too. But = O_QBELOW|O_QABOVE=0A= + * MUST BE always first (at r->cmd).=0A= * NOTE: we use a simple linked list of rules because we never need=0A= * to delete a rule without scanning the list. We do not use=0A= * queue(3) macros for portability and readability.=0A= --- sys/netinet/ip_fw2.c.orig Sat Jul 16 14:55:58 2005=0A= +++ sys/netinet/ip_fw2.c Sat Jul 16 17:06:19 2005=0A= @@ -2251,6 +2251,36 @@=0A= * logic to deal with F_NOT and F_OR flags associated=0A= * with the opcode.=0A= */=0A= + case O_QBELOW:=0A= + match =3D (f->bcnt < ((ipfw_insn_u64 *)cmd)->quota);=0A= + break;=0A= +=0A= + case O_QABOVE:=0A= + match =3D (f->bcnt > ((ipfw_insn_u64 = *)cmd)->quota);=0A= + break;=0A= +=0A= + case O_CHECK_QUOTA:=0A= + {=0A= + struct ip_fw* rule;=0A= + for (rule =3D f->next;=0A= + rule && cmd->arg1 >=3D rule->rulenum;=0A= + rule =3D rule->next)=0A= + if (rule->rulenum =3D=3D cmd->arg1)=0A= + switch (rule->cmd->opcode) {=0A= + case O_QBELOW:=0A= + match =3D (rule->bcnt <=0A= + ((ipfw_insn_u64 *)(rule->cmd))->quota);=0A= + break;=0A= + case O_QABOVE:=0A= + match =3D (rule->bcnt >=0A= + = ((ipfw_insn_u64 *)(rule->cmd))->quota);=0A= + break;=0A= + default: =0A= + break;=0A= + }=0A= + }=0A= + break;=0A= +=0A= case O_NOP:=0A= match =3D 1;=0A= break;=0A= @@ -3373,6 +3403,7 @@=0A= case O_EXT_HDR:=0A= case O_IP6:=0A= case O_IP4:=0A= + case O_CHECK_QUOTA:=0A= if (cmdlen !=3D F_INSN_SIZE(ipfw_insn))=0A= goto bad_size;=0A= break;=0A= @@ -3388,6 +3419,17 @@=0A= case O_ICMPTYPE:=0A= if (cmdlen !=3D F_INSN_SIZE(ipfw_insn_u32))=0A= goto bad_size;=0A= + break;=0A= +=0A= + case O_QBELOW:=0A= + case O_QABOVE:=0A= + if (cmdlen !=3D F_INSN_SIZE(ipfw_insn_u64))=0A= + goto bad_size;=0A= + if (cmd !=3D rule->cmd) {=0A= + printf("ipfw: bogus rule, opcode %d must be first\n",=0A= + cmd->opcode);=0A= + return EINVAL;=0A= + }=0A= break;=0A= =0A= case O_LIMIT:=0A= --- sbin/ipfw/ipfw2.c.orig Sat Jul 16 15:21:06 2005=0A= +++ sbin/ipfw/ipfw2.c Sat Jul 16 17:11:42 2005=0A= @@ -73,6 +73,8 @@=0A= show_sets, /* display rule sets */=0A= test_only, /* only check syntax */=0A= comment_only, /* only print action and comment */=0A= + not_humanval, /* don't use human-readable unit suffixes=0A= + when show boundary values */=0A= verbose;=0A= =0A= #define IP_MASK_ALL 0xffffffff=0A= @@ -277,6 +279,10 @@=0A= TOK_SRCIP6,=0A= =0A= TOK_IPV4,=0A= +=0A= + TOK_QBELOW,=0A= + TOK_QABOVE,=0A= + TOK_CHECK_QUOTA,=0A= };=0A= =0A= struct _s_x dummynet_params[] =3D {=0A= @@ -404,6 +410,9 @@=0A= { "src-ipv6", TOK_SRCIP6},=0A= { "src-ip6", TOK_SRCIP6},=0A= { "//", TOK_COMMENT },=0A= + { "below", TOK_QBELOW},=0A= + { "above", TOK_QABOVE},=0A= + { "check-quota", TOK_CHECK_QUOTA},=0A= =0A= { "not", TOK_NOT }, /* pseudo option */=0A= { "!", /* escape ? */ TOK_NOT }, /* pseudo option */=0A= @@ -1636,6 +1645,10 @@=0A= flags |=3D HAVE_PROTO;=0A= break;=0A= =0A= + case O_QBELOW:=0A= + case O_QABOVE:=0A= + break; =0A= +=0A= default: /*options ... */=0A= if (!(cmd->len & (F_OR|F_NOT)))=0A= if (((cmd->opcode =3D=3D O_IP6) &&=0A= @@ -1857,6 +1870,10 @@=0A= case O_EXT_HDR:=0A= print_ext6hdr( (ipfw_insn *) cmd );=0A= break;=0A= + =0A= + case O_CHECK_QUOTA:=0A= + printf(" check-quota %d", cmd->arg1);=0A= + break;=0A= =0A= default:=0A= printf(" [opcode %d len %d]",=0A= @@ -1872,6 +1889,28 @@=0A= }=0A= }=0A= show_prerequisites(&flags, HAVE_IP, 0);=0A= +=0A= + if (rule->cmd->opcode =3D=3D O_QBELOW || rule->cmd->opcode =3D=3D = O_QABOVE) {=0A= + uint64_t bound =3D ((ipfw_insn_u64 *)(rule->cmd))->quota;=0A= + if (rule->cmd->opcode =3D=3D O_QBELOW) =0A= + printf(" below ");=0A= + else=0A= + printf(" above ");=0A= + if (!not_humanval) {=0A= + if ((bound >> 10) && !(bound & 0x2FF)) {=0A= + if ((bound >> 20) && !(bound & 0xFFFFF)) {=0A= + if ((bound >> 30) && !(bound & 0x3FFFFFFF))=0A= + printf("%uGB", bound >> 30);=0A= + else=0A= + printf("%uMB", bound >> 20);=0A= + } else=0A= + printf("%uKB", bound >> 10);=0A= + } else=0A= + printf("%uB", bound);=0A= + } else=0A= + printf("%u", bound);=0A= + }=0A= +=0A= if (comment)=0A= printf(" // %s", comment);=0A= printf("\n");=0A= @@ -2515,6 +2554,9 @@=0A= " icmp6types LIST | ext6hdr LIST | flow-id N[,N] |\n"=0A= " mac ... | mac-type LIST | proto LIST | {recv|xmit|via} {IF|IPADDR} = |\n"=0A= " setup | {tcpack|tcpseq|tcpwin} NN | tcpflags SPEC | tcpoptions SPEC = |\n"=0A= +" tcpdatalen LIST | below VALUE | above VALUE | check-quota NUM |\n"=0A= +" verrevpath | versrcreach | antispoof\n"=0A= +=0A= " tcpdatalen LIST | verrevpath | versrcreach | antispoof\n"=0A= );=0A= exit(0);=0A= @@ -3677,7 +3719,7 @@=0A= * various flags used to record that we entered some fields.=0A= */=0A= ipfw_insn *have_state =3D NULL; /* check-state or keep-state */=0A= - ipfw_insn *have_log =3D NULL, *have_altq =3D NULL;=0A= + ipfw_insn *have_log =3D NULL, *have_altq =3D NULL, *have_quota =3D = NULL;=0A= size_t len;=0A= =0A= int i;=0A= @@ -4494,6 +4536,66 @@=0A= ac =3D 0;=0A= break;=0A= =0A= + case TOK_QBELOW:=0A= + NEED1("below requires numeric value");=0A= + if (open_par)=0A= + errx(EX_USAGE, "below cannot be part "=0A= + "of an or block");=0A= + if (have_quota)=0A= + errx(EX_USAGE, "only one of below|above is allowed");=0A= + if (cmd->len & F_NOT)=0A= + errx(EX_USAGE,=0A= + "\"not\" not allowed with below option");=0A= + {=0A= + char *end =3D NULL;=0A= + uint64_t bound =3D strtoull(*av, &end, 0);=0A= + if (bound)=0A= + switch (*end){=0A= + case 'G': bound *=3D 1024;=0A= + case 'M': bound *=3D 1024;=0A= + case 'K': bound *=3D 1024;=0A= + };=0A= + cmd->opcode =3D O_QBELOW;=0A= + ((ipfw_insn_u64 *)cmd)->quota =3D bound;=0A= + cmd->len =3D F_INSN_SIZE(ipfw_insn_u64) & F_LEN_MASK;=0A= + have_quota =3D cmd;=0A= + ac--; av++;=0A= + }=0A= + break;=0A= +=0A= + case TOK_QABOVE:=0A= + NEED1("above requires numeric value");=0A= + if (open_par)=0A= + errx(EX_USAGE, "above cannot be part "=0A= + "of an or block");=0A= + if (have_quota)=0A= + errx(EX_USAGE, "only one of below|above = is allowed");=0A= + if (cmd->len & F_NOT)=0A= + errx(EX_USAGE,=0A= + "\"not\" not allowed with above = option");=0A= + {=0A= + char *end =3D NULL;=0A= + uint64_t bound =3D strtoull(*av, &end, = 0);=0A= + if (bound)=0A= + switch (*end){=0A= + case 'G': bound *=3D 1024;=0A= + case 'M': bound *=3D 1024;=0A= + case 'K': bound *=3D 1024;=0A= + };=0A= + cmd->opcode =3D O_QABOVE;=0A= + ((ipfw_insn_u64 *)cmd)->quota =3D bound;=0A= + cmd->len =3D F_INSN_SIZE(ipfw_insn_u64) = & F_LEN_MASK;=0A= + have_quota =3D cmd;=0A= + ac--; av++;=0A= + }=0A= + break;=0A= +=0A= + case TOK_CHECK_QUOTA:=0A= + NEED1("check-quota requires rule number");=0A= + fill_cmd(cmd, O_CHECK_QUOTA, 0, strtoul(*av, NULL, 0));=0A= + ac--; av++;=0A= + break;=0A= +=0A= default:=0A= errx(EX_USAGE, "unrecognised option [%d] %s\n", i, s);=0A= }=0A= @@ -4506,6 +4608,8 @@=0A= done:=0A= /*=0A= * Now copy stuff into the rule.=0A= + * If we have a quota option, the first instruction MUST BE=0A= + * a O_QBELOW or O_QABOVE.=0A= * If we have a keep-state option, the first instruction=0A= * must be a PROBE_STATE (which is generated here).=0A= * If we have a LOG option, it was stored as the first command,=0A= @@ -4514,7 +4618,15 @@=0A= dst =3D (ipfw_insn *)rule->cmd;=0A= =0A= /*=0A= - * First thing to write into the command stream is the match = probability.=0A= + * First write into the command stream quota instruction=0A= + */=0A= + if (have_quota) {=0A= + bcopy(have_quota, dst, F_LEN(have_quota) * sizeof(uint32_t));=0A= + dst =3D next_cmd(dst);=0A= + }=0A= +=0A= + /*=0A= + * write the match probability=0A= */=0A= if (match_prob !=3D 1) { /* 1 means always match */=0A= dst->opcode =3D O_PROB;=0A= @@ -4531,7 +4643,8 @@=0A= dst =3D next_cmd(dst);=0A= }=0A= /*=0A= - * copy all commands but O_LOG, O_KEEP_STATE, O_LIMIT, O_ALTQ=0A= + * copy all commands but O_LOG, O_KEEP_STATE, O_LIMIT, O_ALTQ,=0A= + * O_QBELOW, O_QABOVE=0A= */=0A= for (src =3D (ipfw_insn *)cmdbuf; src !=3D cmd; src +=3D i) {=0A= i =3D F_LEN(src);=0A= @@ -4541,6 +4654,8 @@=0A= case O_KEEP_STATE:=0A= case O_LIMIT:=0A= case O_ALTQ:=0A= + case O_QBELOW:=0A= + case O_QABOVE:=0A= break;=0A= default:=0A= bcopy(src, dst, i * sizeof(uint32_t));=0A= @@ -4848,7 +4963,7 @@=0A= save_av =3D av;=0A= =0A= optind =3D optreset =3D 0;=0A= - while ((ch =3D getopt(ac, av, "abcdefhnNqs:STtv")) !=3D -1)=0A= + while ((ch =3D getopt(ac, av, "abcdefhHnNqs:STtv")) !=3D -1)=0A= switch (ch) {=0A= case 'a':=0A= do_acct =3D 1;=0A= @@ -4879,6 +4994,10 @@=0A= free_args(save_ac, save_av);=0A= help();=0A= break; /* NOTREACHED */=0A= +=0A= + case 'H': /* don't use human-readable output */=0A= + not_humanval =3D 1;=0A= + break;=0A= =0A= case 'n':=0A= test_only =3D 1;=0A= ------=_NextPart_000_0019_01C58A30.81E63C20 Content-Type: text/plain; format=flowed; name="traffic_quota_example.txt"; reply-type=original Content-Transfer-Encoding: 7bit Content-Disposition: attachment;filename="traffic_quota_example.txt" Example: We will enforce traffic shaping and traffic quota in a client's network behind a freebsd gateway. Definitions/policy: 1. clients network: 1.1.1.0/24. 2. Quota policy: unlimited clients: 1.1.1.0/27 100MB/day clients: 1.1.1.32/27 ipfw-set:2 ipfw-range:1000-9999 1GB/week clients: 1.1.1.64/26 ipfw-set:3 ipfw-range:10000-19999 10GB/month clients: 1.1.1.128/25 ipfw-set:4 ipfw-range:20000-29999 3. Shaping policy: 1.1.1.0/27 unlimited 1.1.1.32/27 100Mbps in/out 1.1.1.64/26 10Mbps in/out 1.1.1.128/25 1Mbps in/out quota exceeded 64Kbps in/out ipfw.sh ======= #!/bin/sh ipfw = "/sbin/ipfw" qos = "40000" allow = "65000" lan="em0" wan="em1" # ****************** # * QOS definition * # ****************** # quota exceeded pipes: ${ipfw} pipe 1 config bw 64Kbit/s mask dst-ip 0x000000ff ${ipfw} pipe 2 config bw 64Kbit/s mask src-ip 0x000000ff # 1MB pipes: ${ipfw} pipe 3 config bw 1Mbit/s mask dst-ip 0x000000ff ${ipfw} pipe 4 config bw 1Mbit/s mask src-ip 0x000000ff # 10MB pipes: ${ipfw} pipe 5 config bw 10Mbit/s mask dst-ip 0x000000ff ${ipfw} pipe 6 config bw 10Mbit/s mask src-ip 0x000000ff # 100MB pipes: ${ipfw} pipe 7 config bw 100Mbit/s mask dst-ip 0x000000ff ${ipfw} pipe 8 config bw 100Mbit/s mask src-ip 0x000000ff # ************************* # * RECEIVE Without Quota * # ************************* ${ipfw} add 100 allow ip from any to any in recv ${lan} ${ipfw} add 200 allow ip from any to any in recv ${wan} # *********************** # * 100MB/DAY both ways * # *********************** ${ipfw} add 1000 set 2 allow ip from any to 1.1.1.32/32 out xmit ${lan} check-quota 1001 ${ipfw} add 1001 set 2 skipto ${qos} ip from 1.1.1.32/32 to any out xmit ${wan} above 100M ${ipfw} add 1002 set 2 allow ip from any to 1.1.1.33/32 out xmit ${lan} check-quota 1003 ${ipfw} add 1003 set 2 skipto ${qos} ip from 1.1.1.33/32 to any out xmit ${wan} above 100M .... ${ipfw} add 1062 set 2 allow ip from any to 1.1.1.63/32 out xmit ${lan} check-quota 1063 ${ipfw} add 1063 set 2 skipto ${qos} ip from 1.1.1.63/32 to any out xmit ${wan} above 100M ${ipfw} add 9999 skipto ${allow} pipe 1 ip from any to 1.1.1.32/27 out xmit ${lan} ${ipfw} add 9999 skipto ${allow} pipe 2 ip from 1.1.1.32/27 to any out xmit ${wan} # ********************** # * 1GB/WEEK both ways * # ********************** ${ipfw} add 10000 set 3 allow ip from any to 1.1.1.64/32 out xmit ${lan} check-quota 10001 ${ipfw} add 10001 set 3 skipto ${qos} ip from 1.1.1.64/32 to any out xmit ${wan} above 1G ${ipfw} add 10002 set 3 allow ip from any to 1.1.1.65/32 out xmit ${lan} check-quota 10003 ${ipfw} add 10003 set 3 skipto ${qos} ip from 1.1.1.65/32 to any out xmit ${wan} above 1G .... ${ipfw} add 10126 set 3 allow ip from any to 1.1.1.127/32 out xmit ${lan} check-quota 10063 ${ipfw} add 10127 set 3 skipto ${qos} ip from 1.1.1.127/32 to any out xmit ${wan} above 1G ${ipfw} add 19999 skipto ${allow} pipe 1 ip from any to 1.1.1.64/26 out xmit ${lan} ${ipfw} add 19999 skipto ${allow} pipe 2 ip from 1.1.1.64/26 to any out xmit ${wan} # *********************** # * 10GB/MONTH both ways* # *********************** ${ipfw} add 20000 set 4 allow ip from any to 1.1.1.128/32 out xmit ${lan} check-quota 20001 ${ipfw} add 20001 set 4 skipto ${qos} ip from 1.1.1.128/32 to any out xmit ${wan} above 10G ${ipfw} add 20002 set 4 allow ip from any to 1.1.1.129/32 out xmit ${lan} check-quota 20003 ${ipfw} add 20003 set 4 skipto ${qos} ip from 1.1.1.129/32 to any out xmit ${wan} above 10G .... ${ipfw} add 20254 set 4 allow ip from any to 1.1.1.255/32 out xmit ${lan} check-quota 20255 ${ipfw} add 20255 set 4 skipto ${qos} ip from 1.1.1.255/32 to any out xmit ${wan} above 10G ${ipfw} add 29999 skipto ${allow} pipe 1 ip from any to 1.1.1.128/25 out xmit ${lan} ${ipfw} add 29999 skipto ${allow} pipe 2 ip from 1.1.1.128/25 to any out xmit ${wan} # ************* # * QOS * # ************* # 1.1.1.128/25 each of them has 1MBps in and 1Mbps out shaping ${ipfw} add ${qos} skipto ${allow} pipe 3 ip from any to 1.1.1.128/25 out xmit ${lan} ${ipfw} add ${qos} skipto ${allow} pipe 4 ip from 1.1.1.128/25 to any out xmit ${wan} # 1.1.1.64/26 each of them has 10MBps in and 10Mbps out shaping ${ipfw} add ${qos} skipto ${allow} pipe 5 ip from any to 1.1.1.64/26 out xmit ${lan} ${ipfw} add ${qos} skipto ${allow} pipe 6 ip from 1.1.1.64/26 to any out xmit ${wan} # 1.1.1.32/32 each of them has 100MBps in and 100Mbps out shaping ${ipfw} add ${qos} skipto ${allow} pipe 7 ip from any to 1.1.1.32/27 out xmit ${lan} ${ipfw} add ${qos} skipto ${allow} pipe 8 ip from 1.1.1.32/27 to any out xmit ${wan} # ********* # * allow * # ********* ${ipfw} add ${allow} allow ip from any to any /etc/crontab: ============= # Perform daily/weekly/monthly ipfw counter reset. 0 0 * * * root /sbin/ipfw zero set 2 0 0 * * 0 root /sbin/ipfw zero set 3 0 0 0 * * root /sbin/ipfw zero set 4 ------=_NextPart_000_0019_01C58A30.81E63C20--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?001c01c58a17$5dbe4a40$0100000a>