From owner-freebsd-ipfw@FreeBSD.ORG Sun Oct 31 20:35:33 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0A8BA16A4CE for ; Sun, 31 Oct 2004 20:35:33 +0000 (GMT) Received: from shellma.zin.lublin.pl (shellma.zin.lublin.pl [212.182.126.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id 890F543D45 for ; Sun, 31 Oct 2004 20:35:31 +0000 (GMT) (envelope-from pawmal-posting@freebsd.lublin.pl) Received: by shellma.zin.lublin.pl (Postfix, from userid 1018) id E94C73474C1; Sun, 31 Oct 2004 21:35:58 +0100 (CET) Date: Sun, 31 Oct 2004 21:35:58 +0100 From: Pawel Malachowski To: freebsd-ipfw@freebsd.org Message-ID: <20041031203558.GA49557@shellma.zin.lublin.pl> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="OXfL5xGRrasGEqWY" Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.4.2i Subject: [PATCH] burstable dummynet pipe X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Oct 2004 20:35:33 -0000 --OXfL5xGRrasGEqWY Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: 8bit Hello, Attached (poor) patch (against 5.3-RC1) implements pipes that can be overloaded (exceed their bw) for a (short) period of time. It adds yet another parameter for pipe: `burst x', x in (K)Bytes. Should also work with dynamic (clonable via mask) pipes. Configuring `x bytes burst' means we are allowing to transmit x bytes with increased (by default: ~2x) speed. After that, traffic passes through pipe as usual (bw-limited), however, burst credit is slowly (by default: ~10x lower than bw) accumulated when pipe is idle (low traffic, empty ticks). When x bytes of burst credit is accumulated, one can consume it once again. Etc. No man page provided. Sorry for poor coding. Still, seems to work. ;) Example: % ipfw add ... // pass my test traffic to pipe 1 % ipfw pipe 1 config bw 256kbit queue 5KB burst 110KB % ipfw pipe show 00001: 256.000 Kbit/s 0 ms 110 KB 5120 B 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 % sleep 15 % wget --progress=dot some-file [...] HTTP request sent, awaiting response... 200 OK Length: 10,485,760 [text/plain] 0K .......... .......... .......... .......... .......... 0% 61.80 KB/s 50K .......... .......... .......... .......... .......... 0% 58.14 KB/s 100K .......... .......... .......... .......... .......... 1% 32.05 KB/s 150K .......... .......... .......... .......... .......... 1% 30.49 KB/s 200K .......... .......... .......... .......... .......... 2% 29.59 KB/s 250K .......... .......... .......... .......... .......... 2% 30.30 KB/s 300K .......... .......... .......... .......... .......... 3% 29.59 KB/s 350K .......... .......... .......... .......... .......... 3% 29.41 KB/s 400K .......... .......... ......^C % sleep 10 % wget --progress=dot some-file [...] HTTP request sent, awaiting response... 200 OK Length: 10,485,760 [text/plain] 0K .......... .......... .......... .......... .......... 0% 26.32 KB/s 50K .......... .......... .......... .......... .......... 0% 33.36 KB/s 100K .......... .......... .......... .......... .......... 1% 28.74 KB/s 150K .......... .......... .......... .......... .......... 1% 31.23 KB/s 200K .......^C % sleep 15 % wget --progress=dot some-file [...] HTTP request sent, awaiting response... 200 OK Length: 10,485,760 [text/plain] 0K .......... .......... .......... .......... .......... 0% 61.73 KB/s 50K .......... .......... .......... .......... .......... 0% 58.14 KB/s 100K .......... .......... .......... .......... .......... 1% 32.05 KB/s 150K .......... ...^C -- Paweł Małachowski --OXfL5xGRrasGEqWY Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: attachment; filename="burst--ip_dummynet.c.diff" --- /sys/netinet/ip_dummynet.c-orig Fri Oct 29 21:34:46 2004 +++ /sys/netinet/ip_dummynet.c Sun Oct 31 20:23:33 2004 @@ -580,14 +580,34 @@ * setting len_scaled = 0 does the job. */ q->numbytes += ( curr_time - q->sched_time ) * p->bandwidth; +#if 0 + if ( q->burstmode==DN_BURST_FLUSH && (q->currburst < q->numbytes/8/hz) ) { + printf("Full burst power. ;)\n"); + } + else if ( q->burstmode==DN_BURST_FLUSH ) { /* currbust >= numbytes/8/hz */ + printf("Burst getting empty, slow down a bit\n"); + } +#endif while ( (pkt = q->head) != NULL ) { int len = pkt->m_pkthdr.len; int len_scaled = p->bandwidth ? len*8*hz : 0 ; + if (q->burstmode==DN_BURST_FLUSH) + len_scaled /= 2; /* increase burst speed 2x. this will be <2x because we left SET_TICKS untouched but hey, it still works good enough... */ if (len_scaled > q->numbytes ) break ; q->numbytes -= len_scaled ; + if ( q->burstmode==DN_BURST_FLUSH ) + q->currburst -= len ; /* can go <0, we will fix this later */ move_pkt(pkt, q, p, len); } + + /* If burst credit is empty, announce that we are accumulatig it now. */ + if (q->burstmode==DN_BURST_FLUSH && q->currburst<=0) { + q->currburst = 0 ; + q->burstmode = DN_BURST_ACCUMULATE ; /* will get leftovers if found */ + //printf("Stop flushing burst (empty), start accumulating!\n"); + } + /* * If we have more packets queued, schedule next ready event * (can only occur when bandwidth != 0, otherwise we would have @@ -603,6 +623,21 @@ * queue on error hoping next time we are luckier. */ } else { /* RED needs to know when the queue becomes empty */ +#if 0 + if ( q->burstmode==DN_BURST_ACCUMULATE ) { + /* There are some leftovers, saturation<=bw (can be <). + * After some tests I decided not to accumulate them because + * burst credit grows even when saturation~=bw and burst + * explode from time to time during heavy-downloading + * sessions (it looks weird). */ + q->currburst += ( q->numbytes/(8*hz) ); + if (q->currburst >= p->burst) { + q->currburst = p->burst ; + q->burstmode = DN_BURST_FLUSH ; + //printf("Accumulate leftovers, we can start flushing burst now.\n"); + } + } +#endif q->q_time = curr_time; q->numbytes = 0; } @@ -880,11 +915,14 @@ return NULL ; } q->fs = fs ; + q->currburst = 0 ; /* we will accumulate burst while waiting */ + q->burstmode = DN_BURST_ACCUMULATE ; q->hash_slot = i ; q->next = fs->rq[i] ; q->S = q->F + 1; /* hack - mark timestamp as invalid */ fs->rq[i] = q ; fs->rq_elements++ ; + return q ; } @@ -1224,9 +1262,35 @@ * Fixed-rate queue: just insert into the ready_heap. */ dn_key t = 0 ; + dn_key pst = 0 ; /* preserved schedule time */ + dn_key ct = 0 ; /* computed time, with bw, gives burst credit for idle ticks */ if (pipe->bandwidth) t = SET_TICKS(m, q, pipe); + pst = q->sched_time ; q->sched_time = curr_time ; + + if (pipe->burst && q->burstmode==DN_BURST_ACCUMULATE) { + /* We should use `previous' t, but hey, when passing constant + current traffic, t is usually the same or very close. + So it is acceptable to use current t value for simplicity. */ + ct = curr_time - pst - t; + /* Make sure, we avoid negative. This happens all the time + when bw is full of passing traffic. */ + if (DN_KEY_LT(curr_time-pst,t)) ct=0; + + /* Pipe was idle for some time, increase burst credit + to reflect this. + Burst credit can grow slower or faster (but it is always + connected with pipe bw), just increase or descrease `a' + in this expression: (8*hz*a), a=10 is the default. */ + q->currburst += ct*pipe->bandwidth/(8*hz*10); + if ( q->currburst >= pipe->burst ) { + q->currburst = pipe->burst ; + q->burstmode = DN_BURST_FLUSH ; + } + //printf("dummynet_io(): (t=%lld,curr-sched=%lld) added %lld, now currburst=%d(burst=%d)\n", t,ct,(ct) * pipe->bandwidth /(8*hz*10),q->currburst,pipe->burst ); + } + if (t == 0) /* must process it now */ ready_event( q ); else @@ -1614,6 +1678,7 @@ bcopy(p->if_name, x->if_name, sizeof(p->if_name) ); x->ifp = NULL ; /* reset interface ptr */ x->delay = p->delay ; + x->burst = p->bandwidth ? p->burst : 0 ; /* burst is for fixed-bw pipes */ set_fs_parms(&(x->fs), pfs); --OXfL5xGRrasGEqWY Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: attachment; filename="burst--ip_dummynet.h.diff" --- /sys/netinet/ip_dummynet.h-orig Fri Oct 29 21:34:56 2004 +++ /sys/netinet/ip_dummynet.h Sun Oct 31 19:31:36 2004 @@ -208,6 +208,11 @@ u_int len_bytes ; u_long numbytes ; /* credit for transmission (dynamic queues) */ + int currburst ; /* current burst buffer credit, in bytes */ + u_short burstmode ; +#define DN_BURST_ACCUMULATE 0 /* accumulating burst credit */ +#define DN_BURST_FLUSH 1 /* we can use accumulated burst credit */ + u_int64_t tot_pkts ; /* statistics counters */ u_int64_t tot_bytes ; u_int32_t drops ; @@ -315,6 +320,7 @@ int pipe_nr ; /* number */ int bandwidth; /* really, bytes/tick. */ int delay ; /* really, ticks */ + int burst ; /* burst buffer credit, in bytes */ struct mbuf *head, *tail ; /* packets in delay line */ --OXfL5xGRrasGEqWY Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: attachment; filename="burst--ipfw.c.diff" --- /usr/src/sbin/ipfw/ipfw2.c-orig Fri Oct 29 21:16:23 2004 +++ /usr/src/sbin/ipfw/ipfw2.c Sun Oct 31 20:14:17 2004 @@ -253,6 +253,7 @@ TOK_DROPTAIL, TOK_PROTO, TOK_WEIGHT, + TOK_BURST, }; struct _s_x dummynet_params[] = { @@ -275,6 +276,7 @@ { "delay", TOK_DELAY }, { "pipe", TOK_PIPE }, { "queue", TOK_QUEUE }, + { "burst", TOK_BURST }, { "dummynet-params", TOK_NULL }, { NULL, 0 } /* terminator */ }; @@ -1518,6 +1520,7 @@ for (; nbytes >= sizeof *p; p = (struct dn_pipe *)next) { double b = p->bandwidth; char buf[30]; + char buf2[30]; char prefix[80]; if (p->next != (struct dn_pipe *)DN_IS_PIPE) @@ -1547,8 +1550,14 @@ else sprintf(buf, "%7.3f bit/s ", b); - sprintf(prefix, "%05d: %s %4d ms ", - p->pipe_nr, buf, p->delay); + if (p->burst >= 8192) + sprintf(buf2, "%d KB", p->burst / 1024); + else + sprintf(buf2, "%d B", p->burst); + + sprintf(prefix, "%05d: %s %4d ms %s", + p->pipe_nr, buf, p->delay, buf2); + print_flowset_parms(&(p->fs), prefix); if (verbose) printf(" V %20qd\n", p->V >> MY_M); @@ -2280,6 +2289,18 @@ p.fs.qsize *= 1024; } else if (*end == 'B' || !strncmp(end, "by", 2)) { p.fs.flags_fs |= DN_QSIZE_IS_BYTES; + } + ac--; av++; + break; + + case TOK_BURST: + if (do_pipe != 1) + errx(EX_DATAERR, "burst only valid for pipes"); + NEED1("burst needs credit size\n"); + end = NULL; + p.burst = strtoul(av[0], &end, 0); + if (*end == 'K' || *end == 'k') { + p.burst *= 1024; } ac--; av++; break; --OXfL5xGRrasGEqWY--