From owner-freebsd-ipfw@FreeBSD.ORG Sun May 18 22:48:07 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 98FA237B401; Sun, 18 May 2003 22:48:07 -0700 (PDT) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id 281A743FD7; Sun, 18 May 2003 22:47:23 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.8p1/8.12.3) with ESMTP id h4J5lNQg036073; Sun, 18 May 2003 22:47:23 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.8p1/8.12.3/Submit) id h4J5lNbK036072; Sun, 18 May 2003 22:47:23 -0700 (PDT) (envelope-from rizzo) Date: Sun, 18 May 2003 22:47:23 -0700 From: Luigi Rizzo To: ipfw@freebsd.org Message-ID: <20030518224723.A35944@xorpc.icir.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="qDbXVdCdHGoSgWSk" Content-Disposition: inline User-Agent: Mutt/1.2.5.1i Subject: [FEEDBACK REQUIRED] patch for ipfw/ipfw2 on alpha/sparc64 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2003 05:48:08 -0000 --qDbXVdCdHGoSgWSk Content-Type: text/plain; charset=us-ascii Content-Disposition: inline [Bcc to alpha@ and sparc64@ -- sorry for the crosspost, but i need feedback from people using these boxes] Hi, in the past there have been reports about panics on 64 bit architectures due to unaligned accesses to parts of the ipfw2 rules. The attached patch tries to fix them. I tested that it compiles, but couldn't run it on the relevant architectures for lack of hardware, and have not found yet anyone who wanted to try it. If you care and want to spend a few minutes to test the patch on a Sparc64 or Alpha box and provide feedback, I'd ask you to do the following: STEP 1: try to reproduce the panic + compile a kernel (a recent CURRENT or STABLE) WITHOUT THE PATCH (i.e. with standard sources) with the following options: options IPFIREWALL options DUMMYNET options IPFW2 # only in STABLE (on STABLE, make sure you also have rebuilt ipfw with make -DIPFW2) + [THIS SHOULD TRIGGER A PANIC] reboot with the new kernel and configure ipfw with one of the following configurations, and fire some traffic at the firewall (e.g. "ssh localhost") to try and cause a panic. If these configuration do not cause a panic and you know a better one, please post it. ipfw pipe 1 config ipfw add pipe 1 tcp from any to any 1-65535,1-65535 or ipfw pipe 1 config ipfw add pipe 1 tcp from any to any 1-65535,1-65535,1-65535 or ipfw add allow tcp from any to any 1-65535,1-65535 or ipfw add allow tcp from any to any 1-65535,1-65535,1-65535 STEP 2: test the patch + patch sys/netinet/ and sbin/ipfw/ with the attached patch, rebuild and reinstall kernel and ipfw, and try again to see if the panic does not occur anymore. The patch is for -current, but it should apply to -stable with relatively little effort (see the description below). Feedback would be really appreciated if you want to have this fixed for 5.1 cheers luigi --------------------------------------- Description and proposed patch for ipfw[2] panics on 64-bit architectures. BACKGROUND ipfw2 has been designed so that rules are made of a standard header, followed by a sequence of [micro]instructions (represented by C struct's), each one having a length of a multiple of 32 bits. Because instructions are written one after the other without padding, variables of size > 32bit which reside within each struct might end up unaligned, and the compiler cannot help fixing this. I am listing below some of the approaches that can be used to avoid the problem; because such occurences are extremely rare (there is actually only one instance, "void *pipe_ptr" within "struct _ipfw_insn_pipe"), i believe the first approach to be the easiest to implement and also the most effective in terms of memory occupation and code modifications. Solution #1: handle the critical 64-bit objects with bcopy/bcmp instead of accessing them directly. This is slightly inefficient but fortunately the only instance when it needs to be done is when passing a packet to a dummynet pipe. Also, the size of the copy/compare is known at compile time, so i suppose it might be candidate to some compiler optimizations. Solution #2: make the size of each microinstruction a multiple of 64 bits, so the alignment can be guaranteed by the compiler. This would generate quite an increase of the instruction sizes, and more importantly, would require a significant revision of the ipfw code. The good thing of this approach is that, once done, would put on the compiler the burden of providing the alignment. Solution #3: possibly introduce a padding (in the form of NOP instructions) before those instructions which need alignment. This also requires some amount of extra code to introduce the padding when necessary, and leaves the burden of providing the aligment on the programmer. DESCRIPTION OF THE ATTACHED PATCH The attached patch modifies the following files: src/sys/netinet/ip_dummynet.c src/sys/netinet/ip_fw.h src/sys/netinet/ip_fw2.c src/sbin/ipfw2.c and the changes are the following: sys/netinet/ip_dummynet.c: use bcopy to read/write the "pipe_ptr" field of "struct _ipfw_insn_pipe" on non-i386 architectures (which might be sensitive to alignment problems). This is invoked only in two places, when passing a packet to a dummynet pipe. src/sys/netinet/ip_fw.h @@ -32,11 +32,16 @@ add a comment to explain the alignment problems. @@ -211,7 +216,7 @@ change "int unit" to "int32_t unit" to save space on those architectures where int are 64 bits. @@ -220,10 +225,13 @@ add a comment explaining that "void *pipe_ptr" must be treated carefully @@ -278,7 +286,9 @@ remove the unnecessary field "set_disable" from "struct ip_fw". The set of disabled rulesets is passed up to userspace through the "next rule" pointer (unused otherwise, in that context), which is manipulated using bcopy to avoid aligmnent problems. NOTE: by doing this, we revert to the method used by ipfw2 in 4.x, so we should save the change in that branch. @@ -319,12 +329,12 @@ reorder some fields of _ipfw_dyn_rule to reduce the amount of padding. NOTE: This change is not strictly necessary, and could be omitted if we want to keep the struct unchanged (this is only true for the to 4.x branch, because the "rulenum" field from the same struct should be removed anyways, see the diff below). @@ -334,7 +344,9 @@ remove the "rulenum" field from struct _ipfw_dyn_rule, same as done for "set_disable" in "struct ip_fw". sys/netinet/ip_fw2.c @@ -2017,8 +2017,15 @@ use bcmp/bzero to manipulate pipe_ptr. As the comment says, this is done very infrequently so efficiency is not a concern. @@ -2559,7 +2566,8 @@ @@ -2571,7 +2579,8 @@ use bcopy to pass up info from the kernel to userland through an unused pointer, to avoid aligmnent problems. sbin/ipfw/ipfw2.c @@ -813,8 +813,9 @@ @@ -1454,7 +1458,9 @@ use bcopy to fetch the "set_disable" info into a u_int32_t variable @@ -1213,13 +1214,16 @@ @@ -1690,9 +1696,12 @@ use bcopy to fetch the "rulenum" info into a uint16_t variable. ------------------------------ --qDbXVdCdHGoSgWSk Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="ipfw2.diff" Index: sys/netinet/ip_dummynet.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_dummynet.c,v retrieving revision 1.63 diff -u -r1.63 ip_dummynet.c --- sys/netinet/ip_dummynet.c 27 Mar 2003 15:00:10 -0000 1.63 +++ sys/netinet/ip_dummynet.c 11 May 2003 07:19:56 -0000 @@ -1027,7 +1027,11 @@ if (cmd->opcode == O_LOG) cmd += F_LEN(cmd); +#ifdef I386 fs = ((ipfw_insn_pipe *)cmd)->pipe_ptr; +#else + bcopy(& ((ipfw_insn_pipe *)cmd)->pipe_ptr, &fs, sizeof(fs)); +#endif if (fs != NULL) return fs; @@ -1049,7 +1053,11 @@ } /* record for the future */ #if IPFW2 +#ifdef I386 ((ipfw_insn_pipe *)cmd)->pipe_ptr = fs; +#else + bcopy(&fs, & ((ipfw_insn_pipe *)cmd)->pipe_ptr, sizeof(fs)); +#endif #else if (fs != NULL) rule->pipe_ptr = fs; Index: sys/netinet/ip_fw.h =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_fw.h,v retrieving revision 1.76 diff -u -r1.76 ip_fw.h --- sys/netinet/ip_fw.h 15 Mar 2003 01:13:00 -0000 1.76 +++ sys/netinet/ip_fw.h 11 May 2003 07:20:28 -0000 @@ -32,11 +32,16 @@ * The kernel representation of ipfw rules is made of a list of * 'instructions' (for all practical purposes equivalent to BPF * instructions), which specify which fields of the packet - * (or its metatada) should be analysed. + * (or its metadata) should be analysed. * * Each instruction is stored in a structure which begins with * "ipfw_insn", and can contain extra fields depending on the * instruction type (listed below). + * Note that the code is written so that individual instructions + * have a size which is a multiple of 32 bits. This means that, if + * such structures contain pointers or other 64-bit entities, + * (there is just one instance now) they may end up unaligned on + * 64-bit architectures, so the must be handled with care. * * "enum ipfw_opcodes" are the opcodes supported. We can have up * to 256 different opcodes. @@ -211,7 +216,7 @@ ipfw_insn o; union { struct in_addr ip; - int unit; + int32_t unit; } p; char name[IFNAMSIZ]; } ipfw_insn_if; @@ -220,10 +225,13 @@ * This is used for pipe and queue actions, which need to store * a single pointer (which can have different size on different * architectures. + * Note that, because of previous instructions, pipe_ptr might + * be unaligned in the overall structure, so it needs to be + * manipulated with care. */ typedef struct _ipfw_insn_pipe { ipfw_insn o; - void *pipe_ptr; + void *pipe_ptr; /* XXX */ } ipfw_insn_pipe; /* @@ -278,7 +286,9 @@ struct ip_fw { struct ip_fw *next; /* linked list of rules */ struct ip_fw *next_rule; /* ptr to next [skipto] rule */ +#if 0 /* passed up using 'next_rule' */ u_int32_t set_disable; /* disabled sets (for userland) */ +#endif u_int16_t act_ofs; /* offset of action in 32-bit units */ u_int16_t cmd_len; /* # of 32-bit words in cmd */ u_int16_t rulenum; /* rule number */ @@ -319,12 +329,12 @@ struct _ipfw_dyn_rule { ipfw_dyn_rule *next; /* linked list of rules. */ - struct ipfw_flow_id id; /* (masked) flow id */ struct ip_fw *rule; /* pointer to rule */ ipfw_dyn_rule *parent; /* pointer to parent rule */ - u_int32_t expire; /* expire time */ u_int64_t pcnt; /* packet match counter */ u_int64_t bcnt; /* byte match counter */ + struct ipfw_flow_id id; /* (masked) flow id */ + u_int32_t expire; /* expire time */ u_int32_t bucket; /* which bucket in hash table */ u_int32_t state; /* state of this rule (typically a * combination of TCP flags) @@ -334,7 +344,9 @@ /* to generate keepalives) */ u_int16_t dyn_type; /* rule type */ u_int16_t count; /* refcount */ +#if 0 /* passed up with 'rule' */ u_int16_t rulenum; /* rule number (for userland) */ +#endif }; /* Index: sys/netinet/ip_fw2.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_fw2.c,v retrieving revision 1.28 diff -u -r1.28 ip_fw2.c --- sys/netinet/ip_fw2.c 15 Mar 2003 01:13:00 -0000 1.28 +++ sys/netinet/ip_fw2.c 11 May 2003 07:21:40 -0000 @@ -2017,8 +2017,15 @@ if (cmd->o.opcode != O_PIPE && cmd->o.opcode != O_QUEUE) continue; - if (match == NULL || cmd->pipe_ptr == match) - cmd->pipe_ptr = NULL; + /* + * XXX Use bcmp/bzero to handle pipe_ptr to overcome + * possible alignment problems on 64-bit architectures. + * This code is seldom used so we do not worry too + * much about efficiency. + */ + if (match == NULL || + !bcmp(&cmd->pipe_ptr, &match, sizeof(match)) ) + bzero(&cmd->pipe_ptr, sizeof(cmd->pipe_ptr)); } } @@ -2559,7 +2566,8 @@ for (rule = layer3_chain; rule ; rule = rule->next) { int i = RULESIZE(rule); bcopy(rule, bp, i); - ((struct ip_fw *)bp)->set_disable = set_disable; + bcopy(&set_disable, &(bp->next_rule), + sizeof(set_disable)); bp = (struct ip_fw *)((char *)bp + i); } if (ipfw_dyn_v) { @@ -2571,7 +2579,8 @@ for ( p = ipfw_dyn_v[i] ; p != NULL ; p = p->next, dst++ ) { bcopy(p, dst, sizeof *p); - dst->rulenum = p->rule->rulenum; + bcopy(&(p->rule->rulenum), &(dst->rule), + sizeof(p->rule->rulenum)); /* * store a non-null value in "next". * The userland code will interpret a Index: sbin/ipfw/ipfw2.c =================================================================== RCS file: /home/ncvs/src/sbin/ipfw/ipfw2.c,v retrieving revision 1.23 diff -u -r1.23 ipfw2.c --- sbin/ipfw/ipfw2.c 15 Mar 2003 01:12:59 -0000 1.23 +++ sbin/ipfw/ipfw2.c 11 May 2003 09:00:26 -0000 @@ -813,8 +813,9 @@ int flags = 0; /* prerequisites */ ipfw_insn_log *logptr = NULL; /* set if we find an O_LOG */ int or_block = 0; /* we are in an or block */ + u_int32_t set_disable; - u_int32_t set_disable = rule->set_disable; + bcopy(rule->next_rule, &set_disable, sizeof(set_disable)); if (set_disable & (1 << rule->set)) { /* disabled */ if (!show_sets) @@ -1213,13 +1214,16 @@ { struct protoent *pe; struct in_addr a; + uint16_t rulenum; if (!do_expired) { if (!d->expire && !(d->dyn_type == O_LIMIT_PARENT)) return; } - printf("%05d %*llu %*llu (%ds)", d->rulenum, pcwidth, d->pcnt, bcwidth, + bcopy(d->rule, &rulenum, sizeof(rulenum)); + + printf("%05d %*llu %*llu (%ds)", rulenum, pcwidth, d->pcnt, bcwidth, d->bcnt, d->expire); switch (d->dyn_type) { case O_LIMIT_PARENT: @@ -1454,7 +1458,9 @@ err(EX_OSERR, "malloc"); if (getsockopt(s, IPPROTO_IP, IP_FW_GET, data, &nbytes) < 0) err(EX_OSERR, "getsockopt(IP_FW_GET)"); - set_disable = ((struct ip_fw *)data)->set_disable; + bcopy(((struct ip_fw *)data)->next_rule, + &set_disable, sizeof(set_disable)); + for (i = 0, msg = "disable" ; i < 31; i++) if ( (set_disable & (1<rulenum > rnum) + uint16_t rulenum; + + bcopy(d->rule, &rulenum, sizeof(rulenum)); + if (rulenum > rnum) break; - if (d->rulenum == rnum) + if (rulenum == rnum) show_dyn_ipfw(d, pcwidth, bcwidth); } } --qDbXVdCdHGoSgWSk-- From owner-freebsd-ipfw@FreeBSD.ORG Mon May 19 01:04:32 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8EFE137B401 for ; Mon, 19 May 2003 01:04:32 -0700 (PDT) Received: from garm.mip.co.za (garm.mip.co.za [209.212.106.45]) by mx1.FreeBSD.org (Postfix) with ESMTP id 459C643FBD for ; Mon, 19 May 2003 01:04:27 -0700 (PDT) (envelope-from patrick@mip.co.za) Received: (from root@localhost) by garm.mip.co.za (8.12.1/8.12.1) id h4J7pOSK016828; Mon, 19 May 2003 09:51:24 +0200 Received: from mip.co.za (PATRICK.mip.co.za [10.3.13.181]) by garm.mip.co.za (8.12.1/8.12.1) with ESMTP id h4J7pKh0016784; Mon, 19 May 2003 09:51:22 +0200 Message-ID: <3EC88FF4.8000604@mip.co.za> Date: Mon, 19 May 2003 10:04:04 +0200 From: "Patrick O'Reilly" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0) Gecko/20020530 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Josue References: <002601c31c7f$e4cbed60$1a0c10ac@armazemparaiba.com.br> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-scanner: scanned by Inflex 1.0.12.3 - (http://pldaniels.com/inflex/) cc: freebsd-ipfw@freebsd.org Subject: Re: PC300 on FreeBsd X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2003 08:04:32 -0000 Josue wrote: > Patrick. > > I installed a pc300 on system Freebsd 4.4 and 4.6 too but on reboot ther > board dont recognize. > > Can you help me? > > Josue Campelo - Brazil. Josue, Have you installed the PC300 driver and rebuilt the kernel? The driver for FreeBSD is available on the Cyclades website. Installation is quite simple, just follow their destructions... Patrick. ______________________________________ This e-mail may contain confidential information and may be legally privileged and is intended only for the person to whom it is addressed. If you are not the intended recipient, you are notified that you may not use, distribute or copy this document in any manner whatsoever. Kindly also notify the sender immediately by telephone, and delete the e-mail. When addressed to clients of the company from where this e-mail originates ("the sending company ") any opinion or advice contained in this e-mail is subject to the terms and conditions expressed in any applicable terms of business or client engagement letter . The sending company does not accept liability for any damage, loss or expense arising from this e-mail and/or from the accessing of any files attached to this e-mail. From owner-freebsd-ipfw@FreeBSD.ORG Mon May 19 06:01:39 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5327337B404 for ; Mon, 19 May 2003 06:01:39 -0700 (PDT) Received: from web12703.mail.yahoo.com (web12703.mail.yahoo.com [216.136.173.240]) by mx1.FreeBSD.org (Postfix) with SMTP id 064DF43FBD for ; Mon, 19 May 2003 06:01:39 -0700 (PDT) (envelope-from dsurovtsev@yahoo.com) Message-ID: <20030519130138.28972.qmail@web12703.mail.yahoo.com> Received: from [212.26.136.4] by web12703.mail.yahoo.com via HTTP; Mon, 19 May 2003 06:01:38 PDT Date: Mon, 19 May 2003 06:01:38 -0700 (PDT) From: dmitry surovtsev To: freebsd-ipfw@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: content-based firewall X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2003 13:01:39 -0000 Is there any content-based firewall for FreeBSD, i.e. firewall with capability to analize the packet content (at application level) as it does CheckPoint and Cisco's PIX? Maybe SNORT? Thanks. dmitry __________________________________ Do you Yahoo!? The New Yahoo! Search - Faster. Easier. Bingo. http://search.yahoo.com From owner-freebsd-ipfw@FreeBSD.ORG Mon May 19 06:53:41 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6601037B419 for ; Mon, 19 May 2003 06:53:41 -0700 (PDT) Received: from genua.rfc-networks.ie (genua.rfc-networks.ie [62.77.182.178]) by mx1.FreeBSD.org (Postfix) with ESMTP id 17E4743F3F for ; Mon, 19 May 2003 06:53:40 -0700 (PDT) (envelope-from philip.reynolds@rfc-networks.ie) Received: from tear.domain (unknown [10.0.1.254]) by genua.rfc-networks.ie (Postfix) with ESMTP id 685A054F5C for ; Mon, 19 May 2003 14:53:37 +0100 (IST) Received: by tear.domain (Postfix, from userid 1000) id 1B0CD21150; Mon, 19 May 2003 13:53:38 +0000 (GMT) Date: Mon, 19 May 2003 13:53:38 +0000 From: Philip Reynolds To: freebsd-ipfw@freebsd.org Message-ID: <20030519135338.GA61796@rfc-networks.ie> References: <20030519130138.28972.qmail@web12703.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030519130138.28972.qmail@web12703.mail.yahoo.com> X-Operating-System: FreeBSD 4.7-STABLE X-URL: http://www.rfc-networks.ie Subject: Re: content-based firewall X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: philip.reynolds@rfc-networks.ie List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2003 13:53:41 -0000 dmitry surovtsev 18 lines of wisdom included: > Is there any content-based firewall for FreeBSD, i.e. > firewall with capability to analize the packet content > (at application level) as it does CheckPoint and > Cisco's PIX? Maybe SNORT? I wrote a small application filter, for filtering application data to/from SMTP servers using ipfw's divert socket (actually, I used tee, so only a copy of the packet is sent to the divert socket). What kind of complexity do you need? Regards, -- Philip Reynolds | RFC Networks Ltd. philip.reynolds@rfc-networks.ie | +353 (0)1 8832063 http://people.rfc-networks.ie/~phil | www.rfc-networks.ie From owner-freebsd-ipfw@FreeBSD.ORG Mon May 19 08:26:06 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 412CF37B401 for ; Mon, 19 May 2003 08:26:06 -0700 (PDT) Received: from web12708.mail.yahoo.com (web12708.mail.yahoo.com [216.136.173.245]) by mx1.FreeBSD.org (Postfix) with SMTP id E4C0A43FBD for ; Mon, 19 May 2003 08:26:05 -0700 (PDT) (envelope-from dsurovtsev@yahoo.com) Message-ID: <20030519152605.2456.qmail@web12708.mail.yahoo.com> Received: from [212.26.136.4] by web12708.mail.yahoo.com via HTTP; Mon, 19 May 2003 08:26:05 PDT Date: Mon, 19 May 2003 08:26:05 -0700 (PDT) From: dmitry surovtsev To: philip.reynolds@rfc-networks.ie MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii cc: freebsd-ipfw@freebsd.org Subject: Re: content-based firewall X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2003 15:26:06 -0000 >I wrote a small application filter, for filtering >application data >to/from SMTP servers using ipfw's divert socket >(actually, I used >tee, so only a copy of the packet is sent to the >divert socket). > >What kind of complexity do you need? Thanks for reply. I need to control all smtp, pop, and http traffic. Where can I get your application filter? Is it's source code available as well? Regards, dmitry __________________________________ Do you Yahoo!? The New Yahoo! Search - Faster. Easier. Bingo. http://search.yahoo.com From owner-freebsd-ipfw@FreeBSD.ORG Mon May 19 09:12:07 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DE8F137B401 for ; Mon, 19 May 2003 09:12:07 -0700 (PDT) Received: from srv00.el.com.br (srv00.el.com.br [200.179.165.123]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8D02543FB1 for ; Mon, 19 May 2003 09:12:01 -0700 (PDT) (envelope-from g-paiva@el.com.br) Received: from intranet.el.com.br (srv00.el.com.br [200.179.165.123]) by srv00.el.com.br (elsmtp) with SMTP id B451D71039 for ; Mon, 19 May 2003 13:11:51 -0300 (BRT) Received: from 192.168.1.194 (SquirrelMail authenticated user g-paiva) by intranet.el.com.br with HTTP; Mon, 19 May 2003 13:11:51 -0300 (BRT) Message-ID: <1159.192.168.1.194.1053360711.squirrel@intranet.el.com.br> Date: Mon, 19 May 2003 13:11:51 -0300 (BRT) From: "Paiva, Gilson de" To: freebsd-ipfw@freebsd.org User-Agent: SquirrelMail/1.4.0 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 X-Priority: 3 Importance: Normal Subject: Annoying arp messages won't go away!! ( from freebsd-net@freebsd.org ) X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2003 16:12:08 -0000 Hi everybody, Sorry to cross-post from freebsd-net... I was running a freebsd machine bridging packets on 2 interfaces and acting as my internet router without any problem. Last week I had to change my IP allocation and, due ipfw2 improvements on layer2, I decided not to route packets through this machine anymore, but have a 3 nics bridge, then the annoying "/kernel: -- loop (x) xxarpxx to nicx from nicy (active)" is here :) . The moving arps are from the internet router - attached directly to ep0 - and a ras attached to xl0 . Freebsd keeps telling me the message with this 2 arps moving between its 3 nics. I understanding the arp and bridge basics very well and I think this problem has something to do with this 2 equipaments "scanning" my network with "arp who-has" ( detected with tcpdump ). I even "locked" all my 128 ips arps with arp -s and arp -s pub options but nothing changed. I tryied even to stop messages with net.link.ether.inet.log_arp_wrong_iface=0 , again no success. No google, no man pages, nothing I could do... Running 4.8-stable cvsuped and made world kernel at 15 this month, ipfw2, 3 nics with bridge on them. Did anyone have anything like this or do know any tip? I tried to make it simple, but I understand it's not that easy to mentally draw it. internet_router | | ep0 freebsd rl0 -- wireless network xl0 | | clients, servers and ras before: bridge with xl0 and rl0. This box had an ip used as gateway for internal clients. now: bridge on all nics. Servers and clients have their ip gateway pointed to internet_router. IP network is fine. This box has an ip so I can administer it. ifconfig -a Where's status: active from xl0 and ep0 ? Both are up and running fine... ( thanks rmkml@wanadoo.fr ) xl0: flags=8943 mtu 1500 inet 200.179.xxx.xxx netmask 0xffffff80 broadcast 200.179.xxx.xxx ether 00:60:97:70:12:ec media: Ethernet 10baseT/UTP (10baseT/UTP ) rl0: flags=8943 mtu 1500 ether 00:40:c7:78:06:45 media: Ethernet autoselect (100baseTX ) status: active ep0: flags=8943 mtu 1500 ether 00:60:08:2b:bc:29 media: Ethernet 10baseT/UTP I flushed all ipfw rules and loaded no custom sysctl value, problem remains the same.A piece of my sysctl.conf, network entries: net.inet.icmp.log_redirect=0 net.inet.ip.fastforwarding=1 net.inet.ip.forwarding=1 net.inet.ip.fw.enable=1 net.inet.ip.fw.one_pass=0 net.inet.ip.stealth=1 net.inet.tcp.blackhole=2 net.inet.tcp.keepidle=9000 net.inet.tcp.recvspace=65536 net.inet.tcp.sendspace=65536 net.inet.udp.blackhole=1 net.link.ether.bridge=1 net.link.ether.bridge_cfg=xl0,rl0,ep0 net.link.ether.bridge_ipfw=1 net.link.ether.inet.log_arp_wrong_iface=0 net.link.ether.ipfw=1 kernel conf ? options TCP_DROP_SYNFIN #drop TCP packets with SYN+FIN options ACCEPT_FILTER_DATA options ACCEPT_FILTER_HTTP options IPFW2 options IPFIREWALL #firewall options IPFIREWALL_VERBOSE #print information about options IPFIREWALL_FORWARD #enable transparent proxy support options IPFIREWALL_VERBOSE_LIMIT=100 #limit verbosity options IPFIREWALL_DEFAULT_TO_ACCEPT #allow everything by default options IPFILTER #ipfilter support options IPFILTER_LOG #ipfilter logging options IPDIVERT #divert sockets options IPSTEALTH #support for stealth forwarding options MROUTING #Multicast routing options DUMMYNET options HZ=1000 # strongly recommended options RANDOM_IP_ID options BRIDGE options IPSEC #IP security options IPSEC_ESP #IP security (crypto; define w/ IPSEC) options IPSEC_DEBUG #debug for IP security options ICMP_BANDLIM #Rate limit bad replies -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Paiva, Gilson de Domingos Martins mailto:npd@el.com.br Brazil http://www.el.com.br/ E&L Producoes de Software http://www.FreeBSD.org/ FreeBSD: The Power to Serve =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= ------------------------------------------------------------------------------ Aviso Legal: Esta mensagem pode nao expressar oficialmente as ideias ou vontades da empresa E&L Producoes de Software, sendo responsavel por esta exclusivamente seu autor. From owner-freebsd-ipfw@FreeBSD.ORG Mon May 19 11:01:27 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 530C037B401 for ; Mon, 19 May 2003 11:01:27 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id D313643F75 for ; Mon, 19 May 2003 11:01:26 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h4JI1QUp064323 for ; Mon, 19 May 2003 11:01:26 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h4JI1QKB064318 for ipfw@freebsd.org; Mon, 19 May 2003 11:01:26 -0700 (PDT) Date: Mon, 19 May 2003 11:01:26 -0700 (PDT) Message-Id: <200305191801.h4JI1QKB064318@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: ipfw@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2003 18:01:27 -0000 Current FreeBSD problem reports Critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2003/01/26] kern/47529 ipfw natd/ipfw lose TCP packets for firewalled o [2003/03/23] kern/50216 ipfw kernel panic on 5.0-current when use ipfw o [2003/04/28] kern/51485 ipfw "Fatal trap 12" from bridge code with ipf 3 problems total. Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2002/12/27] kern/46557 ipfw ipfw pipe show fails with lots of queues o [2003/04/18] kern/51132 ipfw kernel part of ipfw1 processes 'to not me o [2003/04/22] kern/51274 ipfw ipfw2 create dynamic rules with parent nu o [2003/04/24] kern/51341 ipfw ipfw rule 'deny icmp from any to any icmp 4 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- a [2001/04/13] kern/26534 ipfw Add an option to ipfw to log gid/uid of w f [2002/01/11] kern/33804 ipfw ipfw bug/problem o [2002/12/07] kern/46080 ipfw [PATCH] logamount in ipfw2 does not defau o [2002/12/10] kern/46159 ipfw ipfw dynamic rules lifetime feature o [2002/12/27] kern/46564 ipfw IPFilter and IPFW processing order is not o [2003/01/05] bin/46785 ipfw [patch] add sets information to ipfw2 -h o [2003/01/15] bin/47120 ipfw [patch] Sanity check in ipfw(8) o [2003/02/06] bin/48015 ipfw make ipfw2 work with iplen ranges o [2003/02/11] kern/48172 ipfw ipfw does not log size and flags o [2003/03/10] kern/49086 ipfw [patch] Make ipfw2 log to different syslo o [2003/03/12] bin/49959 ipfw ipfw tee port rule skips parsing next rul o [2003/04/09] bin/50749 ipfw ipfw2 incorrectly parses ports and port r o [2003/04/20] kern/51182 ipfw ipfw2. -d list shows couters for dynamic o [2003/05/04] bin/51750 ipfw ipfw2.c typos 14 problems total. From owner-freebsd-ipfw@FreeBSD.ORG Tue May 20 03:38:47 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AA74F37B401 for ; Tue, 20 May 2003 03:38:47 -0700 (PDT) Received: from mout1.freenet.de (mout1.freenet.de [194.97.50.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 20F3B43F85 for ; Tue, 20 May 2003 03:38:46 -0700 (PDT) (envelope-from ino-qc@spotteswoode.de.eu.org) Received: from [194.97.50.138] (helo=mx0.freenet.de) by mout1.freenet.de with asmtp (Exim 4.20) id 19I4W3-0002nO-V9 for freebsd-ipfw@freebsd.org; Tue, 20 May 2003 12:38:43 +0200 Received: from pd9501609.dip.t-dialin.net ([217.80.22.9] helo=spotteswoode.dnsalias.org) by mx0.freenet.de with asmtp (ID inode@freenet.de) (Exim 4.20 #1) id 19I4W1-0002GV-Ve for freebsd-ipfw@freebsd.org; Tue, 20 May 2003 12:38:42 +0200 Received: (qmail 2754 invoked by uid 0); 20 May 2003 10:38:41 -0000 Date: 20 May 2003 12:38:41 +0200 Message-ID: From: "clemens fischer" To: "dmitry surovtsev" In-Reply-To: <20030519152605.2456.qmail@web12708.mail.yahoo.com> (dmitry surovtsev's message of "Mon, 19 May 2003 08:26:05 -0700 (PDT)") References: <20030519152605.2456.qmail@web12708.mail.yahoo.com> User-Agent: Gnus/5.1002 (Gnus v5.10.2) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii cc: freebsd-ipfw@freebsd.org Subject: Re: content-based firewall X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2003 10:38:47 -0000 dmitry surovtsev : >>I wrote a small application filter, for filtering >>application data >>to/from SMTP servers using ipfw's divert socket >>(actually, I used >>tee, so only a copy of the packet is sent to the >>divert socket). >> >>What kind of complexity do you need? > > Thanks for reply. I need to control all smtp, pop, and > http traffic. Where can I get your application filter? > Is it's source code available as well? i am interested in this as well. the only thing in this context i remember are the accf_http(9) filters. is the divert-filtering code somewhere publicly accessable? clemens From owner-freebsd-ipfw@FreeBSD.ORG Tue May 20 05:53:52 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6DEC337B401; Tue, 20 May 2003 05:53:52 -0700 (PDT) Received: from srv00.el.com.br (srv00.el.com.br [200.179.165.123]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3CF7D43F3F; Tue, 20 May 2003 05:53:51 -0700 (PDT) (envelope-from g-paiva@el.com.br) Received: from intranet.el.com.br (srv00.el.com.br [200.179.165.123]) by srv00.el.com.br (elsmtp) with SMTP id EE75570E1E; Tue, 20 May 2003 09:53:46 -0300 (BRT) Received: from 192.168.1.194 (SquirrelMail authenticated user g-paiva) by intranet.el.com.br with HTTP; Tue, 20 May 2003 09:53:47 -0300 (BRT) Message-ID: <1254.192.168.1.194.1053435227.squirrel@intranet.el.com.br> In-Reply-To: <1159.192.168.1.194.1053360712.squirrel@intranet.el.com.br> References: <1159.192.168.1.194.1053360712.squirrel@intranet.el.com.br> Date: Tue, 20 May 2003 09:53:47 -0300 (BRT) From: "Paiva, Gilson de" To: freebsd-ipfw@freebsd.org User-Agent: SquirrelMail/1.4.0 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 X-Priority: 3 Importance: Normal cc: freebsd-net@freebsd.org Subject: Bridge + arp messages... X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2003 12:53:52 -0000 Hi everyone ( and sorry for the cross post ), Isn't this supposed to work ( bridge )? internet -- ep0 freebsd xl0 -- local clients Bridging itself works fine, but arp loop messages won't stop... ex: brdg0 /kernel: -- loop (1) 00.00.c8.b3.a5.7e to xl0 from rl0 (active) brdg0 /kernel: -- loop (2) 00.60.97.70.59.bc to rl0 from xl0 (active) freebsd: net.inet.ip.fw.enable=1 net.inet.ip.fw.one_pass=0 net.inet.tcp.blackhole=2 net.inet.tcp.keepidle=9000 net.inet.tcp.recvspace=65536 net.inet.tcp.sendspace=65536 net.inet.udp.blackhole=1 net.link.ether.bridge=1 net.link.ether.bridge_cfg=xl0,ep0 net.link.ether.bridge_ipfw=1 net.link.ether.inet.log_arp_wrong_iface=0 net.link.ether.ipfw=1 freebsd-stable 4.8 ( made 15/05 ) ipfw2 with "open" firewall. -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Paiva, Gilson de Domingos Martins mailto:npd@el.com.br Brazil http://www.el.com.br/ E&L Producoes de Software http://www.FreeBSD.org/ FreeBSD: The Power to Serve =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= ------------------------------------------------------------------------------ Aviso Legal: Esta mensagem pode nao expressar oficialmente as ideias ou vontades da empresa E&L Producoes de Software, sendo responsavel por esta exclusivamente seu autor. From owner-freebsd-ipfw@FreeBSD.ORG Tue May 20 11:32:30 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B8D9B37B401; Tue, 20 May 2003 11:32:30 -0700 (PDT) Received: from out003.verizon.net (out003pub.verizon.net [206.46.170.103]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6C3B443F85; Tue, 20 May 2003 11:32:29 -0700 (PDT) (envelope-from cswiger@mac.com) Received: from mac.com ([129.44.60.214]) by out003.verizon.net (InterMail vM.5.01.05.33 201-253-122-126-133-20030313) with ESMTP id <20030520183228.IGQQ2239.out003.verizon.net@mac.com>; Tue, 20 May 2003 13:32:28 -0500 Message-ID: <3ECA74B8.8010108@mac.com> Date: Tue, 20 May 2003 14:32:24 -0400 From: Chuck Swiger Organization: The Courts of Chaos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4b) Gecko/20030507 X-Accept-Language: en-us, en MIME-Version: 1.0 References: <1159.192.168.1.194.1053360712.squirrel@intranet.el.com.br> <1254.192.168.1.194.1053435227.squirrel@intranet.el.com.br> In-Reply-To: <1254.192.168.1.194.1053435227.squirrel@intranet.el.com.br> X-Enigmail-Version: 0.75.0.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH at out003.verizon.net from [129.44.60.214] at Tue, 20 May 2003 13:32:28 -0500 cc: freebsd-ipfw@freebsd.org cc: freebsd-net@freebsd.org Subject: Re: Bridge + arp messages... X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2003 18:32:31 -0000 Paiva, Gilson de wrote: [ ... ] > internet -- ep0 freebsd xl0 -- local clients > > Bridging itself works fine, but arp loop messages won't stop... > ex: > brdg0 /kernel: -- loop (1) 00.00.c8.b3.a5.7e to xl0 from rl0 (active) > brdg0 /kernel: -- loop (2) 00.60.97.70.59.bc to rl0 from xl0 (active) Try "sysctl net.link.ether.inet.log_arp_wrong_iface=0"... Just to be sure, ep0 and xl0 are on two seperate hubs which are not connected to each other, correct? I don't believe that FreeBSD supports the spanning tree algorithm to prevent bridging loops... -Chuck From owner-freebsd-ipfw@FreeBSD.ORG Tue May 20 11:34:14 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BC08837B401 for ; Tue, 20 May 2003 11:34:14 -0700 (PDT) Received: from genua.rfc-networks.ie (genua.rfc-networks.ie [62.77.182.178]) by mx1.FreeBSD.org (Postfix) with ESMTP id CCAC343F3F for ; Tue, 20 May 2003 11:34:13 -0700 (PDT) (envelope-from philip.reynolds@rfc-networks.ie) Received: from tear.domain (unknown [10.0.1.254]) by genua.rfc-networks.ie (Postfix) with ESMTP id 724D354F5C for ; Tue, 20 May 2003 19:34:12 +0100 (IST) Received: by tear.domain (Postfix, from userid 1000) id BE75F21150; Tue, 20 May 2003 18:34:11 +0000 (GMT) Date: Tue, 20 May 2003 18:34:11 +0000 From: Philip Reynolds To: freebsd-ipfw@freebsd.org Message-ID: <20030520183411.GC61796@rfc-networks.ie> References: <20030519152605.2456.qmail@web12708.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 4.7-STABLE X-URL: http://www.rfc-networks.ie Subject: Re: content-based firewall X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: philip.reynolds@rfc-networks.ie List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2003 18:34:15 -0000 clemens fischer 20 lines of wisdom included: > i am interested in this as well. the only thing in this context i > remember are the accf_http(9) filters. is the divert-filtering code > somewhere publicly accessable? It's actually part of a project I was working on which has to be demonstrated on May 27th. The code needs to be tidied up a good bit before then. When it is available it'll be placed on my webpage. The only thing I'd say is that this functionality and better is available by running a proper firewall ruleset and using internal mailservers and web proxies. Even a simple tcpdump running in the background would give you the same functionality, with a bit of sifting through it. However, I do see the need (or want) for something a little bit easier to debug and run. This is more of a proof-of-concept idea, but I'll certainly leave it available once it's done. Regards, -- Philip Reynolds | RFC Networks Ltd. philip.reynolds@rfc-networks.ie | +353 (0)1 8832063 http://people.rfc-networks.ie/~phil | www.rfc-networks.ie From owner-freebsd-ipfw@FreeBSD.ORG Thu May 22 06:31:49 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E0E8037B401 for ; Thu, 22 May 2003 06:31:49 -0700 (PDT) Received: from ptserver.progtech.net (pD9E89F66.dip.t-dialin.net [217.232.159.102]) by mx1.FreeBSD.org (Postfix) with ESMTP id 15E3B43F3F for ; Thu, 22 May 2003 06:31:48 -0700 (PDT) (envelope-from grossman@progtech.net) Received: from isis.muc.progtech.intern (isis.muc.progtech.intern [10.25.0.100]) by ptserver.progtech.net (8.12.9/8.12.3) with ESMTP id h4MDVo8e039767 for ; Thu, 22 May 2003 15:31:50 +0200 (CEST) (envelope-from grossman) Received: (from grossman@localhost) by isis.muc.progtech.intern (8.11.6/8.9.3) id h4MDVcM02552; Thu, 22 May 2003 15:31:38 +0200 (CEST) Date: Thu, 22 May 2003 15:31:38 +0200 (CEST) Message-Id: <200305221331.h4MDVcM02552@isis.muc.progtech.intern> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="T18iDemCs9" Content-Transfer-Encoding: 7bit From: Rolf Grossmann To: freebsd-ipfw@freebsd.org X-Mailer: VM 7.04 under Emacs 21.2.1 Subject: RFE/patch: Allow specification of default set number and number rules in ipfw2 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2003 13:31:50 -0000 --T18iDemCs9 Content-Type: text/plain; charset=us-ascii Content-Description: message body text Content-Transfer-Encoding: 7bit Hi, I have extended ipfw2 with 2 new options: 1. Allow a default set number to be specified for rules that don't have one. 2. Make ipfw number the rules read from a file instead of having the kernel pick the rule number. That way, I can load a firewall rules file into an empty and disabled set without having to specify numbers with every rule in the file and then atomically switch the old and new rules with ipfw swap. I'd now like to know, what are the chances of getting it integrated with FreeBSD? Do you think it's useful? Should I file a PR (so it's not forgotten, as it certainly won't go in before 5.1)? The attached patch is against the latest -STABLE, but should be no problem to fit with -CURRENT. Let me know what you think. Rolf --T18iDemCs9 Content-Type: text/plain Content-Disposition: inline; filename="ipfw2.patch" Content-Transfer-Encoding: 7bit Index: ipfw.8 =================================================================== RCS file: /export/server/freebsd/FreeBSD/cvs/cvs/src/sbin/ipfw/ipfw.8,v retrieving revision 1.63.2.33 diff -u -r1.63.2.33 ipfw.8 --- ipfw.8 4 Feb 2003 01:36:02 -0000 1.63.2.33 +++ ipfw.8 22 May 2003 12:31:43 -0000 @@ -54,7 +54,7 @@ .Op Ar number ... .Pp .Nm -.Op Fl q +.Op Fl qN .Oo .Fl p Ar preproc .Oo Fl D @@ -62,6 +62,7 @@ .Oc .Op Fl U Ar macro .Oc +.Op Fl S Ar set .Ar pathname .Sh DESCRIPTION The @@ -271,7 +272,24 @@ must be used. The file will be read line by line and applied as arguments to the .Nm -utility. +utility. By using +.Fl S Ar set +the default set number can be given for rules that don't specify one. +Also, with the +.Fl N +flag, one can shift the rule numbering from the kernel to the +.Nm +utility. Rules will be numbered relative to the previous rule using the +same stepsize as the kernel (taken from the sysctl variable +.Ar net.inet.ip.fw.autoinc_step ) . +If the first rule in the file is not explicitly numbered, it gets the +number the kernel will pick (so if no rule in the file is numbered, +the result will be the same with or without the option). Using this +option allows rules to be loaded into a different, possibly disabled, +set (e.g. using the +.Fl S +option), with numbers overlapping the existing rules by only giving a +number for the first rule in the set. .Pp Optionally, a preprocessor can be specified using .Fl p Ar preproc Index: ipfw2.c =================================================================== RCS file: /export/server/freebsd/FreeBSD/cvs/cvs/src/sbin/ipfw/ipfw2.c,v retrieving revision 1.4.2.12 diff -u -r1.4.2.12 ipfw2.c --- ipfw2.c 14 Apr 2003 12:41:37 -0000 1.4.2.12 +++ ipfw2.c 22 May 2003 12:53:37 -0000 @@ -66,6 +66,8 @@ do_expired, /* display expired dynamic rules */ do_compact, /* show rules in compact mode */ show_sets, /* display rule sets */ + dflt_set, /* default set to use in add rules */ + rule_increment, /* client-side rule number increment */ verbose; #define IP_MASK_ALL 0xffffffff @@ -2503,6 +2505,7 @@ * go into actbuf[]. */ static u_int32_t rulebuf[255], actbuf[255], cmdbuf[255]; + static int next_rulenum = 0; ipfw_insn *src, *dst, *cmd, *action, *prev; ipfw_insn *first_cmd; /* first match pattern */ @@ -2539,6 +2542,8 @@ av++; ac--; } + else if (rule_increment > 0) + rule->rulenum = next_rulenum; /* [set N] -- set number (0..30), optional */ if (ac > 1 && !strncmp(*av, "set", strlen(*av))) { @@ -2548,6 +2553,8 @@ rule->set = set; av += 2; ac -= 2; } + else + rule->set = dflt_set; /* [prob D] -- match probability, optional */ if (ac > 1 && !strncmp(*av, "prob", strlen(*av))) { @@ -3252,6 +3259,7 @@ err(EX_UNAVAILABLE, "getsockopt(%s)", "IP_FW_ADD"); if (!do_quiet) show_ipfw(rule, 10, 10); + next_rulenum = rule->rulenum + rule_increment; } static void @@ -3486,7 +3494,7 @@ pid_t preproc = 0; int c; - while ((c = getopt(ac, av, "D:U:p:q")) != -1) + while ((c = getopt(ac, av, "D:U:p:qNS:")) != -1) switch(c) { case 'D': if (!pflag) @@ -3519,6 +3527,19 @@ qflag = 1; break; + case 'N': + c = sizeof(rule_increment); + if (sysctlbyname("net.inet.ip.fw.autoinc_step", + &rule_increment, &c, NULL, 0)) + err(EX_UNAVAILABLE, "Can't get increment value"); + break; + + case 'S': + dflt_set = atoi(optarg); + if (dflt_set < 0 || dflt_set > 30) + errx(EX_DATAERR, "illegal set %s", optarg); + break; + default: errx(EX_USAGE, "bad arguments, for usage" " summary ``ipfw''"); @@ -3615,6 +3636,9 @@ s = socket(AF_INET, SOCK_RAW, IPPROTO_RAW); if (s < 0) err(EX_UNAVAILABLE, "socket"); + + dflt_set = 0; + rule_increment = 0; /* * If the last argument is an absolute pathname, interpret it --T18iDemCs9-- From owner-freebsd-ipfw@FreeBSD.ORG Fri May 23 00:48:12 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 679EC37B401; Fri, 23 May 2003 00:48:12 -0700 (PDT) Received: from mail.dannysplace.net (allxs.xs4all.nl [194.109.223.7]) by mx1.FreeBSD.org (Postfix) with ESMTP id 235F643FB1; Fri, 23 May 2003 00:48:11 -0700 (PDT) (envelope-from danny@dannysplace.net) Received: from [192.168.1.3] (helo=localhost) by mail.dannysplace.net with esmtp (Exim 4.12) id 19J7Hc-0002wj-00; Fri, 23 May 2003 09:48:08 +0200 Received: from pr2.ing.nl (pr2.ing.nl [145.221.92.41]) by www.dannysplace.com (Horde) with HTTP for ; Fri, 23 May 2003 09:48:07 +0200 Message-ID: <1053676087.95fbe1caf5dcd@www.dannysplace.com> Date: Fri, 23 May 2003 09:48:07 +0200 From: danny@dannysplace.net To: freebsd-ipfw@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs X-Scanner: exiscan for exim4 (http://duncanthrax.net/exiscan/) *19J7Hc-0002wj-00*rfOCa1XXKUU* cc: freebsd-questions@freebsd.org Subject: Strange natd problem. X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2003 07:48:12 -0000 Hello all, I have a strange problem. At home I have a standard lan adsl setup. LAN 192.168.10.x ---- FreeBSD 4.8 with ipfw and natd ---- ADSL modem ---- Internet. The natd setup here is what you would expect, 1 IP address and several clients. A few specific port redirects for web/mail etc... Now, this week I setup quake on a box sitting at a hosted location. It's running on FreeBSD 4.8 as well. But there is another firewall before it. It looks like this: (4.3.2.1 to 4.3.2.10) Internet --- FreeBSD 4.4 with ipfw and natd --- FreeBSD 4.8 (192.168.1.1) (192.168.1.250) The natd setup here is simple, there are 10 public IP addresses (4.3.2.1 to 4.3.2.10) and a natd config forwarding these to similar internal ip's (4.3.2.1 = 192.168.1.1 and so on...) natd.conf is: interface xl0 same_ports yes punch_fw 1000:500 log_denied yes log_facility security redirect_address 192.168.1.10 4.3.2.10 redirect_address 192.168.1.1 4.3.2.1 redirect_address 192.168.1.2 4.3.2.2 redirect_address 192.168.1.3 4.3.2.3 redirect_address 192.168.1.4 4.3.2.4 redirect_address 192.168.1.5 4.3.2.5 redirect_address 192.168.1.6 4.3.2.6 redirect_address 192.168.1.7 4.3.2.7 redirect_address 192.168.1.8 4.3.2.8 Now, the firwall for the quake server has the following pertainent rules. These are not the real rules, just an example. 100 divert 8668 ip from any to any via xl0 200 allow udp from any to 4.3.2.1 27960 200 allow udp from 4.3.2.1 27960 to any 300 allow ip from any to 4.3.2.1 27960 300 allow ip from 4.3.2.1 27960 to any 65435 allow icmp from any to any 65535 deny ip from any to any Here is the strange bit... rule 100 matches and (re-inserts) rules 200 never match rules 300 match and allow the quake packets through. I've tried the following protocols specifically. tcp,udp,icmp But it will ONLY match when I say "ip" So could it be that the firewall on my home lan does something with natd, then the firewall on the quake servers lan does something *else* to the packets there by screwing them up? Or does quake just use some strange ip protocol?. From owner-freebsd-ipfw@FreeBSD.ORG Fri May 23 02:39:25 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2DFA037B401; Fri, 23 May 2003 02:39:25 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id BF68C43F75; Fri, 23 May 2003 02:39:24 -0700 (PDT) (envelope-from maxim@FreeBSD.org) Received: from freefall.freebsd.org (maxim@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h4N9dOUp079053; Fri, 23 May 2003 02:39:24 -0700 (PDT) (envelope-from maxim@freefall.freebsd.org) Received: (from maxim@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h4N9dO0o079049; Fri, 23 May 2003 02:39:24 -0700 (PDT) Date: Fri, 23 May 2003 02:39:24 -0700 (PDT) From: Maxim Konovalov Message-Id: <200305230939.h4N9dO0o079049@freefall.freebsd.org> To: eric@beta.MIT.EDU, maxim@FreeBSD.org, ipfw@FreeBSD.org Subject: Re: kern/51485: "Fatal trap 12" from bridge code with ipfw enabled, when passing a traceroute. X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2003 09:39:25 -0000 Synopsis: "Fatal trap 12" from bridge code with ipfw enabled, when passing a traceroute. State-Changed-From-To: open->patched State-Changed-By: maxim State-Changed-When: Fri May 23 02:38:57 PDT 2003 State-Changed-Why: Fixed in rev. 1.63 sys/net/bridge.c in -CURRENT. http://www.freebsd.org/cgi/query-pr.cgi?pr=51485 From owner-freebsd-ipfw@FreeBSD.ORG Fri May 23 03:50:34 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0F19D37B401 for ; Fri, 23 May 2003 03:50:34 -0700 (PDT) Received: from epita.fr (hermes.epita.fr [163.5.255.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id D44E443FA3 for ; Fri, 23 May 2003 03:50:32 -0700 (PDT) (envelope-from le-hen_j@epita.fr) Received: from carpediem (carpediem.epita.fr [10.42.42.5]) by epita.fr id h4NAoVq01327 for ipfw@freebsd.org EPITA Paris France Fri, 23 May 2003 12:50:31 +0200 (MEST) Date: Fri, 23 May 2003 12:50:30 +0200 From: jeremie le-hen To: ipfw@freebsd.org Message-ID: <20030523105030.GA24992@carpediem.epita.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4i Subject: Understanding queue size X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2003 10:50:34 -0000 Hi, ipfw(8) manual page, section "TRAFFIC SHAPER (DUMMYNET) CONFIGURATION" tells: queue {slots | sizeKbytes} Queue size, in slots or KBytes. Default value is 50 slots, which is the typical queue size for Ethernet devices. Note that for slow speed links you should keep the queue size short or your traffic might be affected by a significant queueing delay. E.g., 50 max- sized ethernet packets (1500 bytes) mean 600Kbit or 20s of queue on a 30Kbit/s pipe. Even worse effect can result if you get packets from an interface with a much larger MTU, e.g. the loopback inter- face with its 16KB packets. I understand that if the queue size is too large, then it will imply large delays. But what if the queue is too short ? Does anyone can point me a detailed document which talks about queues ? Or maybe it exists a simple general rule explaining all this stuff... Regards, -- Jeremie aka TtZ/TataZ jeremie.le-hen@epita.fr From owner-freebsd-ipfw@FreeBSD.ORG Fri May 23 07:01:48 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2549937B401 for ; Fri, 23 May 2003 07:01:48 -0700 (PDT) Received: from epita.fr (hermes.epita.fr [163.5.255.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id A0FAE43F3F for ; Fri, 23 May 2003 07:01:44 -0700 (PDT) (envelope-from le-hen_j@epita.fr) Received: from carpediem (carpediem.epita.fr [10.42.42.5]) by epita.fr id h4NE1eq03063 Fri, 23 May 2003 16:01:40 +0200 (MEST) Date: Fri, 23 May 2003 16:01:40 +0200 From: jeremie le-hen To: Michael Sierchio Message-ID: <20030523140140.GB24992@carpediem.epita.fr> References: <20030523105030.GA24992@carpediem.epita.fr> <3ECE21BD.7060308@tenebras.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3ECE21BD.7060308@tenebras.com> User-Agent: Mutt/1.4i cc: ipfw@freebsd.org Subject: Re: Understanding queue size X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2003 14:01:48 -0000 > Why assume that the default values need adjusting? I'm trying to tune some parameters like queue size in order to reduce delays with interactive traffic while uploading. I tried to set various queue size with a pipe which bandwidth is 125KBit/s, from 2KBytes to 1000KBytes, and in fact I saw almost no responsivness variation. I currently use a queue size of 76KBytes, in order to send it in about 5 seconds at 125KBit/s. How do I know it's a good value ? I'm pretty sure there are some rules or some formulas which dictates how to get a good compromise between bandwidth and reactivy, but unfortunatelly, I don't know where to find them. Regards, -- Jeremie aka TtZ/TataZ jeremie.le-hen@epita.fr From owner-freebsd-ipfw@FreeBSD.ORG Fri May 23 07:15:42 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3D8DB37B401 for ; Fri, 23 May 2003 07:15:42 -0700 (PDT) Received: from laptop.tenebras.com (laptop.tenebras.com [66.92.188.18]) by mx1.FreeBSD.org (Postfix) with SMTP id 65B3943F85 for ; Fri, 23 May 2003 07:15:41 -0700 (PDT) (envelope-from kudzu@tenebras.com) Received: (qmail 42833 invoked from network); 23 May 2003 14:15:38 -0000 Received: from queequeg.tenebras.com (HELO tenebras.com) (192.168.188.241) by 0 with SMTP; 23 May 2003 14:15:38 -0000 Message-ID: <3ECE2D09.70609@tenebras.com> Date: Fri, 23 May 2003 07:15:37 -0700 From: Michael Sierchio User-Agent: Mozilla/5.0 (X11; U; Linux i386; en-US; rv:1.3) Gecko/20030312 X-Accept-Language: en-us, en, zh-cn, zh-tw MIME-Version: 1.0 To: jeremie le-hen References: <20030523105030.GA24992@carpediem.epita.fr> <3ECE21BD.7060308@tenebras.com> <20030523140140.GB24992@carpediem.epita.fr> In-Reply-To: <20030523140140.GB24992@carpediem.epita.fr> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: ipfw@freebsd.org Subject: Re: Understanding queue size X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2003 14:15:42 -0000 jeremie le-hen wrote: > I'm trying to tune some parameters like queue size in order to reduce delays > with interactive traffic while uploading. I tried to set various queue size > with a pipe which bandwidth is 125KBit/s, from 2KBytes to 1000KBytes, and > in fact I saw almost no responsivness variation. > > I currently use a queue size of 76KBytes, in order to send it in about 5 > seconds at 125KBit/s. How do I know it's a good value ? I'm pretty sure > there are some rules or some formulas which dictates how to get a good > compromise between bandwidth and reactivy, but unfortunatelly, I don't > know where to find them. Ah, you're adjusting the camel from the wrong end. ;-) It's not queue size you want to adjust. If you want to reduce delays for interactive traffic (telnet? ssh? etc.) assign that traffic to a different queue, and all the queues to the same pipe. Then you get the promised "worst case weighted fair queueing" Once you've done this, then you might consider decreasing the queue size on the interactive traffic. If it's full, it will get priority. I'd think in terms of numbers of packets rather than octets, probably... E.g. (my location) ipfw pipe 1 config bw 768000bit/s ipfw pipe 2 config bw 768000bit/s ipfw queue 11 config pipe 1 weight 10 ipfw queue 12 config pipe 2 weight 10 ipfw queue 13 config pipe 1 weight 50 ipfw queue 14 config pipe 2 weight 50 ipfw add queue 13 ip from $internal_hosts to any ssh,telnet out xmit $external_if ipfw add queue 11 ip from $internal_hosts to any not ssh,telnet out xmit $external_if You get the idea... From owner-freebsd-ipfw@FreeBSD.ORG Fri May 23 08:05:27 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4F21537B405 for ; Fri, 23 May 2003 08:05:27 -0700 (PDT) Received: from epita.fr (hermes.epita.fr [163.5.255.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1C13B43FB1 for ; Fri, 23 May 2003 08:05:26 -0700 (PDT) (envelope-from le-hen_j@epita.fr) Received: from carpediem (carpediem.epita.fr [10.42.42.5]) by epita.fr id h4NF5Iq15237 Fri, 23 May 2003 17:05:18 +0200 (MEST) Date: Fri, 23 May 2003 17:05:18 +0200 From: jeremie le-hen To: Michael Sierchio Message-ID: <20030523150517.GC24992@carpediem.epita.fr> References: <20030523105030.GA24992@carpediem.epita.fr> <3ECE21BD.7060308@tenebras.com> <20030523140140.GB24992@carpediem.epita.fr> <3ECE2D09.70609@tenebras.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3ECE2D09.70609@tenebras.com> User-Agent: Mutt/1.4i cc: ipfw@freebsd.org Subject: Re: Understanding queue size X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2003 15:05:27 -0000 > Ah, you're adjusting the camel from the wrong end. ;-) > > It's not queue size you want to adjust. If you want to reduce > delays for interactive traffic (telnet? ssh? etc.) assign that > traffic to a different queue, and all the queues to the same pipe. > Then you get the promised "worst case weighted fair queueing" In fact, I already have some queues in order to improve delays for interactive traffic. Here they are: pipe 1 config bw 125Kbit/s queue 76KBytes queue 10 config weight 100 pipe 1 mask all queue 20 config weight 100 pipe 1 queue 30 config weight 50 pipe 1 queue 40 config weight 1 pipe 1 # IRC, FTP, ICQ add queue 10 tcp from any to any 6667,5190,21 out via tun0 # SSH small packets (not file transfert via scp) add queue 10 tcp from any to any 22 iplen 1-1000 out via tun0 add queue 10 tcp from any 22 to any iplen 1-1000 out via tun0 # small ICMP, DNS requests and identd replies add queue 10 icmp from any to any iplen 1-200 out via tun0 add queue 10 udp from any to any 53 out via tun0 add queue 10 tcp from any 113 to any out via tun0 # TCP small packets (including SYNs, ACKs) add queue 20 tcp from any to any iplen 1-60 out via tun0 # SSH big packets (scp / rsync) add queue 40 tcp from any to any 22 iplen 1001-1500 out via tun0 add queue 40 tcp from any 22 to any iplen 1001-1500 out via tun0 # everything else add queue 30 tcp from any to any out via tun0 This just works fine, compared to when I didn't have any traffic shapping enabled. But I wanted to improve the responsivness of interactive traffic more and more, so I got concerned with queue size for the main pipe. > Once you've done this, then you might consider decreasing the > queue size on the interactive traffic. If it's full, it will > get priority. I'd think in terms of numbers of packets rather > than octets, probably... I hadn't understood I had to play with queue size for each queues, and not with the one of the main pipe. But that's the key ! :) Does reducing or increasing queue size of the main pipe have any relevance ? Finally, I should reduce size of interactive traffic queues, and increase size of the others. But how much ? Regards, -- Jeremie aka TtZ/TataZ jeremie.le-hen@epita.fr From owner-freebsd-ipfw@FreeBSD.ORG Fri May 23 08:52:56 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 75BAF37B401 for ; Fri, 23 May 2003 08:52:56 -0700 (PDT) Received: from btsoftware.com (213-84-82-9.adsl.xs4all.nl [213.84.82.9]) by mx1.FreeBSD.org (Postfix) with SMTP id A209343FA3 for ; Fri, 23 May 2003 08:52:54 -0700 (PDT) (envelope-from martin@btsoftware.com) Received: from btsoftware.com (viper [192.168.0.1] ) by btsoftware.com (Hethmon Brothers Smtpd) ; Fri, 23 May 2003 17:52:49 +0100 Message-Id: <200305231752.4942408.6@btsoftware.com> Received: from viper by btsoftware.com (Hethmon Brothers Pop3d) ; Fri, 23 May 2003 17:52:48 +0100 From: "Martin Bartelds" To: "ipfw@freebsd.org" Date: Fri, 23 May 2003 17:52:44 +0200 (CDT) Priority: Normal X-Mailer: PMMail 2.20.2382 for OS/2 Warp 4.5 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: Re: Understanding queue size X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Martin Bartelds List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2003 15:52:56 -0000 I think you should consider to use xmit/recv in stead of in/out via. This because the in/out via seems to be related to the direction of the TCP pipe and not the direction of (heavy) packet flow. You can test this using FTP transfers. You'll probably notice a huge difference in SSH response, if you initiate the transfer from your local system compared to an initiation from the outside, despite the heavy flow going out. You also might consider the consequences of http and active/passive FTP transfers. /Martin. On Fri, 23 May 2003 17:05:18 +0200, jeremie le-hen wrote: >> Ah, you're adjusting the camel from the wrong end. ;-) >> >> It's not queue size you want to adjust. If you want to reduce >> delays for interactive traffic (telnet? ssh? etc.) assign that >> traffic to a different queue, and all the queues to the same pipe. >> Then you get the promised "worst case weighted fair queueing" > >In fact, I already have some queues in order to improve delays for >interactive traffic. Here they are: > >pipe 1 config bw 125Kbit/s queue 76KBytes >queue 10 config weight 100 pipe 1 mask all >queue 20 config weight 100 pipe 1 >queue 30 config weight 50 pipe 1 >queue 40 config weight 1 pipe 1 > ># IRC, FTP, ICQ >add queue 10 tcp from any to any 6667,5190,21 out via tun0 ># SSH small packets (not file transfert via scp) >add queue 10 tcp from any to any 22 iplen 1-1000 out via tun0 >add queue 10 tcp from any 22 to any iplen 1-1000 out via tun0 ># small ICMP, DNS requests and identd replies >add queue 10 icmp from any to any iplen 1-200 out via tun0 >add queue 10 udp from any to any 53 out via tun0 >add queue 10 tcp from any 113 to any out via tun0 ># TCP small packets (including SYNs, ACKs) >add queue 20 tcp from any to any iplen 1-60 out via tun0 ># SSH big packets (scp / rsync) >add queue 40 tcp from any to any 22 iplen 1001-1500 out via tun0 >add queue 40 tcp from any 22 to any iplen 1001-1500 out via tun0 ># everything else >add queue 30 tcp from any to any out via tun0 > > >This just works fine, compared to when I didn't have any traffic shapping >enabled. But I wanted to improve the responsivness of interactive traffic >more and more, so I got concerned with queue size for the main pipe. > >> Once you've done this, then you might consider decreasing the >> queue size on the interactive traffic. If it's full, it will >> get priority. I'd think in terms of numbers of packets rather >> than octets, probably... > >I hadn't understood I had to play with queue size for each queues, and not >with the one of the main pipe. But that's the key ! :) > >Does reducing or increasing queue size of the main pipe have any relevance ? > >Finally, I should reduce size of interactive traffic queues, and increase >size of the others. But how much ? > >Regards, >-- >Jeremie aka TtZ/TataZ >jeremie.le-hen@epita.fr >_______________________________________________ >freebsd-ipfw@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw >To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" > From owner-freebsd-ipfw@FreeBSD.ORG Fri May 23 10:06:00 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7333E37B404 for ; Fri, 23 May 2003 10:06:00 -0700 (PDT) Received: from laptop.tenebras.com (laptop.tenebras.com [66.92.188.18]) by mx1.FreeBSD.org (Postfix) with SMTP id C192A43FA3 for ; Fri, 23 May 2003 10:05:59 -0700 (PDT) (envelope-from kudzu@tenebras.com) Received: (qmail 43314 invoked from network); 23 May 2003 17:05:58 -0000 Received: from queequeg.tenebras.com (HELO tenebras.com) (192.168.188.241) by 0 with SMTP; 23 May 2003 17:05:58 -0000 Message-ID: <3ECE54F5.3000301@tenebras.com> Date: Fri, 23 May 2003 10:05:57 -0700 From: Michael Sierchio User-Agent: Mozilla/5.0 (X11; U; Linux i386; en-US; rv:1.3) Gecko/20030312 X-Accept-Language: en-us, en, zh-cn, zh-tw MIME-Version: 1.0 To: jeremie le-hen References: <20030523105030.GA24992@carpediem.epita.fr> <3ECE21BD.7060308@tenebras.com> <20030523140140.GB24992@carpediem.epita.fr> <3ECE2D09.70609@tenebras.com> <20030523150517.GC24992@carpediem.epita.fr> In-Reply-To: <20030523150517.GC24992@carpediem.epita.fr> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: ipfw@freebsd.org Subject: Re: Understanding queue size X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2003 17:06:00 -0000 jeremie le-hen wrote: > I hadn't understood I had to play with queue size for each queues, and not > with the one of the main pipe. But that's the key ! :) > > Does reducing or increasing queue size of the main pipe have any relevance ? You should have pipes for incoming and outgoing traffic that are large enough (in terms of bw) for the burst/connect rate of your connection. Other than that, you may assign multiple queues to that pipe, and bandwidth will be allocated fairly among them based on their weights. > Finally, I should reduce size of interactive traffic queues, and increase > size of the others. But how much ? I'm not sure that this is really what you want to do... interactive latency starts getting to be perceptible at 0.125s and annoying at 0.5s, but this depends on MTU size and other things as well. One way of preventing a nearby or high-powered host from hogging all the bandwidth (that power user with the dual 2.8GHz desktop machine who's downloading mp3s all day) is to use RED or GRED. Do some reading on it before asking how to set the parameters, it's not an exact science. That being said, make the queue size the equivalent to the tolerable delay for interactive processes. You might consider using masks on the interactive queues, since interactive sessions can easily become bulk data transfer sessions (an ssh tunnel). And rememember: il faut garder quelques sourires pour se moquer des jours sans joie. From owner-freebsd-ipfw@FreeBSD.ORG Fri May 23 22:20:32 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 03EBD37B401 for ; Fri, 23 May 2003 22:20:32 -0700 (PDT) Received: from mx1.tekgenesis.net (main.lax1.tekgenesis.net [64.235.239.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6428943F3F for ; Fri, 23 May 2003 22:20:31 -0700 (PDT) (envelope-from jason@wiz.cx) Received: by mx1.tekgenesis.net (Postfix, from userid 105) id 2B7555C62; Fri, 23 May 2003 22:20:31 -0700 (PDT) Received: from webmail.tekgenesis.net (localhost [127.0.0.1]) by mx1.tekgenesis.net (Postfix) with SMTP id C7B575C5A for ; Fri, 23 May 2003 22:20:30 -0700 (PDT) Received: from a24b165n50client248.hawaii.rr.com ([24.165.50.248]) (SquirrelMail authenticated user wiz) by webmail.tekgenesis.net with HTTP; Fri, 23 May 2003 22:20:30 -0700 (PDT) Message-ID: <4156.24.165.50.248.1053753630.squirrel@webmail.tekgenesis.net> Date: Fri, 23 May 2003 22:20:30 -0700 (PDT) From: "Jason Dambrosio" To: freebsd-ipfw@freebsd.org User-Agent: SquirrelMail/1.4.0 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 X-Priority: 3 Importance: Normal X-Spam-Status: No, hits=-4.3 required=5.0 tests=BAYES_00,MSG_ID_ADDED_BY_MTA_3,PRIORITY_NO_NAME, UPPERCASE_25_50,USER_AGENT version=2.53 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.53 (1.174.2.15-2003-03-30-exp) Subject: ipfw2 broken in -current? X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 May 2003 05:20:32 -0000 # ipfw show 65535 2875 1377389 deny ip from any to any # ping lava.net PING lava.net (64.65.64.17): 56 data bytes 64 bytes from 64.65.64.17: icmp_seq=0 ttl=242 time=58.529 ms # ipfw add 100 divert natd ip from any to any via bge0 ipfw: getsockopt(IP_FW_ADD): Invalid argument ipfw: opcode 50 size 1 wrong # uname -a FreeBSD test-server 5.1-BETA FreeBSD 5.1-BETA #12: Fri May 23 18:11:41 HST 2003 I have: options IPDIVERT options IPSTEALTH options IPFIREWALL options IPFIREWALL_FORWARD options IPFIREWALL_VERBOSE options IPFIREWALL_VERBOSE_LIMIT=0 options IPFIREWALL_DEFAULT_TO_ACCEPT and sysctl net.inet.ip.forwarding=1 sysctl net.inet.ip.fastforwarding=1 sysctl net.inet.ip.stealth=1 Jason From owner-freebsd-ipfw@FreeBSD.ORG Sat May 24 01:59:51 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5175637B401 for ; Sat, 24 May 2003 01:59:51 -0700 (PDT) Received: from mail.dannysplace.net (allxs.xs4all.nl [194.109.223.7]) by mx1.FreeBSD.org (Postfix) with ESMTP id 15EE343F3F for ; Sat, 24 May 2003 01:59:50 -0700 (PDT) (envelope-from fbsd@dannysplace.net) Received: from [192.168.100.228] (helo=llama) by mail.dannysplace.net with smtp (Exim 4.12) id 19JUsW-0007ts-00 for freebsd-ipfw@freebsd.org; Sat, 24 May 2003 10:59:48 +0200 Message-ID: <000701c321d2$d32a7650$e464a8c0@llama> From: "Danny Carroll" To: Date: Sat, 24 May 2003 10:59:46 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 X-Scanner: exiscan for exim4 (http://duncanthrax.net/exiscan/) *19JUsW-0007ts-00*kpRTXxKNeo.* Subject: Strange Natd problem. X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Danny Carroll List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 May 2003 08:59:51 -0000 Hello all, I have a strange problem. At home I have a standard lan adsl setup. LAN 192.168.10.x ---- FreeBSD 4.8 with ipfw and natd ---- ADSL modem ---- Internet. The natd setup here is what you would expect, 1 IP address and several clients. A few specific port redirects for web/mail etc... Now, this week I setup quake on a box sitting at a hosted location. It's running on FreeBSD 4.8 as well. But there is another firewall before it. It looks like this: (4.3.2.1 to 4.3.2.10) Internet --- FreeBSD 4.4 with ipfw and natd --- FreeBSD 4.8 (192.168.1.1) (192.168.1.250) The natd setup here is simple, there are 10 public IP addresses (4.3.2.1 to 4.3.2.10) and a natd config forwarding these to similar internal ip's (4.3.2.1 = 192.168.1.1 and so on...) natd.conf is: interface xl0 same_ports yes punch_fw 1000:500 log_denied yes log_facility security redirect_address 192.168.1.10 4.3.2.10 redirect_address 192.168.1.1 4.3.2.1 redirect_address 192.168.1.2 4.3.2.2 redirect_address 192.168.1.3 4.3.2.3 redirect_address 192.168.1.4 4.3.2.4 redirect_address 192.168.1.5 4.3.2.5 redirect_address 192.168.1.6 4.3.2.6 redirect_address 192.168.1.7 4.3.2.7 redirect_address 192.168.1.8 4.3.2.8 Now, the firwall for the quake server has the following pertainent rules. These are not the real rules, just an example. 100 divert 8668 ip from any to any via xl0 200 allow udp from any to 4.3.2.1 27960 200 allow udp from 4.3.2.1 27960 to any 300 allow ip from any to 4.3.2.1 27960 300 allow ip from 4.3.2.1 27960 to any 65435 allow icmp from any to any 65535 deny ip from any to any Here is the strange bit... rule 100 matches and (re-inserts) rules 200 never match rules 300 match and allow the quake packets through. I've tried the following protocols specifically. tcp,udp,icmp But it will ONLY match when I say "ip" So could it be that the firewall on my home lan does something with natd, then the firewall on the quake servers lan does something *else* to the packets there by screwing them up? Or does quake just use some strange ip protocol?. From owner-freebsd-ipfw@FreeBSD.ORG Sat May 24 04:39:34 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4EB9A37B401 for ; Sat, 24 May 2003 04:39:34 -0700 (PDT) Received: from pasmtp.tele.dk (pasmtp.tele.dk [193.162.159.95]) by mx1.FreeBSD.org (Postfix) with ESMTP id 37B3743F3F for ; Sat, 24 May 2003 04:39:33 -0700 (PDT) (envelope-from elector@elector.dk) Received: from electorw (cpe.atm0-0-0-2211007.0x50a0337e.arcnxx6.customer.tele.dk [80.160.51.126]) by pasmtp.tele.dk (Postfix) with ESMTP id 3087DB4EC; Sat, 24 May 2003 13:39:26 +0200 (CEST) Message-ID: <200305241341140515.0EB4D5DA@pasmtp.tele.dk> X-Mailer: Calypso Version 3.30.00.00 (4) Date: Sat, 24 May 2003 13:41:14 +0200 From: "Elector" To: freebsd-ipfw@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable Subject: Prioritizing traffic and NAT. X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 May 2003 11:39:34 -0000 Hi there, first time I post here. But I'm soon to be hairless, if I didn't= try something else, than just reconfigging and reconfigging without= getting anywhere... I'll try to draw my setup here. Hope it looks alright/understandable. WAN | | FreeBSD | | | | PC1 PC2 PC1 and PC2 are on a 10.0.0.0/8 private IP-net. Fairly simple setup, where the FreeBSD NAT's PC1 and PC2 to the Internet,= and that works just fine.The problem is that I run a FTP server and a= couple of other services on the FreeBSD box, and when users are= downloading like nutts from it, PC1 and PC2 (and the FreeBSD box for that= matter) can barely use the Internet. So I would like to priortize the= traffic, as in giving PC1 and PC2 higher priority to the Internet than the= FreeBSD box itself. First of all, I'm just trying to prioritize outgoing traffic and incomming= apart from eachother. I know saturating outgoing traffic will slow down= incomming to a crawl, but that's not the biggest problem right now. First= I just want to get this to work, then I can worry about the other things= later. :-) My problem is that the config below, doesn't seem to prioritize anything,= well allmost anything. It works on the internal LAN traffic, and I can set= the speeds to whatever I want there and it works, BUT on the WAN side it= doesn't seem to work at all - that is no prioritizing whatsoever. I really REALLY hope someone here has a good sugestion. --- ipfw config file --- #first flush the old rules -f flush #set up queue for traffic on a 2M/512K ADSL connection. #outgoing traffic is the major bottleneck here, and I set it a bit lower= than max, to avoid saturating the line pipe 10 config bw 500Kbit/s pipe 20 config bw 2Mbit/s #LAN traffic should still run at 100Mbit pipe 99 config bw 100Mbit/s #outgoing traffic should be prioritized, so that the LAN PC's has has= higher priority than the router-box. queue 10 config pipe 10 weight 100 queue 20 config pipe 10 weight 1 #incomming traffic should be prioritized, so that the LAN PC's has has= higher priority than the router-box. queue 30 config pipe 20 weight 100 queue 40 config pipe 20 weight 1 #LAN traffic should still run at 100Mbit queue 99 config pipe 99 weight 1 #enable NAT/PAT add 50 divert natd all from any to any via rl0 add 100 allow ip from 10.0.0.0/8 to 10.0.0.0/8 #LAN traffic should still run at 100Mbit add 150 queue 99 ip from me to any out via fxp0 add 151 queue 99 ip from any to me in via fxp0 #LAN traffic to the outside world should be highest prioritized =3D get all= the BW they need. add 200 queue 10 ip from 10.0.0.0/8 to any in via fxp0 add 201 queue 30 ip from any to 10.0.0.0/8 out via fxp0 #box traffic to the ouside world (runs a FTP server among others) should be= lowest prioritized =3D only use #the BW that arn't used by the LAN PC's. add 300 queue 20 ip from me to any out via rl0 add 301 queue 40 ip from any to me in via rl0 #allow all traffic - Cisco PIX in front of box takes care of the actual= firewalling add 999 allow all from any to any -------------- From owner-freebsd-ipfw@FreeBSD.ORG Sat May 24 10:53:54 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4D9AB37B401 for ; Sat, 24 May 2003 10:53:54 -0700 (PDT) Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id 095A743F3F for ; Sat, 24 May 2003 10:53:51 -0700 (PDT) (envelope-from ru@whale.sunbay.crimea.ua) Received: from whale.sunbay.crimea.ua (ru@localhost [127.0.0.1]) h4OHrkEd044386 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 24 May 2003 20:53:47 +0300 (EEST) (envelope-from ru@whale.sunbay.crimea.ua) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.12.9/8.12.8/Submit) id h4OHriG1044381; Sat, 24 May 2003 20:53:44 +0300 (EEST) (envelope-from ru) Date: Sat, 24 May 2003 20:53:44 +0300 From: Ruslan Ermilov To: Jason Dambrosio Message-ID: <20030524175344.GB42456@sunbay.com> References: <4156.24.165.50.248.1053753630.squirrel@webmail.tekgenesis.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="DBIVS5p969aUjpLe" Content-Disposition: inline In-Reply-To: <4156.24.165.50.248.1053753630.squirrel@webmail.tekgenesis.net> User-Agent: Mutt/1.5.4i cc: freebsd-ipfw@freebsd.org Subject: Re: ipfw2 broken in -current? X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 May 2003 17:53:54 -0000 --DBIVS5p969aUjpLe Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, May 23, 2003 at 10:20:30PM -0700, Jason Dambrosio wrote: > # ipfw show > 65535 2875 1377389 deny ip from any to any > # ping lava.net > PING lava.net (64.65.64.17): 56 data bytes > 64 bytes from 64.65.64.17: icmp_seq=3D0 ttl=3D242 time=3D58.529 ms > # ipfw add 100 divert natd ip from any to any via bge0 > ipfw: getsockopt(IP_FW_ADD): Invalid argument > ipfw: opcode 50 size 1 wrong > # uname -a > FreeBSD test-server 5.1-BETA FreeBSD 5.1-BETA #12: Fri May 23 18:11:41 HS= T 2003 >=20 > I have: >=20 > options IPDIVERT > options IPSTEALTH > options IPFIREWALL > options IPFIREWALL_FORWARD > options IPFIREWALL_VERBOSE > options IPFIREWALL_VERBOSE_LIMIT=3D0 > options IPFIREWALL_DEFAULT_TO_ACCEPT >=20 > and >=20 > sysctl net.inet.ip.forwarding=3D1 > sysctl net.inet.ip.fastforwarding=3D1 > sysctl net.inet.ip.stealth=3D1 >=20 grep ipfw /var/run/dmesg.boot, if it says "divert disabled" it means that you forgot to recompile/reinstall your kernel properly with the "options IPDIVERT". Cheers, --=20 Ruslan Ermilov Sysadmin and DBA, ru@sunbay.com Sunbay Software AG, ru@FreeBSD.org FreeBSD committer, +380.652.512.251 Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age --DBIVS5p969aUjpLe Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (FreeBSD) iD8DBQE+z7GoUkv4P6juNwoRAkkwAJ99hOXKKvFBA77KVguW41IMcIcv9ACbBrhl xbW/+kvzJnByGEqQL2k9vBA= =PH0F -----END PGP SIGNATURE----- --DBIVS5p969aUjpLe--