From owner-freebsd-net@FreeBSD.ORG Wed Mar 9 12:08:54 2005 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3DC9C16A4CE for ; Wed, 9 Mar 2005 12:08:54 +0000 (GMT) Received: from mail.astra-sw.com (mail.astra-sw.com [82.140.87.237]) by mx1.FreeBSD.org (Postfix) with ESMTP id 55C3043D4C for ; Wed, 9 Mar 2005 12:08:53 +0000 (GMT) (envelope-from Nickolay.Kritsky@astra-sw.com) Received: from exchange.stardevelopers4msi.com ([192.168.64.10]) by mail.astra-sw.com (8.12.11/8.12.11) with ESMTP id j29C8qtF070924 for ; Wed, 9 Mar 2005 15:08:52 +0300 (MSK) Content-Class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable Date: Wed, 9 Mar 2005 15:12:01 +0300 X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: multiple uplinks from ISP thread-index: AcUknogz3NIbLGUpQF+PuP9H1w4tmAAAg5cA From: "Nickolay Kritsky" To: "mc" , Subject: RE: multiple uplinks from ISP X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Mar 2005 12:08:54 -0000 Getting VMware network to network can be hard. But that's life: no pain, = no gain. Again, you can ask quagga port man: boris@tagnet.ru . I think = he knows a lot about multipath routing with or without quagga. PS: to all -net people. I think that such question is quite often here. = Maybe we can add short chapter about multipath routing in the handbook? = Explaining if it is possible, and if not, why. -----Original Message----- From: mc [mailto:mc@netx.com.hk] Sent: Wednesday, March 09, 2005 2:53 PM To: Nickolay Kritsky; freebsd-net@freebsd.org Subject: Re: multiple uplinks from ISP In fact, the biggest problem with me is that I don't have any = development=20 machines for building a test network, in other words I cannot do = experiments=20 at anytime I want. I usually need to plan the experiment in details and = do=20 the experiment with some idle hot backup machine in the network, or I = can=20 also use things like VMware to setup a testing network, but getting = VMware's=20 network to work as expected is headache.... outgoing/incoming ratio: its the reverse actually :P. out/in is roughly=20 equals to 10:1, usually 12Mbps incoming and 99Mbps outgoing. actually = most=20 the traffic is just generated by a single web server. > If it's like 1:10, maybe you won't need multipath routing. You will = use=20 only one interface for sending packets, and you will get them back via = two=20 interfaces. Think about it. In this case - everything that you need is = two=20 equal-cost routes to your network on the ISP side. Remember the KISS = idea=20 :-) imho this setup have several drawbacks...at least if the sending link = fails,=20 the packets would not automatically go to the other interface. also,=20 firewalling could be made difficult if the packets are distributed like=20 this. ----- Original Message -----=20 From: "Nickolay Kritsky" To: "mc" ; Sent: Wednesday, March 09, 2005 19:24 Subject: RE: multiple uplinks from ISP 1. Yes I think that should be enough. 2. Um, that's a good question. I guess I don't know the answer. you can ask quagga maintainer about the details of quagga multipath = routing.=20 Maybe it just changes the gateway, say 10 times in a sec? Maybe it = patches=20 kernel binary code, who knows? The best way to know would be to build = some=20 test environment. What you need is two machines with 3 interfaces each. = One=20 would emulate the ISP side, one will be your side. and test. Plug them = in=20 between of some IP link and see what happens with tcpdump and other = tools. And, as it suddenly came to my mind, there is another question: what is = your=20 outgoing/incoming traffic ratio? If it's like 1:10, maybe you won't need = multipath routing. You will use only one interface for sending packets, = and=20 you will get them back via two interfaces. Think about it. In this case = -=20 everything that you need is two equal-cost routes to your network on the = ISP=20 side. Remember the KISS idea :-) Nick -----Original Message----- From: mc [mailto:mc@netx.com.hk] Sent: Wednesday, March 09, 2005 1:58 PM To: Nickolay Kritsky; freebsd-net@freebsd.org Subject: Re: multiple uplinks from ISP dst-ip is not supported on one side of the switch. src-mac does not work too, due to the fact that this would lead to a = biased result, causing most of the traffic goes thru the first link. dst-mac would not work as the machine is sending traffic to a single = router. > fxp0: 1.2.3.1/30 > fxp1: 1.2.3.5/30 > em0: 10.123.123.102/24 Does this imply I just need to ask my ISP for two /30 and two default gateways and that's it? No other 'special' configuration or registration procedures would be needed? One more question, did you mean if I am to use quagga as the bgp daemon, = I don't need to apply some kernel patches for the eq cost multipath to = work? 'coz if my memory serves, quagga or other routing daemons just insert/delete/update the route entries in the kernel, they do not take = part in any packet routing decisions. ----- Original Message -----=20 From: "Nickolay Kritsky" To: "mc" ; Sent: Wednesday, March 09, 2005 18:35 Subject: RE: multiple uplinks from ISP Why can't you use dst-ip hashing? You are using /24 network for your = client machines, no? If FEC uses IP addresses for hashing that you are ok. If = it uses MAC addresses for hashing, you need to test something else. Regarding your initial post here is my proposal: fxp0: 1.2.3.1/30 fxp1: 1.2.3.5/30 em0: 10.123.123.102/24 Your ISP gives you 2 more /30 nets for your uplinks You should have two default gateways on fxp0 and fxp1 (1.2.3.2 and = 1.2.3.6 respectively) ISP AS should have two routes to your network with the same weight. Problem: FreeBSD natively does not support two different routes to the = same destination. AFAIK this is by design. Solution: It can be solved using custom patch (I think I have seen such = for 4.x systems) or using external routing daemon like quagga. Nick -----Original Message----- From: mc [mailto:mc@netx.com.hk] Sent: Wednesday, March 09, 2005 1:06 PM To: freebsd-net@freebsd.org Subject: Re: multiple uplinks from ISP Hi, I am using cisco 29xx and 3xxx switches. The problem with FEC is that I = have no way to use dst-ip hashing as the load balancing option on these two switches, and that would cause biased utilization on a certain link = only, i.e. impossible to utilize 2*100=3D200Mbps. and...if I were really to use FEC as the solution, I will need to get = some much expensive switches from cisco, which is quite unaffordable and imho unnecessary in fact... ----- Original Message -----=20 From: "Nickolay Kritsky" To: "mc" ; Sent: Wednesday, March 09, 2005 17:58 Subject: RE: multiple uplinks from ISP hello I do not think you should mess a lot with interdomain routing here. Such = a scenario (multiple uplinks from the same ISP) IMHO is better be solved = on the layer 2. What you need is some technology that utilizes two Ethernet ports at = once. About a week or two ago on this list was discussed similar setup using = Cisco technology. Search for subject "ng_fec and Cisco 2931". I f your ISP is using the switch/router that supports FEC, you could do this trick. Also most 3com intelligent switches support aggregating links via multiple 100Mbit channels. If you have put 3com equipment on both sides of your internet connection you'll can get what you want. Hope that helps. BTW the first and best thing to do is to ask such question to your ISP. Nick -----Original Message----- From: mc [mailto:mc@netx.com.hk] Sent: Wednesday, March 09, 2005 12:32 PM To: freebsd-net@freebsd.org Subject: Re: multiple uplinks from ISP Hi, The main problem is that I have no idea at all how should I setup everything..and what do I need from my ISP......I just know it was = possible, but I can't recall the details inside, and a simple google did not = return anything helpful to me. I agree with you that fbsd (or any other linux) is much better than = cisco in terms of stability. The cisco routers at my site are crashing like cron = jobs while the fbsd boxes usually have long uptimes. :) off topic: I used to be a network admin some time ago, but no longer = true now....and unfortunately, in the past I had only very few chances to = deal with interdomain routing, mainly in lab. I'm afraid I have forgotten everything by now :( ----- Original Message -----=20 From: "Goran Gajic" To: Sent: Wednesday, March 09, 2005 6:01 Subject: Re: multiple uplinks from ISP > > Hi, > > I have used succesfuly FBSD 5.2.1 as BGP router and it is rock stable = with > quagga (check out www.quagga.net) - more stable then 30k $ Cisco 7206 = :)) > Problem is if you have AS and LIR and if you don't there are other > solutions. Of course much depends is your uplink ISP willing to = cooperate. > > Regards, > gg. > > > >> Hi all, >> >> If I have the following on hand... >> - 2 FastEthernet uplinks from ISP >> - 1 GigabitEthernet port on my switch >> - a subset of a /24 allocated by ISP >> The gigabit ethernet link should be connecting to my internal = network. > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"