Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 3 May 2001 21:45:11 -0700
From:      "Ted Mittelstaedt" <>
To:        "Wai Chan" <>
Cc:        <freebsd-net@FreeBSD.ORG>, <freebsd-questions@FreeBSD.ORG>
Subject:   RE: outgoing traffic load balancing with multiple ISP
Message-ID:  <000001c0d455$00a50fa0$>
In-Reply-To: <>

Next in thread | Previous in thread | Raw E-Mail | Index | Archive | Help
Your in a bad spot there, because even if you stuff
the pipe to ISP#1, then how much additional bandwidth
you get out of them is entirely dependent on how stuffed THEIR
pipe is to ISP#2.  Also, unless ISP#2 has a bigger feed to
the core than you have to them, you won't gain anything.

But, I have an idea that I'll suggest that might possibly make some
sense here.

What you go do is you set yourself up as a _transit_ AS.
Then you go to ISP#1 and offer to route THEIR traffic
through you as an alternate route to ISP#2.

This gains ISP#1 multiple paths to ISP#2 - ie: More
Bandwidth.  Plus, it gives them a redundant backup in case their
direct link to ISP#2 goes down.

In exchange for this ISP#1 agrees to credit your bill to
zero - in effect, you are now serving as a feed to them
that costs them nothing.

ISP#1 then pre-pends it's BGP advertisement towards ISP#2 an
extra hop, and thus now you have a situation where traffic
to ISP#1 is load-balanced, thus it's like they get double the
bandwidth to their feed, and you of course get your bill reduced
to just the circuit costs of maintaining the link.

Naturally of course your feeds both are going to become very
busy carrying all the extra traffic of ISP#1's, but in the
situation you just described, your going to end up getting
just as much bandwidth access to the Internet core as you
would if you were able to load-balance between both ISP's.

Later on, of course, once your contract to ISP#2 expires,
you can simply get a feed to, say, ISP#3 and provide an
even better service to ISP#1, as well as to yourself.

Now, understandably ISP#1 may be reluctant to get involved in
something like this.  But, your giving them something that
they need, and furthermore your kind of cementing your
relationship to them.  If they ultimately balk, then they
end up with a pissed-off customer who will be telling everyone around
town how terrible they are, (and losing the revenue, because you
might just decide to stop paying them and then they have no choice
but to sell it to collections for 10 cents on the dollar, or
hire a lawyer and burn up all the money they would make off of you
in legal fees)  Ultimately in this situation if you both cooperate
then all of you will be better off.

About the only other thing that you might be able to do, assuming that
you have your original contracts, is to see if there's a loophole
which would allow you out of the contract for ISP#1.  For example
if there's a Service Level Agreement.   But in my experience if
you try this your going to destroy any chance of a future win-win
relationship with ISP#1.

Ted Mittelstaedt            
Author of:          The FreeBSD Corporate Networker's Guide
Book website:

>-----Original Message-----
>From: owner-freebsd-questions@FreeBSD.ORG
>[mailto:owner-freebsd-questions@FreeBSD.ORG]On Behalf Of Wai Chan
>Sent: Thursday, May 03, 2001 6:26 PM
>To: Ted Mittelstaedt
>Cc: freebsd-net@FreeBSD.ORG; freebsd-questions@FreeBSD.ORG
>Subject: RE: outgoing traffic load balancing with multiple ISP
>Our old network admin signed a stupid 5 years contract with ISP 1
>(the ISP 1
>uses ISP 2 for next hop), and we added ISP 2 when the new network admin
>arrived (this network admin is smarter becuase he only signed 1 year
>contract with ISP 2).  We cannot get rid of ISP 1 because of the stupid
>contract.  Anyway, stand BGP/route load balancing doesn't work
>very well for
>us because all the traffic goes out and in through ISP 2.
>We don't want to leave ISP 1's pipe empty.  If the outgoing
>traffic is using
>the IP provided by ISP 1, then the returned traffic will be using ISP 1
>provided pipe.  It applies to ISP 2 also.  That's why I am trying to force
>half of the traffic (http) use ISP 1 provided IP, and the other
>half use ISP
>2 provided IP.
>best wishes,
>Wai Chan
>-----Original Message-----
>From: Ted Mittelstaedt []
>Sent: Thursday, May 03, 2001 02:36 PM
>To: Wai Chan
>Cc: freebsd-net@FreeBSD.ORG; freebsd-questions@FreeBSD.ORG
>Subject: Re: outgoing traffic load balancing with multiple ISP
>Quoting Wai Chan <>:
>> I have two ISPs (two different serial links to my router).  I want 50%
>> of
>> all outgoing traffic go through ISP 1 with ISP 1 provided IP address as
>> source address, and the other 50% of all outgoing traffic go through ISP
>> 2
>> with ISP 2 provided IP address as source address.
>Is there any reason you can't just build a second router and plug
>the serial link from the second ISP into it, then take the inside
>interfaces of both routers and plug them into the same hub, and
>run multiple
>address ranges on the hub?  Then put half of your systems on one IP range
>and half of them on the other.
>I realize that your not going to divide the traffic up 50% this way but
>the problem is (and was just thashed over in this list less than a week
>ago) that ISP #1 cannot route IP numbers supplied by ISP #2, and ISP #2
>cannot route IP numbers supplied by ISP #1, unless you have been given
>entire netblocks by both ISP's and are running BGP with both and are
>advertising those netblocks.  Even then, load balancing is a tricky problem
>because it's almost entirely dependent on the destination IP numbers that
>traffic from your servers is going to.  If you were multihomed
>with your own
>in the manner you would find that in most cases, traffic is going to favor
>interface over the other.  Generally, if your careful in picking your feeds
>it's not going to be worse than 60/40 one way or the other, but if you do
>something like using a very well connected network for one link
>and a poorly
>connected network for the other, it can be as bad as 90/10.
>Have you investigated bonding or multilink with one of your ISP's?  There's
>also a way to multiplex T1's together that some ISP's support.
>Ted Mittelstaedt            
>Author of:          The FreeBSD Corporate Networker's Guide
>Book website:
>To Unsubscribe: send mail to
>with "unsubscribe freebsd-questions" in the body of the message

To Unsubscribe: send mail to
with "unsubscribe freebsd-questions" in the body of the message

Want to link to this message? Use this URL: <$00a50fa0$1401a8c0>