Date: 22 May 2002 12:36:14 +0400 From: "Vladimir B. " Grebenschikov <vova@sw.ru> To: Oleg Chebotarev <chebotarev@yahoo.com> Cc: freebsd-current@FreeBSD.org Subject: Re: multi default routes in freebsd !? Message-ID: <1022056574.460.30.camel@vbook.express.ru> In-Reply-To: <3CEAB378.D4B7F9E@mindspring.com> References: <20020521032746.14287.qmail@web11607.mail.yahoo.com> <3CE9DD37.88DDCABC@mindspring.com> <1021965925.557.9.camel@vbook.express.ru> <3CEA0FC0.E71EC770@mindspring.com> <1021977292.24428.30.camel@vbook.express.ru> <3CEAB378.D4B7F9E@mindspring.com>
next in thread | previous in thread | raw e-mail | index | archive | help
=F7 Wed, 22.05.2002, =D7 00:52, Terry Lambert =CE=C1=D0=C9=D3=C1=CC:
> "Vladimir B. Grebenschikov" wrote:
> > > Multipath routing is not as useful as you imply. Neither is
> > > round-robin'ing between a set of paths. It assumes that the
> > > pool retention time on the router is longer than the drain time
> > > for a single path, such that you end up with a higher aggregate
> > > throughput than you would otherwise get. Most of the time,
> > > with what you are suggesting, you will get the same throughput,
> > > you will just get differential pipe utilization (using B =3D=3D !A).
> > > When this isn't the case, the amount of latency for a single
> > > path is such that you end up with only a small fractional
> > > improvement, when there is any improvement at all.
> >=20
> > Lets imagine - we have 3 links 2Mbit/s on different interfaces.
> > I want to join them all, but I have no control of other end (provider)
> > so I can't build netgraph-joiner.
> >=20
> > Solution with installing 3 routes (through BGP of course, one BGP
> > session per link) solves problem.
> >=20
> > I have 6 Mbit/s summary bandwith.
>=20
> No, you don't.
>=20
> Without the cooperation of the tother end, you don't have
> control of the symmetry of the return route. So maybe
> your packets are round-robin'ed out interfaces, but they
> all come back through the same interface, because you have
> no control of the other end.
On other end I have providers CISCO router _with_ standard BGP multipath
feature, so I have symmetric situation.
> The only way around this is to have the default routes outbound
> be totally seperate from the inbound (don't eat your inbound
> bandwidth witho outbound load). The unidirectional troughput
> goes to "1", and your total throughput doubles. But it never
> gets beyond double.
I do not talk about _default_ all I have said can right for any set of
routes.
> This is why BGP is a better deal: it implicitly enlists the
> cooperation of the other end of the link.
Once again, BGP can deal with multipath, it can utilize multipath if we
have more then one link with same weight/etc.
> > > The primary failure of this is that it can't detect when a
> > > route goes down, so you are screwed when that happens.
> >=20
> > If interface goes down route will be DOWN by kernel.
> > So it is not problem.
>=20
> No.
>=20
> ,---------------.
> | BOX WITH TWO |
> ,-------| DEF. ROUTES |-------.
> route #1 | `---------------' | route #2
> | |
> ,---------------. ,---------------.
> | ROUTER "A" | | ROUTER "B" |
> `---------------' `---------------'
> | |
> | |
> good link dead link
>=20
> The link between the box and "router B" remains up. Therefore the
> box fails to note that packets sent via "route #2" never get to
> their destination.
No, if link BOX<->routerB fails kernel must down one of two routes with
same prefix (you said default).
If from side of BOX it is noot seen thar carrier (whatever) on link
BOX<->routerB down - BGP session over this link will down so, BGP
software will down route.
I am not happy with hack pach when one route have more then one gateway,
may opinion to allow insert more then one full-featured route to one
prefix into kernel FIB, but it is implementation issue.
> It's ridiculous to think that the interface for "route #2" will
> somehow "magically" down itself on "box" because, several hops
> down the line, the line is dead.
If you have direct link, say serial - kernel will detect interface down
easily, if you have some kind of multihop - multihop BGP will save
situation, but it is unwise scheme (I think). So in this case you better
to build:
,---------------.
| BOX WITH TWO |
,-------| DEF. ROUTES |-------.
route #1 | `---------------' | route #2
| |
,---------------. ,---------------.
| ROUTER "A" | | ROUTER "B" |
`---------------' `---------------'
| | link X
| | BGP here
good link ,----------------.
| | Another router |
| `----------------'
routes source ----------------------/
=20
So all routes comes from "routes source" (It can be core router of
provider with full view or default issuing router)
And, if "link X" fails set of routes (or only default) will disappear
from BOX<->routerB link, and BGP software will remove second path
routes.
> > Anyway if problem happens without downing interface BGP will detect
> > problem and down routes.
>=20
> Only if you are using BGP. And if you are using BGP, you don't
> need the hack you want, BGP will take care of it for you.
Again multipath IS in BGP concept not it is FreeBSD kernel hack for BGP
because can't do multipath.
> >=20
> > It is usual practice to use, say OSPF for internal routing (inside one
> > AS). Yes I understand that some netgraph solutions can help
> > (multilink PPP or ng_one2many, I am extensively use netgraph)
> > But if routing protocol there solution for it with alid link down
> > detection and so why we need to use some "workarounds" to emulate
> > protocol behavior ?
>=20
> You need to read the "README":
>=20
> ftp://ftp.flirble.org/pub/unix/hacks/FreeBSD/README.MPATH
>=20
> Specifically, you need to read:
>=20
> WHAT IT DOES NOT DO
>=20
> It doesn't detect when remote hosts are down. This is not
> the job of the kernel. It's not a routing protocol, it's
> not an automatic failover system.
>=20
> So it is not a "routing protocol solution" in any sense of things.
As I already said I am not fight for these patches - for me it is ugly
hack, I fight for multiple routes for one prefix in kernel.
> > > There is also a VRRP implementation for FreeBSD. I've posted
> > > the URL for it before. In combination, Virtual Router Redundancy
> > > Protocol *and* multipath are, together, roughly equivalent to
> > > using BGP (assuming both your routers are running the VRRP code).
> >=20
> > No, VRRP can't help if you want to use summary bandwidth, but helps a
> > lot if you are want to get redundancy (I think so because I am using
> > VRRP on my core routers since 4.2).
>=20
> I don't think you can get what you want without the cooperation
> of the other end of the link. If you want FEC, then use Bill Paul's
> FEC code, which does the channel bonding that you seem to want. But
> you will need the other end to cooperate.
I know, anyway thinks like FEC will only solve problem of connecting
two boxes by some number of links but will not solve problem of
many x many connection.
=20
>=20
> > BGP can use multipath, as well as OSPF.
> > Possibility of kernel to store some number of routes for one prefix in
> > FIB can't replace BGP and, on other hand, BGP itself can't replace such
> > kernel feature.
>=20
> I already said that I think the code should be committed to the
> main line kernel, and preserved.
>=20
>=20
> > These two things almost orthogonal.
>=20
> Yes. They are. BGP solves the problem you are trying to solve,
> while the multipath is useful, but doesn't solve your problem.
>=20
>=20
> > PS:
> >=20
> > My opinion - it is useful feature for FreeBSD kernel, often used now
> > as good routing platform.
>=20
> I already said that I think the code should be committed to the
> main line kernel, and preserved.
May be it is better to discuss a bit what we want to have in kernel
exactly ? I think better to have another rt_entry structure for
second/third/etc routes for some prefix. Only modifications it will take
- lookup algorithm=20
( I think radix tree code will need some modification )
- forwarding code ( need to choose one of few routes )
- code for add/delete/get routes (something like "allow multipath" =20
option for addition and "remove all" for deletion)
> -- Terry
--=20
Vladimir B. Grebenschikov
vova@sw.ru, SWsoft, Inc.
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1022056574.460.30.camel>