Date: Thu, 18 Mar 1999 19:31:07 -0800 From: Julian Elischer <julian@whistle.com> To: Christopher Sedore <cmsedore@maxwell.syr.edu> Cc: "'freebsd-net@freebsd.org'" <freebsd-net@FreeBSD.ORG> Subject: Re: clustering/load balancing Message-ID: <36F1C4FB.41C67EA6@whistle.com> References: <262C3DA9BE0CD211971700A0C9B413A1CBD7@exchange.maxwell.syr.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
Christopher Sedore wrote: > > > From: Julian Elischer [SMTP:julian@whistle.com] > > > > Christopher Sedore wrote: > > > > > > OK, so I've worked some more on the IP clustering/load balancing > > stuff. > > > I've hacked together a clustering daemon, and reworked the way that > > the > > > kernel stuff is handled (it uses a flag and ipfw rules to do its > > thing). > > > > > > > rather cool but how about the following setup running with no mods > > at all.. > > +-------[Machine B] > > | > > [internet]-----[ Machine A]-----+-------[Machine C] > > | > > +-------[Machine D] > > > > Machine A's internet port is on ed0 > > firewall rules: > > > > on Machine A: > > ipfw add 100 fwd B tcp from 0.1.0.0:0.3.0.0 to A 80 in recv ed0 > > ipfw add 100 fwd C tcp from 0.2.0.0:0.3.0.0 to A 80 in recv ed0 > > ipfw add 100 fwd D tcp from 0.3.0.0:0.3.0.0 to A 80 in recv ed0 > > > > on machine B: > > ipfw add 100 fwd localhost tcp from any to A 80 in recv ed0 > > on machine C: > > ipfw add 100 fwd localhost tcp from any to A 80 in recv ed0 > > on machine D: > > ipfw add 100 fwd localhost tcp from any to A 80 in recv ed0 > > > > This shared the load among all 4 machines A, B, C, and D > > depending on the two bottom bits of the 2nd byte of the source > > address. > > > > I haven't tried this but I think it SHOULD work.. > > (of course you could use a 5th machine as the load sharer) > > > > what do you think? > > > It should work, though the fwd localhost would have to imply that we > should accept A's address as our own (not sure if that is implied or > not, or if I'm just missing something). That's what ipfw fwd does if the fwd address is local. BUT it doesn't cahnge the destination on the packet to be local, rather, it changes the socket to think it's somewhere else.. > The difference is that you are > solely dependant on machine A. If A croaks, you're done. That is, > you've got a single machine depenancy--fine for load sharing, but > perhaps not optimal for clustering with fault tolerance. If you have to > have aliases for A's address, I think that B, C, and D would need > separate links back to A to eliminate IP address conflict issues, or > you'd need mods similar to mine. No that's the thing about IPFW fwd.. The addressed on the packets being sent out of B,C,D will look as if the packets have come from A, but they will have physical layer addresses that will make them be accepted by A and forwarded. There are no IP address conflicts. The socket that catches the packet on B,C,D will actually think it is on A Amazingly this works.. Netstat will even report that you have a foreign socket on your system :-) You'll need an up-to date ip_input.c for this to work.. there's a bug in 3.1-RELEASE. > > You may have additional problems with high load on A. I know that our > (admittedly behind the times) PP200 FreeBSD firewall can get a bit busy > when we pass the 4000pps (that is 4000pps on the in and the out > interfaces simultaneously) mark. Sure then just dedicate one machine to do the diversion and such. > This means that you'd need additional > metrics to determine how much loading A should get vs the rest, > recognizing that you may increase application latency on A as the load > on B, C, and D increase. Of course, if you were using FreeBSD for A > anyway my methodology would leave excess CPU cycles for waste on that > box. (Also, I don't have any code for differential load control either, > and it could certainly happen in my scenario though the effects would > probably not be as dramatic). > > -Chris You should be able to throw a P60 in there as 'A' from the junk heap and get away with it... julian To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?36F1C4FB.41C67EA6>