Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 15 Dec 2001 08:28:35 -0800
From:      Terry Lambert <tlambert2@mindspring.com>
To:        Anthony Atkielski <anthony@freebie.atkielski.com>
Cc:        Valence Logrus <valence@symboliq.org>, advocacy@FreeBSD.ORG
Subject:   Re: Hotmail _still_ runs FreeBSD!
Message-ID:  <3C1B7A33.D05FA85A@mindspring.com>
References:  <20011215041054.H40531-100000@arctic.icelab.net> <030701c18575$2ca87570$0a00000a@atkielski.com>

next in thread | previous in thread | raw e-mail | index | archive | help
Anthony Atkielski wrote:
> That would correlate with what I read as well.  Apparently Hotmail has used
> Solaris for a long time, but with a highly customized MTA for their mail
> system.  I doubt that any off-the-shelf mail system can handle 60 million
> accounts with no scaling issues, and Microsoft Exchange Server certainly
> could not (MS tried and failed to replace the custom system with MXS), even
> though MXS is a fine e-mail system in itself.

For the sake of argument, let's say HotMail does have 60 million
subscribers.  And let's say further that they each receive 5 emails
per day.  That's 300 million emails.

You could handle that load with 120 1GHz FreeBSD boxes running fairly
stock Sendmail (you would need two common patches), which translates
to about three racks of 1U boxes, for just the incoming mail load.
Add another 40 machines for hot spare and bursty traffic (4 racks),
and you are covered with an incredibly small amount of colocation
real-estate. Say you multiply this by 5 (20 racks total) for the
outbound mail servers, the SSL based authentication phase (required
for Passport(tm) integration), and the other integrated services.

Of course, HotMail cheats; one of their cheats is to write incoming
mail to a RAM disk /tmp directory, and then ack it with the "250 OK",
as if it had been committed to stable storage (the need for 120
sendmail machines is predicated on committing the messages to stable
storage before ack'ing them as if they had been committed to stable
storage).

Running custom designed MTA software would drop the number of
machines required significantly.  For example, you aren't relaying
any inbound email: everything is local delivery.  Similarly, the
outbound email is all relayed to remote mail servers; the web
interface could even contact the remote server directly, should a
given email be for one or more recipients constrained to a single
target site.

You'd still want a lot of machines to front end the web interfaces,
particularly since the CGI's have become incredibly complicated for
no good user interface or business reasons (unless you count forcing
the use of custom controls "a good business reason").

FWIW: The FreeBSD servers are pretty much limited to DNS and other
infrastructure pieces, rather than customer facing code.


> The continuing use of Solaris probably does not represent superiority of
> Solaris or UNIX so much as it shows that a customized solution is needed,

Doubtful.  The main issue is integration and accounting, which is
often neglected in email services.  I suspect that account aging
policy controls (10 days after first login a signin is required to
avoid account inactivation; less thant 30 days between subsequent
signins is required to avoid inactivation, and 90 days to avoid
deletion), and passport integration are the big issues; it's probably
also why some of the .NET infrastructure was ported to FreeBSD.

> and customizing Microsoft Exchange Server is not an option, even for MS, as
> it is too complicated to modify for just one use.

Actually, Exchange would be fairly easy to fix to handle that type
of load, though you would need some cooperation from the NT/2000/XP
developement team to optimize a couple of rough corners that are
obvious bottlenecks to scalability on that order.


> Of course, with
> open-source software that is much simpler in design, it's far easier to
> modify it to do whatever you want.  By now, Hotmail's e-mail software has
> probably evolved (or mutated!) quite a bit from whatever base was first used
> to develop it.

Surprising as this is... Microsoft has Microsoft source code.

I rather expect that the main reason they have not done more to
get away from FreeBSD/Solaris is a resource/reward issue.  The
primary value of Hotmail to Microsoft, at this point, I think, is
as a portal entry into the Passport/.NET single sign on system:
it lets them immediately claim 60,000,000 subscribers for it, and
it hooks the credentials automatically, so even if you are not
intending to be a Passport user, you are a Passport user.

They are probably incredibly short handed, at this point, since
merely staying in "maintenance mode" wins them through to their
apparent goal, and it's no longer a valid advertising revenue
play, now that the value of "clickthrough" has depreciated since
the middle of last year.


> I suspect that if Hotmail did want to migrate off Solaris, FreeBSD would be
> a very obvious choice for an open-source replacement, and it would be far
> easier than trying to migrate to Windows.

If I could do the migration to Windows, then they could do the
migration to Windows...

-- Terry

To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-advocacy" in the body of the message




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3C1B7A33.D05FA85A>