Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 24 May 1996 09:11:21 -0400
From:      "Louis A. Mamakos" <louie@TransSys.COM>
To:        dennis@etinc.com (Dennis)
Cc:        "Karl Denninger, MCSNet" <karl@mcs.com>, hackers@freebsd.org
Subject:   The view from here (was Re: ISDN Compression Load on CPU)
Message-ID:  <199605241311.JAA04673@whizzo.transsys.com>
In-Reply-To: Your message of "Thu, 23 May 1996 15:34:55 EDT." <199605231934.PAA29049@etinc.com> 
References:  <199605231934.PAA29049@etinc.com> 

next in thread | previous in thread | raw e-mail | index | archive | help

I'll briefly put my UUNET hat on, the one that I've worn when I was
directly involved the design, specification and recommendation of
hardware selections of well in excess of $10M last year.

Consider that you have many, many unmanned POP locations, all over the
planet.  These are generally co-located in telephone central offices
which are not manned 24x7.  We colocate in interexchange carrier
central offices for a variety of reasons, including cost (you don't
need to purchase $5000/mo local loops on each of your DS3 trunks).
There is also good environmental conditions and power available in the
form of -48V DC power plants that run all of the telco transmission
stuff.

We do have contracted "remote hands" (NOT "remote brains!") which can
be dispatched when required to go touch things, swap boards, etc.
This costs about approximately $100 per hour, with something like a 3
hour minimum.  And in some cases, they could be a few hours away
depending on the time.  And in some cases, break other stuff that used
to be working 'cause they bumped into it or unplugged the wrong thing.
So the way that you want to think about these installations is that
the hardware is in a room filled with poison gas, and the last thing
that you want to do is send a human in to diddle with stuff.

[Did you know that one of the leading causes of telecom related
outages is classified as "human intrusion failure"?]

Now, PC platforms in this environment: we've searched far and wide for
a PC platform which can be deployed in these spaces, not for actual
network traffic, but for support purposes such as RADIUS servers, DNS
servers, NNTP news senders, etc.  Try to find one which does not
require a keyboard and display to do system maintenance activities.
Try to find one which also operates from -48VDC.  Try to find one
which is *completely* controllable from a serial port.  

You can't find one that works anywhere near as well as a Sun Sparc
platform, where you can do *anything* you need from the serial port,
like change the boot device, format the disk, boot the system, etc.
In fact, GNP makes a NEBS complient Sparc platform suitable for just
this environment; it's really *very* cool.  Cool enough that we are
going to incur the considerable extra cost of maintaining a Solarius
platform as well as the BSDI platforms we already have.

For computing applications which require bulk storage, I have no
problem with installing extra disks and boot media to be able to
QUICKLY recover the system with NO PHYSICAL INTERVENTION.  It's just
not reasonable to have to have someone visit a site literally on the
other side of the planet in Singapore to punch the reset button.

And for access server sorts of products, there is just no good reason
why I have to wait for my box to 'fsck' should they crash and reboot.
There is no reason why there has to be spinning magnetic media, which
will eventually crap out and have to be replaced, in a piece of
hardware which is intimately involved in service delivery when there
are fine alternatives available which use more reliable technology.
Imagine flash memory instead; heck, the Cisco 7500 routers we're
deploying have 8MB of flash on the RP, and a pair of 8MB and 20MB
PCMCIA flash cards plugged into them as well.

> I really hate this garbage argument. Novell servers with uptimes
> over a year are commonplace...PC bus, spinning media and all.
> you only have problems with drives that are too fast, too hot and
> too overworked, which simply isnt the case with a router scenario.

Consider this: you as a service provider have some set of costs
incurred in providing a service to your customers.  

- The cost of transmission and telecom services is a big one; it's also
somewhat difficult to reduce in a big way since the pricing is
somewhat inflexible for your T3 or ISDN PRI.

- The cost of the hardware is the next thing that people think about.
In either two ways:  

  1) it's a fixed, one-time expense, compared to the recurring
  expenses; so there is a CASH issue.  

  2) the actual COST is relatively low if you consider that you're
  depreciating the hardware over 60 months (or whatever you convice the
  IRS of).  Compared to the recurring cost of the stuff you plug into
  the box, the hardware in many cases is almost "free."  Assuming you
  don't have a CASH problem.

- The operations cost.  That is, how much do you keep spending to keep
the box in a state whereby it continues to generate revenue.  This
includes stuff like maintenance contracts, power, and LABOR.  This is
the one area where Smart People can directly affect the cost incurred.
If you expend more one-time dollars at the outset, you can reduce the
recurring expense, and you win in the long run.  And don't forget the
OPPORTUNITY COST associated with operating this equipment; if you're
spending your (limited) time whacking away keeping one box running,
you're not doing some other new revenue producing thing, even if you
wanted to and otherwise the resources.

So when a vendor comes to me with a box which has absurd operating
costs because it has to be touched, or takes up too much rack space or
the operations component can't be automated, then he's not getting
very far in the door.  This is the area that an ISP can directly apply
his experience and brains to optimize and hopefully cause the REVENUE
to exceed the COST at the end of the day, so I can go buy beer.
Err.. make the stockholders happy; yeah, that's it :-)

So, while optimizing the capital cost of acquiring hardware may be
attractive, it's only part of the larger picture you need to consider.
[It helps to be able to drive a spreadsheet to be able to tell if the
right thing is happening..  That's why the other partition of the disk
has Windows95 on it.]  The thing that the engineering folks at UUNET
are just obsessed with is scalable solutions; if it can't be scaled
up, they you'll become a victim of your own success.

While I will not claim that my requirements are the same as very many
others, you cannot dismiss them at just the ravings of a lunatic.
I've put my money (more than $10M of it) where my mouth is.  Sorry for
the long message; I'll take my UUNET hat off now, and go away quietly.

Louis Mamakos




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199605241311.JAA04673>