Date: Wed, 19 Feb 2003 09:00:16 -0800 From: Terry Lambert <tlambert2@mindspring.com> To: Paul Robinson <paul@iconoplex.co.uk> Cc: chat@freebsd.org Subject: Re: Open source (was RE: Hi!Dear FreeBSD!) Message-ID: <3E53B820.BC50E3AA@mindspring.com> References: <IPEDKJGCDFHOPEFKLIDHGEJGCCAA.paul@iconoplex.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
Paul Robinson wrote:
> OK, I've moved this over to -chat where it belongs more than -hackers,
> trimmed the CCs, and turned this into a bit of a rant. For those just tuning
> in, Terry thought it would be a good idea to write a full-on GIS map
> rendering system so you could find your local user groups in a cool way.
This grossly misrepresents my position. My position is that there
is no software to provide this service, and that people who want
this service should write such software, if they truly want the
service.
The problem with demands for a particular service while ignoring the
existance of enabling infrastructure should be obvious.
Maybe I should have said "Send Patches", like everyone else does...
> That's besides the point. Apologies for length of mail, but be thankful this
> is now only 35% of the size it was going to be before I trimmed it - I had a
> fun lunchtime! :-)
8-).
> Terry Lambert wrote:
>
> > Sure, if you'll let me point out again that the original poster
> > wanted the maps to be clickable. 8-) 8-).
>
> GIS != Imagemaps. :-)
Little imagemaps have to come from somewhere. The active regions,
when clicked, also have to be converted on the basis of known map
coordinates relative to the image, which is in turn, relative to
the image type. Imagemaps don't really cut it, I think.
> > FWIW: the important gateing factors on any Open Source project are:
> >
> > 1) Motivation (a problem to solve, that people can agree on)
>
> You don't get paid for OSS work, so you'll work on what you feel like, and
> only when you feel like it. This is completely reasonable. At the moment I'm
> working with somebody on a proposal for taking over the competition at
> the5k.org and the one thing we're trying to avoid is filling it with
> bureaucracy simply because to do so would bring it to a grinding halt. We
> deal with that rubbish all day long - we don't need to do it all night long
> too. :-)
The idea that motivation boils down to "what I feel like doing today"
is an oversimplification. Anyone who has seriously followed the
travails of the FreeBSD project, or any of dozens of other successful
projects, must be aware of this: there is a cost to cooperation, and
it must be paid by the participants, in order to obtain the payment
they are seeking. Whether this is ego gratification, or some other
form of payment, the transaction involved is always one of simple
economics.
One commonly (and cynically) cited payment is "ego gratification".
While there are some people in the FreeBSD project motivated by
performing almost purely mechanical modifications of large numbers of
files, in order to get their login name on the most recent CVS stamp
and in the $Id$ tag of source files, this is not, I think, the
primary motivation for most of the project participants.
This really has nothing to do with bureaucracy, or the lack thereof:
process is necessary in terms of establishing feedback systems for
any project or business, but process is not the product (with rare
exceptions, such as Microsoft, USL, or IBM). The goal is a system
of self-regulating, homeostatic systems, which can maintain an
equilibrium point for an extended perio of time, and which then
self-correct from preterbations, back to that, or another stable
point.
You would be hard-put to simply design a new set of systems, and
simply "throw the switch", and have the new organization spring,
full grown, from Zeus's head, and be successful (you can do this
with "for-pay" organizations, but not with volunteer projects,
unless you control a sufficient number of the participants; every
merger-and-acquisition does this, usually to an extreme degree).
If you are truly going to attempt this, with the blessing of the
present project participants, then you must be prepared to identify
all the equalibria points between the current state and your target
state, if you aren't going to lose people over the discontinuities.
In a "for pay" organization, you have money as a motivator; that
is, you have voluntary participation, with economic exchange for
such participation. The amount you can change your organization
out from under the participants is *much* higher in a "for pay"
organization, since you have an established means to make up margin,
and you have a very large margin to begin with.
Participation in a pure voluntary project has economic payment, too,
but the margin is much, much smaller, and it is self selected by
each participant. While you can group almost all participants into
rough categories, which permits you to adjust rewards on the basis
of each participants wants, you do not have the leeway for change
that you have with a "for pay" organization.
> > 2) Working code (something that comes close to solving the
> > problem, or from which people can see a solution)
>
> This is critical, and is the reason why we're now seeing more "mainstream"
> applications like OpenOffice appear on open operating systems. If you don't
> have a decent framework that is easy to work with, it's perceived to be too
> risky to try and build something on top that is relatively complicated.
> Also, companies are starting to realise that giving away code is not the end
> of the world.
OpenOffice is a different phenomenon entirely. It is a copy of a
commercial product, with very little in terms of innovation. For
example, after the use of controlling access to support, training,
and documentation as a means of copy protection in the late 1980's
and throughout the 1990's, to pick up my prior example, it would be
incredibly foolish to embed these requirements in future work. What
kind of idiot would create a word processor that required documentation,
without some overriding business/systems need for a need for someone to
provide it? What is so incredibly hard about merely the *idea* of word
processing, such that this should be necessary?
One answer is that software vendors have marched up market, following
the pattern recognized by Clayton M. Christensen on "The Innovators
Dilemma". In other words, they are able to successfully defend their
existing market by way of increased complexity, and the inability of
Open Source equivalents to manage complexity. We see this in the use
of "embrace and extend" tactics, and in the pushing of increasingly
(and technically unjustifiably) complex standards through the public
standards processes.
> > 3) Community (communications and peers to provide a context in
> > which the work can take place)
>
> Or in the case of some mailing lists around here, people scream at each
> other...
That is a matter of control of the feedback systems. There are a
number of ways that this can be managed so as to damp out disruptive
influences which do not lead to information exchange and/or other
means of progress. Control of SPAM and trolls has and will continue
to stay ahead of the value curve for any forum supporting a successful
project. This is definitional.
> > A lot of people have #1, so they declare a Source Forge project, try
> > to cookie-cutter #3 (impossible to do), and leverage having #1 and #3
> > into someone creating #2 (also impossible to do).
>
> Indeed. Sourceforge is littered with the debris-like manifestations of good
> intentions.
Source Forge is, to my mind, a failure, which has arisen from a
fundamental understanding of mutual altruism networks. One could
make the same case for GPL, which attempts to build enforced
altruism networks. Much of this misperception comes from sources
such as "The Cathedral and the Bazaar", which have a fundamentally
incorrect model.
> > As a matter of fact, I claim that, given any #2, I can *find* #1,
> > and *create* #3.
>
> What you're saying is that projects are easy to bootstrap if there is
> already a decent chunk of code in existence. In other words, it's easy. This
> does not take into account the fact that at some point, somebody, somewhere,
> has to have the vision to come up with something genuinely new, recognise
> it's difficult, and go ahead with it's implementation anyway.
No, actually, you have to have working code. If "a decent chunk of
code", Mozilla would not have screwed itself so thoroughly, and the
Net/2 release would have been sufficient for BSD-based Open Source
projects to have popped up everywhere. Yet Mozilla *did* screw itself,
and BSD-based Open Source projects all waited for the trigger of 386BSD
and the working code it provided.
The problem is the nature of the people involved in volunteer projects.
Outside an academic, or sometimes corporate, research setting, where a
small team is paid to look into something useful or innovative, there
is little or no motivation to to attempt the difficult.
Instead, your pool of volunteers is made up of people who like to
tinker, and are not satisfied with the way things are, but those
people generally have no great leadership skills or vision of a
future that they are then prepared to work hard at making real.
You don't really get a lot of innovation in Open Source projects,
once they hit a certain level of organization; instead, you get an
initial impetus, and then a lot of drift, with occasional infusions
of brilliant work by lone innovators or small teams (usually academic)
who care strongly enough about their ideas to fight and win against
incredible opposition by inertia.
> Some examples:
>
> - A decent Visio-like program for X. The current bunch that try to emulate
> it don't cut the mustard
> - Something like Macromedia Stuido MX - Quanta just doesn't get there at
> all.
> - Here's a radical idea: a compiler that is command-line compatible with gcc
> but is available under a BSD license. Then, all the GNU stuff to be
> re-implemented under a BSD license. OK, this is a politcal point, but there
> is value there.
>
> These things will get done one day, when somebody is either paid to do it,
> or has the guts, conviction, time and motivation to do them.
Not really. They won't get done, or they will get done to spite
someone making a statement like this one.
The first two won't get done because Open Source is inherently
incapable of prodcutization. The people you see involved in
Open Source are programmers, who are frustrated with their lack
of control. It's either control of the design, control of the
features of software already in hand, or control of what they
perceive as quality. In each case, these are things which they
are denied by their employers in their "day jobs", because, being
engineers, they do not have a shared sense of esthetics with the
people who employ them.
Even so, having an offended sense of esthetics does not imbue one
with "good taste". It doesn't take more than a mediocre engineer
to be offended at having to perform a "quick and dirty hack" to
get a product out the door, with a management promise of "we will
revisit the problem and correct the implementation later". And a
mediocre engineer will not create a brilliant design, they will
only create a mediocre design.
The third thing may or may not get done. As you rightly point out,
it's a political issue, and politics is a stronger motivation than
ego: with ego, all tha's necessary is that you win; with politics,
it is also necessary that someone else lose. You invest doubly in
politics. But in general, issues like this are not relevent to the
enlightened individual, if they are recognizable as tactical, and
not strategic. Do I personally care whether FreeBSD use GCC? No;
it's a tactical technology: it has no strategic value. For that
same reason, I did not try to argue Andrew Tridgell out of putting
the code I browbeat him into releasing under a BSD license: today,
SAMBA is under the GPL, and is unlikely to ever spawn a BSD licensed
equivalent for a myriad of good reasons, all having to do with being
able to motivate the people donating their sweat equity to the
project.
The point is, you come closest to motivating project like this with
political rhetoric, yet almost all motivation will fall far short of
incenting a project.
It is not enough to declare a project. You must lead. And the way
you must lead is through already working code, because few of the
people you attract will be more than information age tinkers.
> > That sounds like most modern commercial software, to me, since it's
> > got legacy design factors from the 1980's/1990's causing it to need
> > documentation, support, and training materials as part of the (no
> > longer relevent) copy protection systems that grew up around the
> > software developement process.
>
> And of course, FreeBSD has no legacy factors in at all! :-) That might be
> unfair, as pretty much everything that is important has either undergone an
> evaluation and change, is going through it now, or is penned in for it soon.
> The real problem with projects as complicated as say FreeBSD or Mozilla is
> that there are occasions when too many cooks turn up. Complicated things are
> those things most cooks don't understand, simple things they do, therefore
> we get "bikeshed syndrome". How many analogies can I mix up? :-)
Too many? 8-).
The real issue with management of complexity by groups of tinkers is
that it's not going to happen in such a way as to make things work,
except by accident.
For FreeBSD, Julian Elischer has been one of the primary forces that
has, historically, pushed technology into the project. My SYSINIT()
code, Netgraph, the existance of SCSI drivers, and now KSE, as well
as dozens of other major and minor technological advancements, owe
their existance in FreeBSD to Julian Elischer. But Julian is an
exception, and it has cost him a lot of accumulated captial, in many
ways, to push these innovations forward against strong opposition by
the unenlightened tinkers, who see all such changes as non-reversible
risks: a tinker operates by doing something small, which can then be
undone, if it turns out to have been a bad idea.
There are other people I could use as examples in this regard, but
Julian is probably the best, and will, I think, be least offended by
being held up to scrutiny.
> > Seriously, it took a *lot* of skill to come up with the first Word
> > Processor that needed documentation for people to be able to use it
> > ("PC Write"). The author, Bob Wallace, said at one convention where he
> > spoke, "Software...", gestured expressively above and to the sides of
> > his head, "...is all up here. I sell manuals.".
>
> Yeah, I can see that. Makes sense in the context of a 1980's software
> market. It's not the case any more though. Last night I read a book by an
> ex-Microsoft employee named Andrew Barr entitled "Proudly Serving my
> Corporate Masters". It's a bit tongue in cheek, but he does raise a good
> point:
Microsoft has this paradigm engineered into its genes, and so does
any company that writes software to the Microsoft genome, and any
company that copies Microsoft or one of those companies, either to
cross a platform chasm, or to try to be "first to be second" in the
market dominated by one of these players.
In a lot of ways, Jeff Raskin is right, and still no one is listening.
In two of your examples of products that you think should/will be
written, you yourself fell into this trap of cloning from bad gene
stock.
> MS is split into three groups who are responsible for developing and
> delivering products, whether that be Windows, Office, MS "Bob", whatever.
> These are Program Managers who write specs and listen to users, Developers
> who take the specs and write the actual code, and Testers who take the
> Developers code and break it as much as possible to make sure the code is
> stable.
This is the myth of "Code Complete": by breaking this down this way,
you build a process that is capable of having a crank turned, and
when it is turned, out the other end of the machine comes something
that is identical in philosophy to the things which are already out
there, and selling well, and therefore you think your thing will sell
well, too.
At a former employer, there was an engineer who believed in this
philosophy so completely, they wanted everyone to accept Steve
McConnell as their personal saviour. An excellent engineer, but
they could not understand my reluctance to join their religion,
because they saw everything in terms of process: for them, the
process *was* the product, and any business was just a side effect.
It was very much like being back at USL.
> Barr's point was that in the early days, Microsoft was developer-orientated.
> Developers got to choose what products they would develop and how they would
> be developed. The products produced were great for a certain class of
> individuals who could think the same way those developers did. The result
> was MS-DOS. Barr thinks this era came to an end when MS Windows 3.0 was
> released. From that point on, the PMs took control and user interests
> dictated the direction of the company. User feedback became king. As a
> result, we got Win95 (yuk!), then WinNT (better), then WinXP (getting
> there), and the track will continue - they are realising now what Apple did
> in the early 1980's. He predicts this era will come to an end soon though,
> and it will be the testers that become dominant within the cycle. They will
> be determining when a product is ready to be released, not the PMs.
It would be interesting to see this, but I doubt he's right. The
Windows developement paradigm is 1980's-loaded to ensure that there
is an investment in end-user training, such that there will be a
cost associated with moving to another platform. Microsoft gets you
coming and going: there is a cost in training for Microsoft products,
and there is a cost in retraining for the new platform, because the
new platform has emulated the Microsoft barrier model.
I'd like to hope that this is accidental, but in my heart, I believe
it's because it's ingrained into software design philosophy, all the
way to the first year in colledge, or even high school courses. One
piece of evidence for this is all the people who've never seriously
studied CS, some of whom have never attended any college at all, who
all believe that they are God's gift to software engineering. You
see this sort of person posting place like "Slashdot" and other
developer forums, where you get a semi-illiterate rant, full of pool
spelling and shortened versions of English words. The worst part of
it is that they believe that they are communicating their point
effectively and persuasively because all their IRC buddies are posting
followups with the same poor grammar and spelling.
I expect that these, and other obstacles, mean that the testing
departments will never have final signing authority on product
releases, and that if they did, the products that ended up being
created through that process, would meet neither user expectations,
nor corporate sales goals.
One question that you should ask yourself on this is "Why do people
buy new versions of Microsoft products when they already own the old
versions of these products?"
The followup question to that is "Why, if product developement is
changed to a fully user and quality-centric model, would people
continue to buy new versions of these products?".
If you can't answer the followup question, the you've just validated
a business reason for not changing the model.
> With Open Source, we've seen a similar cycle. In the early days, dev effort
> went into the kernel, drivers for equipment that developers had lying
> around, programming tools, mail clients that worked great for people like us
> (mutt! yes, I love mutt! My Mum wouldn't have a clue though), and so forth.
> Then, people started wanting to get their parents, their bosses, the sales
> guy downstairs, the guy at the grocers, everybody, to start using the
> software. But these people don't care about how cool the virtual memory
> subsystem is. They don't care about KSE. They care about being able to send
> e-mails easily, surfing the web and sending a letter to Auntie Doris. We're
> now on that bell curve.
This isn't really true. People will be able to sell Late Adopters,
given this model, and given a huge investment in training and support
(the Open Source alternative to employing gifted people with English
degrees to write documentation). The current investment a new user
makes in learning the Microsoft usage paradigm is $2,500 per seat.
That's the barrier to entry for Open Source, unless it copies all of
everything exactly.
Selling the Early Adopters might also be possible; you sell the early
adopters on *precisely* things like "how cool" and "KSE". To do this,
you have to compete with industry buzzwords, like ".Net", "Palladium",
and so on: the things which might mean a change in the way things are
done, entirely.
But these groups together comprise a tiny portion of the total bell
curve, and you aren't reaching the mainstream. You can't. The value
proposition for the mainstream is whether or not they can hire a temp
for a week in accounting, and that person will be able to use the tools
in place, without needing training.
> The problem is (and yes, it is a severe problem), the developers will always
> be in charge and dictate the direction and therefore the usefullness of any
> system. Take for example the recent debate as to whether core should be
> reformed, over on -chat. Without wanting to go over that again, the
> developers insisted they were completely in charge and nobody had a right to
> say anything about anything unless they were submitting diffs. THAT is the
> reason why ultimately Open Source will fail: it's not in any developer's
> interest to listen to the requirements of those individuals who do not have
> the time, expertise or motivation to implement something themselves without
> the developer receiving some other gain (such as money). This is reasonable,
> but unless another way of working can be found, users will start walking
> away.
This brings me back to my earlier discussion of margin. There is
only so much distance you can move from an equalibrium point on
your way to the next one, before you hit inelasticity in the
economics. Good luck on "the5k.org". I think you will need it.
But change is possible, if you chart a true course. Originally,
there was just "the core team"; it self-assembled out of the people
doing the "386BSD 0.5 Interim Release", based on the people to whom
I handed off responsibility for unofficial patchkit (Jordan, Nate,
Rod, et. al.). Then it was self-perpetuated, by internal nomination
and torch-passing. Then it went to pure internal election. Now it
is election by the larger community of committers.
This was a (more or less) natural progression, and it took time to
overcome the economic elasticity. Even so, certain people whose
threshold was exceeded for "too long" have fallen by the wayside as
a result of these transitions of power. But it hasn't been the
bloodbath it would have been, had the switch been attempted over
night.
> There is all the cultural stuff as well, but that's more orientated to the
> Gen-X idealism of how licensing should work. In the end, that'll fall on
> it's face too, but that's for another day. This mail is long enough already.
> :-)
Culture actually has very little to do with it; selection of
participation based on license is one thing that needs a relatively
high degree of familiarity with the issues to make a decision on.
Most people end up conforming their views to the community, rather
than selecting a community on that basis, in my experience. 8-).
> Anyway, now I've put this over onto -chat, I'm sure this will evolve into a
> nightmarish thread that will continue ad infinitum.
You can only hope... ;^).
-- Terry
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-chat" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3E53B820.BC50E3AA>