98 Million Bugs Strong -- RelEng Class 2: Linus' Law

This post continues Adventures in Release Engineering and the Sebastian's student-course on Release Engineering at Olin.

We talked about core Open Source concepts: transparency, release early release often, work upstream, scratch your own itch, and Linus' law. My take on Linus' Law is success. Eric Raymond wanted Emacs to move to the bazaar model, and they did. Open source projects now face different issues -- such as "who supports OpenOffice?"

Linus' Law in 1999
Some background. In 1997, when Eric Raymond first presented the Cathedral and the Bazaar, the source for GNU Emacs was locked down to core developers until a scheduled release date came along. This "cathedral model" was the reality of many open source projects. Raymond had the insight to state

"given enough eyeballs, all bugs are shallow"
 -The Cathedral and the Baazar (1999)

and this became Raymond's blazing sword against the cathedral. If you the core developer kept your source to yourself, you got to spend endless hours fixing bugs. Bug fixing is inglorious work, and the promise of offloading bug fixes, tantalizing.

Currently open source runs into a different wall. Raymond calls it "treating your users as co-developers." I call it "who supports OpenOffice." I am sad to announce that no mater how much I treat my grandmother as a co-developer, she will not be able to diagnose OpenOffice redraw issues.

Excuse me for a second. My apologies OpenOffice community. Even cut to the barebones, building an office suite is a complex undertaking. Though you are the target of this blog post, we really do appreciate all the work that has gone into the project.

Right. According to their site, OpenOffice has been downloaded 98 million times. I'd wager that doesn't include distribution through linux distros. So why after 98 million downloads is OpenOffice so antiquated and buggy? Co-developers. When your sister's, boyfriend's, best friend does an OpenOffice mail-merge and it wigs out, he doesn't have the skills to fix the issue. When your IT guy does a mail-merge with snazzy unix-like command line scripts, he can address almost any issue he runs into. The eyeballs that peruse GNU Emacs make all Emacs bugs shallow. The eyeballs that peruse OpenOffice do not... and there are over 98 million of them! Instead of treating your users as co-developers, Raymond would have been more accurate to say "treat your developers-- who may happen to be users -- as co-developers."

Of course, there are other factors to OSS project success than technical experience. Community management, release engineering, commercial support, ownership, altruistic targets, etc. Scratch off the right combination of OSS community characteristics and you win the lottery. Scratch the wrong ones and win OpenOffice. Here are a few of my contributing factor breakdowns:

What it has going for it
* Enticing advanced CS problems (nerd crack)
* Nerd cred. (you have a patch in the kernel?)
* Self-serving altruism (give the gift of playing with linux)
* Intense commercial support (many paid kernel hackers)
* Users are developers

* Organization of ownership (components split out to owners, IE the installer)
* Nerd cred.
* Self-servering altruism (support of web standards)
* Commerical / academic support
* Users are developers

* Scratches own itch
* Simplifying, limited scope
* Strong personal ownership

What's not so hot:
* Iron-fisted organization
* Endless feature mentality
* Bystander effect
* Non-essential for most developers
* Little commercial interest to anyone except Sun/Oracle

Some day I'll have a complete taxonomy for open source projects, but today is not that day.

Edit 11:59, 26 Sep 2011: Forgot a paragraph =[.
Linus' Law and Release Engineering
The funny thing is that release engineers don't exactly fall under Linus's law. Perhaps this is my UX bias, but in my opinion, it is the role of the release engineer to support stable, usable software. We are the ones out there saying "whoa, let’s not update all Ubuntu users to Python 3 just because it came out yesterday" (hey there Arch linux). We help projects meet usability goals, create installers, package, and determine when a project is ready to increment the version number. Thought I feel strong allegiance to the bazaar model, I would help an OSS project maintain a cathedral model (as has been suggested for Diaspora) if it greatly helped releases.
</edit>

- Colin
Asciitar

Resources
http://www.kalzumeus.com/2010/09/22/security-lessons-learned-from-the-diaspora-launch/  (particularly "Take Care With Releasing Software to End Users")

Adventures in Release Engineering

My dear college (Olin) is feeling small... really small... three-person-class small. But not only small, also student led. This semester I'm lucky enough to be learning release engineering from a war-torn Red Hat release engineering expert (2 year my younger, curses!): Sebastian Dziallas. And I've only got two peers to compete with for attention.

It looks something like this:

Release Engineering 
OSS project + student + actual release eng work + mentorship = learning (and fun)

Though I've organized a few hackathons, and heard from OSS community leaders like Jeff Mitchell, but I think this experience will be unique: an In Vivo learning experience. Education with context -- real world context.

So what to engineer? We'll see. I'm looking at NB, a very neat collaborative document annotation tool. Wow, marking up documents, what kind of a paper-pushing bureaucrat am I, you're wondering. No, but it really is a neat tool. Designed for academic reading settings, NB lets students mark up readings with questions and comments. Confusion can be cleared during the reading process. But I'll save you the sales pitch; NB is just a idea at this point.

So how does one start a course on release engineering? Talking about different scales of software production. We outlined our thoughts on startups, large software companies. We came up with the standard fare, startups just run. The code is ugly but it doesn't matter, its all about users. Large corporations are process heavy and move in developed areas. They take shelter in enormous manpower, though sometimes inefficient, can spring to action at the sign of a threat.

Through the semester I imagine we'll have to tailor our understanding to OSS projects, and within that, volunteer projects (as opposed to people who hack on the kernel for their benevolent corporation). I expect to learn a lot about volunteer projects, and the function and passion of their contributors.

Here's to a good semester!