Innovation and Open Source

IT Managers' Journal [via LWN] has an article about Mitch Kapor's take on the open source software development model. The opening paragraph caught my eye.

The software development model that produced Lotus 1-2-3 and Microsoft Windows and created long lists of multi-millionaires in the process is all but over, says Mitch Kapor, founder of Lotus, the EFF, and Open Source Application Foundation. In its place stands open source software, where the model is, "Do good work, make a decent living and maybe make the world a better place." [Chris Gulker]

Of course, this is all well and good coming from someone who made his own fortune from software like, er, Lotus 1-2-3. More interesting than that, though, is that Kapor is now ploughing a considerabke amount of that fortune into the Open Source Software Foundation. There is something about this which feels uncomfortably like using wealth created using the old model to subsidise the new model. Whether or not the old model created any value is neither here nor there if the new model cannot raise the capital needed to get started, or if it unsustainably depletes the available pool.

As the article alludes to later, we might separate more-of-the-same software (the vast bulk of it) from genuinely-something-new software.

Kapor wondered aloud where innovation would come from in the future. "It won't come from commercial software," he said, citing Microsoft's stranglehold on conventional applications and operating systems. But he went on to say that open source would breed innovation only if project teams were willing to take more risks. [Chris Gulker]

A lot of celebrated open source projects are from the more-of-the-same category — KDE, GNOME, OpenOffice — however good they are. The Linux kernel is more-of-the-same, as are many of the GNU tools. A notable exception in the GNU project is the HURD kernel, which is genuinely-something-new, but is also a decade or so behind schedule and not yet complete. It's intended place at the heart of the GNU system has been filled in the mean time by more-of-the-same Linux.

Of course, a lot of celebrated closed source software is more-of-the-same, too. As an obvious example, Microsoft Office was never revolutionary (and there are still things it can't do as well as its competitors did a decade ago, but that's another story).

Perhaps looking at where innovation in software has happened in the past will help with working out where it might happen in the future. I believe that innovation has happened predominantly in an academic setting, in which class I include Universities and research centres with an academic ethos. Assuming that is a reasonable generalisation (and I'd be keen to have exceptions pointed out) the obvious question is why this might be so. Because innovation is something that people do, and it is in these organisations where people have traditionally been able to find enough space to innovate.

On the one hand, then, we might expect to see innovation in the future coming from the same place that it always has done in the past. On the other, there is so much greed around now that there are precious few places left where people can find enough space to innovate, but that really becomes a question of how much innovation will happen, not of where it will happen.

So innovation doesn't really happen in the space in which open and closed source software compete. However software innovation has to move from innovator to potential users, and it might still be worth thinking about which software development and financing structures do most to support this transition. There it seems certain that open source has at least the potential to get an innovation to the widest possible audience.

Comments

[Bill Seitz]

I think it's worth noting that the *ideas* for innovative products often are born in academic environments, but those creators often have to move outside academica to actually generate a *product*. To some extent this is a desire for financial return, but I think it's also due to academia's lack of interest in implementation: there are points for having original ideas, but once you've had the idea, actually implementing it is not considered relevant.

[Hamish]

Absolutely true, Bill, and a certain amount of pressure on academics to think about how their work fits in with "real" world is justified. It so happens that most of the academics who I work with day to day have a firm grasp of that already. Indeed the two projects I work on are

* FloodRiskNet (http://www.floodrisknet.org.uk/) network, which generates active involvement from academics and "practitioners"

* A project called "Towards the Next Generation of Computer Models for the Prediction of Flood Level and Inundation Extent" (website at experimental stage) in which representatives of the potential users of the research have been present at every progress meeting so far.

I thought while writing the original post about whether to start ruminating about the different types of innovation, which must include the multitude of small but nonetheless innovative steps in getting a "big" innovation to market, as well as the "big" innovation itself. To say that "innovation doesn't really happen in the space in which open and closed source software compete" is silly, I admit.

If you have a widget to manufacture, there is a huge amount of work in making it work as reliably as required, for as long as is needed, in the conditions in which it will be used, and so on.

The point is though that these smaller innovations tend to occur to a greater degree when the original idea is well enough developed that its potential is visible. Whatever model you use to fund it, the work is more likely to get done and less demanding of thinking space, assuming that the right connections are made between early stage innovators and bringing-it-to-market type people. And that is one of the things that research networks are for.

Another point to remember is that innovation is not just technical, and making a success of a new technology may well involve creating the (social) environment in which it can flourish. Thus we have sociotechnical innovations (I am indebted to Mike Abbott http://www.knowledge-engineering.org/ for this observation).