Saturday, January 24, 2004

The NeverEnding Story/Software

In the past ten years I have noticed a new class of software projects emerging. For lack of a better term I call them organic projects, since they take on a life of their own -- they really have no real endpoint and releases, when they occur, are a matter of convenience not a potential endpoint. Often these projects go through continuous release and evolve not only as the understanding of the requirements evolve, but also grow because they create new capabilities, that inspire a new burst of requirements, that create new capabilities, that inspire ... . Data collection and data mining projects fall into this category as do other internal projects that support corporate operations. The web as a whole is an evolving meta project of this class, where requirements evolve through use. Many Open Source projects follow a similar path but often their evolution is inspired by a changing developer and user base. Indeed I see more projects falling into this category as software permeates more of our lives and more individuals have the capability to author and modify software. Another defining attribute of this software is that it normally lasts decades (if not longer since software has only been around for decades!).

These projects are not well accommodated by current Software Engineering models, theory and practice. Agile methods come close, but they do not seem to consider long time scales. Incremental and spiral models, although spawning lots of releases, are workng to an end -- in incremental you prioritize the features and in spiral you do that plus risk assessment. Similarly design, development, testing, maintenance, (name your own) are affected.

So what do you think? Is this a new class of software and do you have other (better) examples of this? Is it worth beginning a thread on this blog that discusses this class of software and thoughts on the software engineering of these evolving systems?