Tuesday, December 12, 2006

What defines software as OSS?

Open source software (OSS), a term first coined in 1998 [3], has spurred many products in its short history, most notably the Linux operating system, the Apache server, and the so-called killer-apps such as BIND and Sendmail.

But what really defines software as OSS? The answer is not as simple as it may seem. Generally, OSS is a software product distributed by license, which conforms to the Open Source Definition [11], the best known of which are GNU General Public License (GPL) and Berkeley Software Distribution (BSD). Unlike the traditional closed source software (CSS), OSS can be freely used, modified, and redistributed. Its source code is also freely accessible. Today, it is common to find CSS projects evolve into OSS in order to obtain its benefits. Occasionally, OSS projects become CSS, but usually for different reasons. An extensive explanation and analysis of OSS development can be found in [3].

An OSS project begins simply. A single developer or group of developers writes the first version of the software and make it freely available over the Net, inviting other developers to participate and contribute pieces of code. The evolution of the product being developed is normally coordinated by its creators. Among the responsibilities of the coordinators is configuration management, release scheduling, deciding which code contributions will be accepted, and other management activities. We should note that sometimes this group of coordinators has a great deal of power (strong control on code submission, issuing of clearly defined specifications), which makes the process look like a traditional one. It can be said that the vitality and the success of the projects depend strongly on the coordination of the process.

OSS development emphasizes the maintainability of the software released. Making software available on the Internet allows developers around the world to contribute code, adding new functionality (parallel development), improving the present one, and submitting bug fixes to the current release (parallel debugging). The coordinators of the project ultimately decide the contributions and bug fixes to accept, incorporating them into the main code and final product release. Indeed, the same process of code contribution and bug fixing is continued in this circular manner.

This type of development has both advantages and disadvantages. In various cases OSS seems to have solved many of the problems of traditional software engineering methods, since it has been possible to produce reliable, high quality, and low-cost software in a brief amount of time. The existence of a large pool of testers and developers facilitates debugging and the true peer review of the code results in better code. The availability of the source code allows someone to make modifications to meet his or her own needs, while the lack of black boxes is important for someone to inspect the code for its correctness and to assure dependability issues. However, very few empirical studies have been available to support or reject these claims [8, 10].

The disadvantages of OSS development include absence of complete documentation or technical support. Moreover, there is strong evidence that projects with clear and widely accepted specifications, such as operating systems and system applications, are well suited for the OSS development model. However, it is still questionable whether systems like ERP could be developed successfully as OSS projects. (For more on the advantages and disadvantages of OSS development, see [3, 5].)

A ready interpretation of the OSS development process is that of a perpetual maintenance task. Developing an OSS system implies a series of frequent maintenance efforts for debugging existing functionality and adding new ones to the system. These two forms of maintenance are known as corrective and perfective maintenance, respectively.

Since it is still difficult to monitor the open source process, it is reasonable to measure and assess the resulting product, that is, the delivered code. The purpose of this article is to report and discuss the results of a case study examining the maintainability of the source code delivered by open source development. To this end, we measured the source code of five OSS projects, which in several cases involved development in CSS fashion. We have assessed the results according to a maintainability index derived from analysis of industrial CSS systems. We contend maintainability is the core quality issue in OSS development.

Source Code Measurement

A well-known conjecture in modern software engineering is that external quality characteristics are correlated to internal quality characteristics. The measurement of source code provides useful information for the assessment of its quality, predicting to some extent the external system quality characteristics, such as maintainability, reliability, extensibility, and portability.

Measurements may be used to obtain a picture of the quality both of a single component and of an entire program. Typical software metrics are the size of the code (measured in lines of code, number of statements, and so on) and the code complexity (measured through complexity figures such as the cyclomatic complexity). Setting an acceptable range for each metric considered assesses code quality, in which every measured value should fall in. A set of such ranges along with a number of efficient program-writing rules defines a programming standard, that is, any comparison to these values can lead to a representative view of the quality of the tested programs. Software organizations interested in software development set their own standards. An example is the NSA standard [1], which is derived from the analysis of 25 million lines of software written for NSA.

In our analysis we used a set of measurement programs available in the Debian GNU/Linux release, and a number of Perl scripts for managing measurement results. The metrics considered are among the most widely reported and used in the literature and are listed here:

* Number of lines of code (LOC) measures the physical size of the program code, excluding blank lines and comments.
* Percentage of lines of comments with respect to the number of lines of code (PerCM) describes the self-descriptiveness of the code.
* Halstead Volume (V). Halstead [4] defined four metrics that can be measured from a program's source code: n1 (the number of distinct operators), n2 (the number of distinct operands), N1 (the total number of operators) and N2 (the total number of operands). Based on them, he defined program vocabulary n (given by n = n1 + n2) and program length N (given by N = N1 + N2). Finally, he defined Volume, a composite metric given by the formula V = N * (LOG2 n). Halstead Volume provides an alternative measure for the size of a program.
* Cyclomatic Complexity V(g). Proposed by McCabe [7], this metric counts the number of independent paths in the control flow graph of a program component. Its value depends on the number of branches caused by conditional statements (if-then-else). It measures the structural complexity of the component.

It would be quite difficult to reach a consensus about the metric ranges that OSS code should respect in order to define an OSS-specific programming standard. In this study, rather than comparing with a predefined standard, we preferred to directly compare maintainability measures of open source with those obtained for closed source software, or simply to observe the trend of these measures and derive conjectures about improvement or deterioration of code quality. Because it may be difficult to obtain a single picture about maintainability from many measures derived from many different metrics, we preferred to use a different approach in our study based on the composite metric, Maintainability Index (MI), chosen by SEI [6] as the most suitable tool for measuring the maintainability of systems with high-quality requirements. We chose MI because we believe that OSS should conform to such standards in order to compete with CSS. Moreover, various OSS projects, including some we studied, are often part of large scale, critical applications