FP-05 Software Measurement
Status
2002-06: Approved by SESC Executive Committee.
References
Higher Level Policies and
Plans
None Referenced.
Lower Level Policies and
Procedures
None Referenced.
Other References
ISO/IEC FDIS 15939, Information Technology -- Software Engineering -- Software Measurement Process, (ISO/IEC JTC1/SC7 07n2560)
Policy Statement
SESC uses the view and terminology of software measurement explained in Annex A of ISO/IEC 15939. This view unifies a wide range of necessary measurement concepts. Software measurement is about the assessment and prediction of well-defined attributes of well-defined entities (i.e., process, product, resource). The text and figure below provides an overall framework, defines key concepts, and provides a vocabulary of terms that should be used in SESC standards on the subject of software measurement. The key terms are distinguished by the use of italics.
The measurement information model is a structure linking information needs to the relevant entities and attributes of concern. Entities include processes, products, projects, and resources. The measurement information model describes how the relevant attributes are quantified and converted to indicators that provide a basis for decision-making.
The selection or definition of appropriate measures to address an information need begins with a measurable concept: an idea of which measurable attributes are related to an information need and how they are related. The measurement planner defines measurement constructs that link these attributes to a specified information need. Each construct may involve several types or levels of measures.
This measurement information model (see Figure) identifies the basic terms and concepts with which the measurement analyst must deal. The measurement model helps to determine what the measurement planner needs to specify during measurement planning, performance, and evaluation.
The Figure illustrates the relationships among the key components of the measurement information model. The model defines three types of measures: base measures, derived measures, and indicators. The information content of measures increases as they become closer in the model to the information need. The individual components of the generic information model are described below.
1 Entity
An entity is an object (for example, a process, product, project, or resource) that is to be characterized by measuring its attributes. Typical software engineering objects can be classified as products (e.g., design document, source code, and test case), processes (e.g., design process, testing process, requirements analysis process), projects, and resources (e.g., the programmers and the testers). An entity may have one or more properties that are of interest to meet the information needs. In practice, an entity can be classified into more than one of the above categories.
2 Measurable attribute
An attribute is a property or characteristic of an entity that can be distinguished quantitatively or qualitatively by human or automated means. An entity may have many attributes, only some of which may be of interest for measurement. The first step in defining a specific instantiation of the measurement information model is to select the attributes that are most relevant to the measurement users information needs. A given attribute may be incorporated in multiple measurement constructs supporting different information needs.
3 Base measure
A base measure is an attribute and the method for quantifying it. A base measure is functionally independent of other measures. A base measure captures information about a single attribute. Data collection involves assigning values to base measures. Specifying the expected range and/or type of values of a base measure helps to verify the quality of the data collected.
3.1 Measurement method
A measurement method is a logical sequence of operations, described generically, used in quantifying an attribute with respect to a specified scale. The operations may involve activities such as counting occurrences or observing the passage of time. The same measurement method may be applied to multiple attributes. However, each unique combination of an attribute and a method produces a different base measure. Some measurement methods may be implemented in multiple ways. A measurement procedure describes the specific implementation of a measurement method within a given organizational context.
3.1.1 Type of measurement method
The type of measurement method depends on the nature of the operations used to quantify an attribute. Two types of method may be distinguished:
· Subjective quantification involving human judgment
· Objective quantification based on numerical rules such as counting. These rules may be implemented via human or automated means.
3.1.2 Scale
A scale is an ordered set of values, continuous or discrete, or a set of categories to which the attribute is mapped. The measurement method maps the magnitude of the measured attribute to a value on a scale. A unit of measurement often is associated with a scale.
3.1.2.1 Type of scale
The type of scale depends on the nature of the relationship between values on the scale. Four types of scales are commonly defined:
· Nominal the measurement values are categorical. For example, the classification of defects by their type.
· Ordinal the measurement values are rankings. For example, the assignment of defects to a severity level.
· Interval the measurement values have equal distances corresponding to equal quantities of the attribute. For example, cyclomatic complexity has the minimum value of one, but each increment represents an additional path.
· Ratio the measurement values have equal distances corresponding to equal quantities of the attribute where the value of zero corresponds to none of the attribute. For example, the size of a software component in terms of LOC.
The method of measurement usually affects the type of scale that can be used reliably with a given attribute. For example, subjective methods of measurement usually only support ordinal or nominal scales.
3.1.2.2 Unit of measurement
A unit of measurement is a particular quantity, defined and adopted by convention, with which other quantities of the same kind are compared in order to express their magnitude relative to that quantity. Only quantities expressed in the same units of measurement are directly comparable. Example of units include the hour and the meter.
4 Derived measure
A derived measure is a measure that is defined as a function of two or more base measures. Derived measures capture information about more than one attribute. Simple transformations of base measures (for example, taking the square root of a base measure) do not add information, thus do not produce derived measures. Normalization of data often involves converting base measures into derived measures that can be used to compare different entities.
4.1 Measurement Function
A measurement function is an algorithm or calculation performed to combine two or more base measures. The scale and unit of the derived measure depend on the scales and units of the base measures from which it is composed as well as how they are combined by the function.
5 Indicator
An indicator is an estimate or evaluation of specified attributes derived from a model with respect to defined information needs. Indicators are the basis for analysis and decision-making. These are what should be presented to measurement users. Measurement is always based on imperfect information, so quantifying the uncertainty, accuracy, or importance of indicators is an essential component of presenting the actual indicator value. Therefore, an interpretation of indicators is performed to provide the desired information product.
5.1 Measurement Model
A measurement model is an algorithm or calculation combining one or more base and/or derived measures with associated decision criteria. It is based on an understanding of, or assumptions about, the expected relationship between the component measures and/or their behavior over time. Models produce estimates or evaluations relevant to defined information needs. The scale and measurement method affect the choice of analysis techniques or models used to produce indicators.
5.1.1 Decision criteria
Decision criteria are numerical thresholds or targets used to determine the need for action or further investigation, or to describe the level of confidence in a given result. Decision criteria help to interpret the results of measurement. Decision criteria may be calculated or based on a conceptual understanding of expected behavior. Decision criteria may be derived from historical data, plans, and heuristics, or computed as statistical control limits or statistical confidence limits.
6 Measurable concept
A measurable concept is an abstract relationship between attributes of entities and information needs. For example, an Information need may be the need to compare the software development productivity of a project group against a target rate. The Measurable Concept in this case is "software development productivity rate". To evaluate the concept might require measuring the size of the software products and the amount of resource applied to create the products (depending on the chosen model of productivity). Additional examples of Measurable Concepts include quality, risk, performance, capability, maturity, and customer value.
Rationale
Notes
The text above was paraphrased from Annex A of FDIS 15939. The final standard is expected to be substantively identical.
History
2002-1: This draft updates the policy in accordance with Feb 2001 direction from the SESC XC to adopt the approach taken in ISO/IEC 15939.
Maintenance Needs Identified
None.