Chapter 3 Software Quality Requirements – Software Quality Assurance

Chapter 3
Software Quality Requirements

After completing this chapter, you will be able to:

  • describe the history of the concepts conveyed by software quality models;
  • understand the different characteristics and sub-characteristics of software quality as outlined in the ISO 25010 international standard;
  • use the concepts to specify the software quality requirements of a software product;
  • explain the positive and negative interaction between the quality characteristics of software as described in the ISO 25010 international standard;
  • understand the concept of software traceability.

3.1 Introduction

All software is an element of a system, whether it be a computer system in which the software may be used on a personal computer, or in an electronic consumer product like a digital camera. The needs or requirements of these systems are typically documented either in a request for quote (RFQ) or request for proposal (RFP) document, a statement of work (SOW), a software requirements specification (SRS) document or in a system requirements document (SRD). Using these documents, the software developer must extract the information needed to define specifications for both the functional requirements and performance or non-functional requirements required by the client. The term “non-functional,” as applied to requirements, is deprecated and is not used in the Institute of Electrical and Electronics Engineers (IEEE) 730 standard [IEE 14].

Software quality assurance (SQA) must be able to support the practical application of these definitions. To achieve this, many concepts proposed by software quality models must be mastered. This chapter is dedicated to presenting the models as well as the software engineering standards available for correctly defining performance or non-functional (i.e., quality) requirements of software. Using these SQA practices early in the software development life cycle will ensure that the client receives a quality software product that meets his needs and expectations.

The above definition of a quality model implies that the quality of software can be measured. In this chapter, we describe the research carried out over the years that culminated in a definition of a software quality model. Quality assurance, in certain industries and business models overseen by the software developer, requires a more formal management of software quality throughout the software life cycle. If it is not possible to evaluate the quality of the resulting software, how can the client be in a position to accept it? Or at least, how can we prove that the quality requirements were met?

In order to support this condition, a software quality model is used so that the client can:

  • define software quality characteristics that can be evaluated;
  • contrast the different perspectives of the quality model that come into play (i.e., internal and external perspectives);
  • carefully choose a limited number of quality characteristics that will serve as the non-functional requirements for the software (i.e., quality requirements);
  • set a measure and its objectives for each of the quality requirements.

It is therefore necessary for the model to demonstrate its ability to support the quality requirements definition, and subsequently, their measurement and evaluation. We have seen in the previous chapter that quality is a complex concept. Quality is often measured by considering a specific perception of quality. Humankind has mastered measuring physical objects for centuries, but even today, many questions remain about measuring a software product and the ability to measure it in an objective manner. The role and importance of each quality characteristic are still difficult to clearly define, identify, and isolate. What's more, software quality is often a subjective concept that is perceived differently depending on the point of view of the client, user, or software engineer.

This chapter will provide the SQA practitioner with the knowledge to use the ISO 25010 software quality model concepts. In this way, he can initiate processes and support software engineers in their development, maintenance, and software acquisition projects. We will begin by presenting the history of the different models and standards designed to characterize software quality. This will be followed by a discussion of the concept of software criticality and its value. We will then introduce the concept of quality requirements and present a process for defining quality requirements. Lastly, the technique of software traceability, which assures that a requirement has truly been integrated into the software, will be outlined. In a later chapter, we will cover traceability in greater detail.

3.2 Software Quality Models

Unfortunately, in software organizations, software quality models are still rarely used. A number of stakeholders have put forth the hypothesis that these models do not clearly identify all concerns for all of the stakeholders involved and are difficult to use. In the next two sections, we will see that this is simply an excuse for not formally defining and evaluating the quality of software before it is delivered to the client.

Let us begin by considering the five quality perspectives described by Garvin (1984) [GAR 84]. His study makes a link between the work of renowned software quality experts and the proposals of quality models of the era. He questioned whether these models take the different perspectives of quality into account:

  • Transcendental approach to quality: The transcendental view of quality can be explained as follows: “Although I can't define quality, I know it when I see it.” The main problem with this view is that quality is a personal and individual experience. You would have to use the software to get a general idea of its quality. Garvin explains that software quality models all offer a sufficient number of quality characteristics for an individual or an organization to identify and evaluate within their context. In other words, the typical model sets out the quality characteristics for this approach, and it only takes time for all users to see it.
  • User-based approach: A second approach studied by Garvin is that quality software performs as expected from the user's perspective (i.e., fitness for purpose). This perspective implies that software quality is not absolute, but can change with the expectations of each user.
  • Manufacturing-based approach: This view of software quality, in which quality is defined as complying with specifications, is illustrated by many documents on the quality of the development process. Garvin stipulates that models allow for defining quality requirements at an appropriate level of specificity when defining the requirements and throughout the life cycle. Therefore, this is a “process-based” view, which assumes that compliance with the process leads to quality software.
  • Product-based approach: The product-based quality perspective involves an internal view of the product. The software engineer focuses on the internal properties of the software components, for example, the quality of its architecture. These internal properties correspond to source code characteristics and require advanced testing techniques. Garvin explains that if the client is willing to pay, then this perspective is possible with the current models. He describes the case of NASA, who was willing to pay an extra thousand dollars per line of code to ensure that the software aboard the space shuttle met high quality standards.
  • Value-based approach: this perspective focuses on the elimination of all activities that do not add value, for example the drafting of certain documents as described by Crosby (1979) [CRO 79]. In the software domain, the concept of “value” is synonymous with productivity, increased profitability, and competitiveness. It results in the need to model the development process and to measure all kinds of quality factors. These quality models can be used to measure these concepts, but really only for insiders and mature organizations.

We will now describe these quality models. Over the past 40 years, researchers have attempted to define “THE” model of software quality. Of course, it has taken a while to get there. The following section describes the prior initiatives that influenced the software quality standard ISO 25000 [ISO 14a] that we use today: McCall, Richards, and Walter, and the IEEE 1061 [IEE 98b] standard.

3.2.1 Initial Model Proposed by McCall

McCall and his colleagues have been attributed with the original idea for a software quality model [MCC 77]. This model was developed in the 1970s for the United States Air Force and was designed for use by software designers and engineers. It proposes three perspectives for the user (see Figure 3.1) and primarily promotes a product-based view of the software product:

  • Operation: during its use;
  • Revision: during changes made to it over the years;
  • Transition: for its conversion to other environments when the time comes to migrate to a new technology.

Figure 3.1 The three perspectives and 11 quality factors of McCall et al. (1977) [MCC 77].

Each perspective is broken down into a number of quality factors. The model proposed by McCall and his colleagues lists 11 quality factors.

Each quality factor can be broken down into several quality criteria (see Figure 3.2). In general, the quality factors considered by McCall are internal attributes that fall under the responsibility and control of software engineers during software development. Each quality factor (on the left side of Figure 3.2) is associated with two or more quality criteria (which are not directly measurable in the software). Each quality criterion is defined by a set of measures. For example, the quality factor “reliability” is divided into two criteria: accuracy and error tolerance. The right side of Figure 3.2 presents the measurable properties (called “quality criteria”), which can be evaluated (through observation of the software) to assess quality. McCall proposes a subjective evaluation scale of 0 (minimum quality) to 10 (maximum quality).

Figure 3.2 Quality factors and criteria from McCall et al. (1977) [MCC 77].

The McCall quality model was primarily aimed at software product quality (i.e., the internal perspective) and did not easily tie in with the perspective of the user who is not concerned with technical details. Take, for example, a car owner who is not concerned with the metals or alloys used to make the engine. He expects the car to be well designed so as to minimize frequent and expensive maintenance costs. Another criticism regarding this model was that it involved far too many measurable properties (approximately 300).

3.2.2 The First Standardized Model: IEEE 1061

The IEEE 1061 standard, that is, the Standard for a Software Quality Metrics Methodology [IEE 98b], provides a framework for measuring software quality that allows for the establishment and identification of software quality measures based on quality requirements in order to implement, analyze, and validate software processes and products. This standard claims to adapt to all business models, types of software, and all of the stages of the software life cycle. The IEEE 1061 standard presents examples of measures without formally prescribing specific measures.

Figure 3.3 illustrates the structure of the major concepts proposed in this quality model. At the top tier, we can see that software quality requires prior specification of a certain number of quality attributes, which serve to describe the final quality desired in the software. The attributes desired by clients and users allow for the definition of the software quality requirements. There must be a consensus with respect to these requirements within the project team, and their definitions should be clearly written in both the project and technical specifications.

Figure 3.3 Framework for measuring software quality as per the IEEE 1061 [IEE 98b].

Quality factors suggested by this standard are assigned attributes at the next tier. At the tier below that, and only if necessary, subfactors can then be assigned to each quality factor. Lastly, measures are associated with each quality factor, allowing for a quantitative evaluation of the quality factor (or subfactor).

This model is interesting since it provides defined steps to use quality measures in the following situations:

  • Software program acquisition: In order to establish a contractual commitment regarding quality objectives for client-users and verify whether they were met by allowing their measurement when adapting and releasing the software.
  • Software development: In order to clarify and document quality characteristics on which designers and developers must work in order to respect the customer's quality requirements.
  • Quality assurance/quality control/audit: In order to enable those outside the development team to evaluate the software quality.
  • Maintenance: Allow the maintainer to understand the level of quality and service to maintain when making changes or upgrades to the software.
  • Client/user: Allow users to state quality characteristics and evaluate their presence during acceptance tests (i.e., if the software does not meet the specifications agreed upon by the developer, the client can negotiate to his advantage conditions to have them met; for example, the client could negotiate free software maintenance for a specific period of time or the addition of functions at no additional cost).

The following steps are proposed under the IEEE 1061 [IEE 98b] standard:

  • Start by identifying the list of non-functional (quality) requirements for the software as of the beginning of the specifications elicitation. To define these requirements, contractual stipulations, standards, and company history must be taken into account. Set a priority for these requirements and try not to resolve any conflicting quality requirements at this time. Make sure that all participants can share their opinion when collecting information.
  • Make sure that you meet everybody involved to discuss the factors that should be considered.
  • Make a list and make sure to resolve any conflicting points of view.
  • Quantify each quality factor. Identify the measures to evaluate this factor and the desired objective in order to meet the threshold and level of quality expected.
  • Have measures and thresholds approved. This step is important, since it will identify the possibility of carrying out these measurements within your organization. The standard suggests that you customize and document Figure 3.4 for your specific project.
  • Perform a cost–benefit study to identify the costs of implementing the measures for the project. This may be necessary due to:
  • additional costs to enter information, automate calculations, interpret, and present the results;
  • costs to modify support software;
  • costs for software assessment specialists;
  • the purchase of specialized software to measure the software application;
  • training required to apply the measurement plan.
  • Implement the measurement method: Define the data collection procedure, describing storage, responsibilities, training, etc. Prototype the measurement process. Choose which part of the software on which to apply the measures. Use the result to improve the cost-benefit analysis. Collect data and calculate values observed from quality factors.
  • Analyze the results: Analyze the differences between the measurements obtained and the expected values. Analyze significant differences. Identify measures outside the expected limits for further analysis. Make decisions based on quality (redo or continue). Use validated measures to make predictions during development. Use measures to validate the quality during tests.
  • Validate the measures: It is necessary to identify measures that can predict the value of quality factors, which are numeric representations of quality requirements. Validation is not universal, but must be done for each project. To validate measures, the standard recommends using the following techniques:
  • Linear correlation: If there is a positive correlation, the measure can be used as a substitute for the factor.
  • Identification of variations: If a factor goes from F1 (time t1) to F2 (time t2), the measure must change in the same way. This criterion ensures that the measure chosen can detect changes in quality.
  • Consistency: If F1 > F2 > F3, then M1 > M2 > M3. This allows us to sort products based on quality.
  • Foreseeability: (Fa − Fp)/Fp < A (Fa actual factor at time t, Fp anticipated factor at time t, A constant). This factor evaluates whether the measurement formula can predict a quality factor with the desired accuracy (A).
  • Power of discrimination: The measures must be able to distinguish between high-quality software and low-quality software.
  • Reliability: This measure must show that in P%, the correlation, identification, consistency, and foreseeability are valid.

Figure 3.4 Quality model for an ISO 25010 software product.

Source: ISO/IEC 25010. Reproduced with permission of the Standards Council of Canada.

The IEEE 1061 [IEE 98b] standard has allowed us to put measurement into practice and to link product measures with client-user requirements. Since this standard was classified as a guide, it was not very popular outside of the military for several reasons:

  • seen as being too expensive;
  • some did not see its usefulness;
  • the industry was not ready to use it;
  • suppliers did not want to be measured in this way by their clients.

This American standard influenced international software quality standards. The ISO 25000 [ISO 14a] standard is presented next.

3.2.3 Current Standardized Model: ISO 25000 Set of Standards

It was during the eighth international conference in 1980, at The Hague in the Netherlands, that Japan proposed setting up an ISO committee to standardize an internationally recognized software quality model. A work group was created, workgroup 6, and it was assigned to Professor Motoei Azuma (an Emeritus Professor at Waseda University, Tokyo, Japan) who, in turn, asked for help from the international community to study the proposals and possible solutions.

The international standardization of a software quality model, the ISO 9126 standard [ISO 01], was published for the first time in 1991. As we can see by the terminology used by McCall et al. (1977) [MCC 77] and the IEEE 1061 [IEE 98b] standard, there are a number of definitions and terms that were reused. The ISO 9126 [ISO 01] standard has attempted to promote the systematic implementation of software quality measures since 1991. However, it is little known and not often used in industry and was replaced by the ISO/IEC 25000 standard [ISO 14a]. We continue to see that manufacturers, suppliers, and major consulting firms avoid the formal use of the standard in their services. The main reason is that it creates requirements for quality and for guarantees that they are trying to avoid. Gradually and inevitably, this standard will become essential for software professionals.

On February 11, 1993, the Treasury Board of Canada issued an internal directive, Directive NCTTI 26 entitled “Software Evaluation—Quality Characteristics of Software and Usage Guidelines” [CON 93]. This directive proposes the practical use of the International Organization for Standardization (ISO) standard. This internal standard for information technology at the Treasury Board supports the government's policy for improving the management of information technology, which requires the adoption of quality management practices, including control systems, in order to promote the acquisition, use and management of information technology resources in an innovative and cost effective manner. Since the software industry had gained a certain level of maturity and that software was now an essential component of a number of government products and services, it was necessary to be concerned with their quality. They went on to say that given the increasing demands for quality and safety, in the future, evaluation of software quality should be completed using a quality model proposed by ISO.

The Treasury Board concluded that there were basically two ways to determine the quality of a software product: (1) assess the quality of the development process, and (2) assess the quality of the final product. The ISO 25000 [ISO 14a] standard allows for the evaluation of the quality of the final software product.

In some cases, ministries and agencies may decide not to use this standard, particularly when it is much more advantageous in regards to performance or cost, it is for the general advantage of the Government of Canada, or if the ministry or agency has a contractual obligation or is a partner of an international treaty (such as NATO).

Since 2005, ISO 25000 [ISO 14a] has provided a series of standards for the evaluation of software quality. The purpose of this standard is to provide a framework and references for defining the quality requirements of software and the way in which these requirements will be assessed.

The ISO 25000’s series of standards recommends the following four steps [ISO 14a]:

  • set quality requirements;
  • establish a quality model;
  • define quality measures;
  • conduct evaluations.

Note that the ISO 25000 [ISO 14a] standard was selected by the Software Engineering Institute (SEI) as a useful reference for the improvement of performance processes described in the Capability Maturity Model Integration (CMMI®) model. The CMMI model, which describes software engineering models and standards, will be presented in Chapter 4.

The ISO 25010 standard identifies eight quality attributes for software, as illustrated in Figure 3.4.

To illustrate how this standard is used, we will describe the characteristic of maintainability, which has five sub-characteristics: modularity, reusability, analyzability, modifiability, and testability (see Table 3.1). Maintainability is defined, under ISO 25010 [ISO 11i], as being the level of efficiency and efficacy with which software can be modified. Changes may include software corrections, improvements, or adaptation to changes in the environment, requirements, or functional specifications.

Two different perspectives, the internal and external points of view, of the maintainability of the software are often presented in software engineering publications [LAG 96]. Approached from an external point of view, maintainability attempts to measure the effort required to troubleshoot, analyze, and make changes to specific software. From an internal point of view, maintainability usually involves measuring the attributes of the software that influence this change effort. The internal measurement of maintainability is not a direct measurement, that is, a single measurement for software cannot be used and multiple attributes must be measured to draw conclusions about its internal maintainability [PRE 14].

Note that the ISO 25010 standard proposes a wide range of measures for maintainability: size (e.g., number of lines of modified code), time (internal—in terms of software execution, and external—as perceived by the client), effort (individual or for a task), units (e.g., number of production failures, number of attempts to correct a production failure), rating (results of formulas, percentages, or ratios for several characteristics or types of measures, e.g., correlation between the complexity of the software and the testing effort when modifying the software).

From an internal point of view, in order for software to be maintainable, designers must pay special attention to its architecture and internal structure. Architecture and software structure measures are generally extracted from the source code through observations of the characteristics represented in the form of a graph1 describing its classes, methods, programs, and functions of the source code of programs. Graph studies help to determine the level of complexity of the software. Even today, there are a large number of publications focused on the static and dynamic evaluation of source code. These studies are inspired by those performed in the 1970s by McCabe (1976) [MCC 76], Halstead (1978) [HAL 78], and Curtis (1979) [CUR 79].

Today we see a large number of commercial software programs and open-source, such as Lattix, Cobertura, and SonarQube, which can measure the internal characteristics of source code. These products contain ready-to-use measures and also enable the user to design new measures based on their specific requirements. Boloix [BOL 95] specified that it is the interpretation of these measures that is difficult, since they are very specialized, and there are not many mechanisms to summarize information for decision making. Software and SQA practitioners often end up with highly technical measures without a lot of added value that can be communicated directly to management or their clients.

Professor April [APR 00] describes the experience Cable & Wireless Inc. had using three commercial programs used to measure the internal quality of software (Insure++, Logiscope, and Codecheck) and how certain characteristics of maintainability can be quantified from source code. In this way, a well-designed software can be built with independent, modular, and concealed components with clear boundaries and standardized interfaces.

By using the principle of concealing information when designing the software, greater benefits can be obtained when changes are needed during tests or after release. This technique has the added advantage of protecting the other parts of the application when changes are needed. Any “side effects” can thus be reduced as a result of the change [BOO 94].

Routine coupling measures, for their part, help determine whether the software components are independent or not. It is specifically this code measure that helps to identify whether there will be “side effects” upon modifying the source code. A measure of the internal documentation of a given software may also be automatically carried out using these and similar tools. The internal documentation of software helps the maintenance programmer to understand, in more detail, the meaning of the variables and the logic behind a group of instructions. Some measures point to the necessity of using a simple programming style. Certain measures will evaluate whether the programmers upheld programming standards over time.

A substantial part of maintenance costs goes toward functional adaptations, which become necessary given the changes required by users. “According to our observations and data, we also see that well-structured software makes it easier to make changes than with poorly designed software” [FOR 92].

We know that structured programming techniques are based on breaking down a complex problem into simpler parts and force each component to have a single input and a single output. The most common source code measures evaluate complexity and size, and help programmers form an opinion on the number of sets of test cases required as well as the complexity of the decisions in the source code.

Note that certain non-functional requirements can have a negative interaction with each other. For example, for usability and efficiency characteristics, the additional code and the time required to execute this code to increase usability will take up more storage space and entail greater processing times, potentially negatively affecting the efficiency of the code.

Table 3.2 illustrates, for other quality characteristics, the positive (+), negative (−), or neutral (0) interactions. It is important to explain to users that they have choices to make, and that each choice will have implications.

Table 3.1 Quality Factors of the ISO 25000 Standard [ISO 14a]

Factor Description
Performance efficiency Performance relative to the amount of resources used under stated conditions
• Time behavior Degree to which the response and processing times and throughput rates of a product or system, when performing its functions, meet requirements (benchmark)
• Resource utilization Degree to which the amounts and types of resources used by a product or system when performing its functions meet requirements
• Capacity Degree to which the maximum limits of a product or system parameter meet requirements
Functional suitability Degree to which a product or system provides functions that meet stated and implied needs when used under specified conditions
• Functional completeness Degree to which the set of functions covers all the specified tasks and user objectives
• Functional correctness Degree to which a product or system provides the correct results with the needed degree of precision
• Functional appropriateness Degree to which the functions facilitate the accomplishment of specified tasks and objectives. As an example: a user is only presented with the necessary steps to complete a task, excluding any unnecessary steps
Compatibility Degree to which a product, system or component can exchange information with other products, systems or components, and/or perform its required functions, while sharing the same hardware or software environment
• Coexistence Degree to which a product can perform its required functions efficiently while sharing a common environment and resources with other products, without detrimental impact on any other product
• Interoperability Degree to which two or more systems, products, or components can exchange information and use the information that has been exchanged
Usability Degree to which a product or system can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use
• Appropriateness recognizability Degree to which users can recognize whether a product or system is appropriate for their needs. Appropriateness recognizability will depend on the ability to recognize the appropriateness of the product or system's functions from initial impressions of the product or system and/or any associated documentation.
• Learnability Degree to which a product or system can be used by specified users to achieve specified goals of learning to use the product or system with effectiveness, efficiency, freedom from risk, and satisfaction in a specified context of use
• Operability Degree to which a product or system has attributes that make it easy to operate and control
• User error protection Degree to which a system protects users against making errors
• User interface aesthetics Degree to which a user interface enables pleasing and satisfying interaction for the user
• Accessibility Degree to which a product or system can be used by people with the widest range of characteristics and capabilities to achieve a specified goal in a specified context of use
Reliability Degree to which a system, product or component performs specified functions under specified conditions for a specified period of time
• Maturity Degree to which a system meets needs for reliability under normal operation
• Availability Degree to which a system, product or component is operational and accessible when required for use
• Fault tolerance Degree to which a system, product, or component operates as intended despite the presence of hardware or software faults
• Recoverability Degree to which, in the event of an interruption or a failure, a product or system can recover the data directly affected and re-establish the desired state of the system
Security Degree to which a product or system protects information and data so that persons or other products or systems have the degree of data access appropriate to their types and levels of authorization
• Confidentiality Degree to which a product or system ensures that data are accessible only to those authorized to have access
• Integrity Degree to which a system, product or component prevents unauthorized access to, or modification of, computer programs or data.
• Non-repudiation Degree to which actions or events can be proven to have taken place, so that the events or actions cannot be repudiated later
• Accountability Degree to which the actions of an entity can be traced uniquely to the entity
• Authenticity Degree to which the identity of a subject or resource can be proved to be the one claimed
Maintainability Degree of effectiveness and efficiency with which a product or system can be modified by the intended maintainers
• Modularity Degree to which a system or computer program is composed of discrete components such that a change to one component has minimal impact on other components
• Reusability Degree to which an asset can be used in more than one system, or in building other assets
• Analyzability Degree of effectiveness and efficiency with which it is possible to assess the impact on a product or system of an intended change to one or more of its parts, or to diagnose a product for deficiencies or causes of failures, or to identify parts to be modified
• Modifiability Degree to which a product or system can be effectively and efficiently modified without introducing defects or degrading existing product quality
• Testability Degree of effectiveness and efficiency with which test criteria can be established for a system, product, or component and tests can be performed to determine whether those criteria have been met
Portability Degree of effectiveness and efficiency with which a system, product, or component can be transferred from one hardware, software or other operational or usage environment to another
• Adaptability Degree to which a product or system can effectively and efficiently be adapted for different or evolving hardware, software or other operational or usage environments
• Installability Degree of effectiveness and efficiency with which a product or system can be successfully installed and/or uninstalled in a specified environment
• Replaceability Degree to which a product can be replaced by another specified software product for the same purpose in the same environment

Table 3.2 Examples of Interactions Between Quality Attributes [EGY 04]

  Effect

Requirement attribute

Functionality

Functionality

+

Efficiency

Usability

+

Reliability

Security

Recoverability

0

Accuracy

0

Maintainability

Efficiency 0 +/− + 0
Usability + +/− + + 0 + + 0
Reliability 0 0 + + 0 0 0 0
Security 0 + + 0 0 0
Recoverability 0 + + 0 + 0 0
Accuracy 0 + 0 0 0 + 0
Maintainability 0 0 0 + + 0 0 +

In conclusion, all software quality models have a similar structure with the same goals. The presence of more or fewer factors, however, is not indicative of a good or bad model. The value of a model of software quality is revealed in its practical use. It is important to completely understand how it works as well as the interaction between factors. Today, the model to use is the ISO model, since it represents an international consensus.

What is important for software developers and SQA is using a standardized model on which they can rely. It is important to use the definitions of the model proposed by ISO 25010 with suppliers. In this way, it is not the personal proposal of a specialist, but the use of a model and definitions that are published internationally and are indisputable.

3.3 Definition of Software Quality Requirements

We have provided an overview of the use of software quality models. In this section, we look at the process of defining quality requirements for software (i.e., a process that supports the use of a software quality model). However, before delving into quality requirements, we will first discuss all requirements expressed by stakeholders during software development projects.

In engineering, especially in public and private RFPs, requirements are the expression of a documented need of what a product or service should be or should provide. They are most often used formally (specified formally), especially in systems engineering and critical system software engineering.

A vision document or an operational concept document, or a specifications document (requirements definition) will be prepared in order to define the high-level problem areas and solutions for the software system to be created. Typically, this document will describe the context of the application with elements such as: the description and business objectives of the stakeholders and key users; the target market and the possible alternatives; the assumptions, dependencies, and constraints; the inventory of the product features to be developed as well as their priority; and requirements for infrastructure and documentation. This document should be clear and concise. It serves as a guide and a reference throughout the software requirements analysis.

In the classical engineering approach, requirements are considered to be prerequisites to the design and development stages of a product. The requirements development phase may have been preceded by a feasibility study, or a design analysis phase for the project.

Once the stakeholders have been identified, activities for software specifications can be broken down into:

  • gather: collect all wishes, expectations, and needs of the stakeholders;
  • prioritize: debate the relative importance of requirements based on, for example, two priorities (essential, desirable);
  • analyze: check for consistency and completeness of requirements;
  • describe: write the requirements in a way that can be easily understood by users and developers;
  • specify: transform the business requirements into software specifications (data sources, values and timing, business rules).

Requirements for software can be summarized as presented in Figure 3.5.

Figure 3.5 Context of software requirements elicitation.

Requirements are generally grouped into three categories:

  1. Functional Requirements: These describe the characteristics of a system or processes that the system must execute. This category includes business requirements and functional requirements for the user.
  2. Non-Functional (Quality) Requirements: These describe the properties that the system must have, for example, requirements translated into quality characteristics and sub-characteristics such as security, confidentiality, integrity, availability, performance, and accessibility.
  3. Constraints: Limitations in development such as infrastructure on which the system must run or the programming language that must be used to implement the system.

Requirements are notoriously difficult to present to an ideal degree. Often, specialized business analysts are used to bridge the gap between software users and software specialists. These analysts have the experience, training and certification required to run meetings with users, express requirements in such a way that they can be easily conveyed as software specifications and also be understood by the end users.

For a few years now, an interest group, the International Institute of Business Analysts, offers certification for business analysts.


Requirement elicitation techniques often take into account the needs, desires, and expectations of all parties involved that must be prioritized. All stakeholders must be identified before this exercise can begin. At the very beginning of the process, an initial analysis activity aims to identify the users’ business requirements. Software requirements are generally documented using text, diagrams, and vocabulary that will be easily understood by users and clients. Business requirements describe certain events that take place during the carrying out of a business process and seek to identify business rules and activities that could be taken into account by the software.

Business requirements are then expressed in terms of functional requirements. A functional requirement describes the function that will be set up to meet a business requirement. They must be clearly expressed and be consistent. No requirement should be redundant or conflict with other requirements. Each requirement must be uniquely identified and be easily understood by the client. They should also be documented. The requirements that will not be taken into account shall be clearly identified as being excluded along with the reason for the exclusion. It is clear that the management of requirements and functional specifications, and of their quality, is an important factor in customer satisfaction.

Can the quality of a requirement be defined? Yes, good software requirements will have the following characteristics:

  • Necessary: They must be based on necessary elements, that is, important elements in the system that other system components cannot provide.
  • Unambiguous: They must be clear enough to be interpreted in only one way.
  • Concise: They must be stated in a language that is precise, brief, and easy to read, which communicates the essence of what is required.
  • Coherent: They must not contradict the requirements described upstream or downstream. Moreover, they must use consistent terminology throughout all requirements statements.
  • Complete: They must all be stated fully in one location and in a manner that does not oblige the reader to refer to other texts to understand what the requirement means.
  • Accessible: They must be realistic regarding their implementation in terms of available finances, available resources, and within the time available.
  • Verifiable: They must allow for the determination of whether they are met or not based on four possible methods—inspection, analysis, demonstration, or tests.

The software developer should take a course in software requirements. One chapter of the SWEBOK® Guide is dedicated to this topic. The software engineer, for his part, has specific standards, such as ISO 29148 [ISO 11f] or IEEE 830 [IEE 98a] to which he may refer. These standards describe the recommended activities for listing all requirements in sufficient detail to allow for the designing of the product and implementing of its quality assurance (including tests). These standards present the activities that must be carried out by the software engineer: the description of each stimulus (input), of each response (output) and of all processing (functions) of the software.

All requirements must be identifiable and traceable. A different format is proposed for writing documentation based on software criticality. Therefore, the software engineer must have a thorough grasp of business analysis and make sure to carry out the following activities regarding software requirements:

  • plan and manage the requirements phase;
  • elicit the requirements;
  • analyze the requirements and related documentation;
  • communicate and ensure approval;
  • evaluate the solution and validate requirements.

Many specialized studies exist in the field of software requirements. This book does not attempt to reiterate this knowledge, but rather to provide an overview. The next section shows how the software engineer must proceed in order to identify quality requirements.

3.3.1 Specifying Quality Requirements: The Process

Quality is often specified or described informally in RFPs, a requirements document, or in a systems requirement document. The software designer must interpret each functional and non-functional requirement to prepare quality requirements from these documents. To do this, he must follow a process. The quality requirements specifications process will allow for:

  • correctly describing quality requirements;
  • verifying whether the practices in place will allow for the development of software that will meet the client's needs and expectations;
  • verifying or assessing that the software developed meets the quality requirements.

To identify quality requirements, the software engineer will have to carry out the steps described in Figure 3.6. These steps can be done at the same time as defining the functional requirements.

Figure 3.6 Steps suggested for defining non-functional requirements.

A non-functional requirement is something required but that does not add business rules in a business process. Some examples of a non-functional requirement include the number of users and transactions supported, transaction response time, disaster recovery time, and security. Therefore, all ISO 25010 quality characteristics and sub-characteristics fall under non-functional or performance requirements.

We will focus here on describing the quality aspects expected by clients that would not necessarily be discussed by business analysts during business requirements discussions.

The importance of non-functional requirements must not be underestimated. When developing the requirements of a system, greater importance is often given to business and functional requirements and describing the processing that the system should provide so that users can complete their tasks. For example, an unstable system or a system whose interface is greatly lacking but that fully meets functional requirements cannot be deemed a success.

It is the responsibility of the software engineer to look at each quality characteristic in the ISO 25010 model and discuss whether it should be taken into account in the project. Quality requirements should also be verifiable and stated clearly in the same way as functional requirements. If the quality requirements are not verifiable, they could be interpreted, applied, and evaluated differently by different stakeholders. If the software engineer does not clarify non-functional requirements, they could be overlooked during software development.

The first activity in Figure 3.6 describes identifying stakeholders. In fact, stakeholders are any person or organization that has a legitimate interest in the quality of the software. A certain number of stakeholders will express different needs and hopes based on their own perspective. These needs and hopes may change during the system life cycle and must be checked when there is any change.

Stakeholders rarely specify their non-functional requirements since they only have a vague understanding of what quality really is in a software product. In practice, they perceive them more as general costs. It is important that you remember to include a representative from the infrastructure group, security group, and any other functional group within the company.

For subcontracted projects, quality requirements will often appear as part of the contractual agreement between the acquirer and the representative.

The acquirer may also require a specific evaluation of the product quality when a certain level of software criticality is required and human lives could be endangered.

The second activity in Figure 3.6 involves developing the questionnaire that presents the external quality characteristics in terms that are easy to understand for managers. This questionnaire introduces the quality characteristics and asks the respondent to choose a certain number of them and identify their importance (see Table 3.3).

Table 3.3 Example of Quality Criteria Documentation

Quality characteristics Importance
Reliability Indispensable
User-friendliness Desirable
Operational safety Non-applicable

The third activity consists of meeting with stakeholders to explain to them what is involved in identifying quality characteristics. The next activity consists of consolidating the different quality requirements put forth and incorporating the decisions and descriptions into a requirements document or a project quality plan. Quality characteristics may be presented in a summary table by specifying their importance as, for example, “Indispensable,” “Desirable,” or “Non-applicable.”

Next, for each characteristic, the quality measure must be described in detail and include the following information (see also the cases described in this chapter):

  • quality characteristic;
  • quality sub-characteristic;
  • measure (i.e., formula);
  • objectives (i.e., target);
  • example.

The last step in defining quality requirements involves having these requirements authorized through consensus. After the quality requirements have been accepted, at different milestones of the project, typically when assessing a project step, these measures will be evaluated and compared with fixed and agreed upon objectives in the specifications.

Of course, in order to do this, the measurement must be implemented. Challenges to implementing a measurement program will be described in a later chapter.

3.4 Requirement Traceability During the Software Life Cycle

Throughout the life cycle, client needs are documented and developed in different documents, such as specifications, architecture, code, and user manuals. Moreover, throughout the life cycle of a system, many changes regarding client needs should be expected. Every time a need changes, it must be ensured that all documents are updated. Traceability is a technique that helps us follow the development of needs as well as their changes.

Traceability is the association between two or more logical entities, such as requirements and the elements of a system. Bidirectional traceability allows us to follow how the requirements have changed and been refined over the life cycle. Traceability is the common thread that links the elements together: when an element is linked upstream to one element and this element is itself linked downstream to another, a chain of cause and effect is thereby formed. In the chapter on audits and validation, we will look at this topic in more detail.

3.5 Software Quality Requirements and the Software Quality Plan

The IEEE 730 standard specifies, in section 5.4 entitled “Assessing the Product for its Conformity with the Requirements Established” that, in the context of an acquisition of software, the function of SQA should be to ensure that the software products comply with established requirements. It is therefore necessary to produce, collect, and validate evidence to prove that the software product meets the required functional and non-functional requirements.

Moreover, section 5.4.6 of IEEE 730 is entitled “Measuring Products” and specifies that the selected measures of software quality and documentation accurately represent the quality of software products. The following tasks are required by the IEEE 730 [IEE 14]:

  • Identify the necessary standards and procedures;
  • Describe how the measures and attributes chosen adequately represent the quality of the product;
  • Use these measures and identify gaps between objectives and results;
  • Ensure the quality of the product measurement procedures and efficiency throughout the project.

The IEEE 730 recommends that the SQA plan define the concept of product quality for continuous improvement. This document should address the fundamental issues of functionality, external interfaces, performance, quality characteristics, and computational constraints imposed by a specific implementation. Each requirement must be identified and defined so that its implementation can be validated and verified objectively.

3.6 Success Factors

Following is a list of excuses that you may have heard at times from people who do not believe in the importance of quality [SPM 10]:

  • “I do not have to be concerned with quality. My client is only interested in costs and deadlines. Quality has nothing to do with these things.”
  • “My project did not specify quality objectives.”
  • “Quality cannot be measured. We never know the number of bugs that we did not find.”
  • “Relax… it is not software that controls a nuclear plant or a rocket.”
  • “A client might require greater productivity or quality, but not both at the same time.”
  • “To us, quality is important. We do process audits constantly.”
  • “We make quality software since we follow the ISO international standards.”
  • “That's not an error… it's a bug”
  • “We must deliver a very high quality software product, so we set aside lots of time to test it.”
  • “This project is of high quality because the QA guys look at our documents.”
  • “This software is of high quality because we reused 90% of it from another project”
  • “What is quality… how do you measure quality?”
  • “Complexity of code has nothing to do with quality.”
  • “Quality is the number of bugs in the software shipped to the client.”
  • “We must not do more than is asked for in the contract. The contract said nothing about quality.”
  • “If this does not work now, we will repair it later at the client's.”
  • “The quality of this software is very high. We found 1,000 bugs during testing.”
  • “We have no more time to test… we have to deliver.”
  • “We will let the client find the bugs”
  • “We have a very tight schedule. We do not have time to do inspections.”

3.7 Further Reading

  1. GALIN D. Software Quality: Concepts and Practice. Wiley-IEEE Computer Society Press, Hoboken, New Jersey, 2017, 726 p.

3.8 Exercises

  1. Describe the definition of a quality model for the software field, as well as what it must allow the user to easily do.

  2. Consider the McCall quality model:

    1. What do you think about removing the Testability attribute from the model?
    2. Should it be included in Maintainability?

  3. It has been determined that the expandability and survivability attributes are equivalent to flexibility and reliability.

    1. Is there added value to keeping both?
    2. Can they be integrated given the equivalence?
  4. Software quality—Compromises:

    1. In the following space, explain that the optimization of one quality factor can be done to the detriment of another quality factor. Explain your reasoning for the following factors:
        Quality factor Quality factor Reasoning
      1 Maintainability Efficiency (of execution)  
      2 Reusability Integrity  
    2. It is necessary to establish the complementary and opposing links between quality factors. In the following table, for each possible correspondence, specify whether it is complementary (C) or opposing (X), and provide an example to support your choice.
        Integrity Reliability Performance Testability Security Maintainability
      Integrity
      Reliability
      Performance
      Testability
      Security
      Maintainability
    3. Establish the links between software classes and quality factors. Use the following table. For each software class, give a specific example. For example, for sensitive data, you could use a bank system database. Then for each link between software class and quality factor, assign an importance criterion (indispensable (I) or desirable (D)); provide an argument in favor of this association in the appropriate box. Again, using the example of the bank database, data integrity is an indispensable quality factor, since data must always be correct, regardless of the operation performed or external actions, such as a server crash. Note: All classes do not necessarily have a relationship with all quality factors.
        Integrity Reliability Ergonomics Testability Security Maintainability

      Human life in danger e.g.:

      Long life e.g.:

      Experimental system e.g.:

      Real-time application e.g.:

      Embedded application e.g.:

      Sensitive data e.g.:

      Embedded systems e.g.:

      Belonging to a range e.g.:

  5. The request for a proposal to develop a laboratory management software for a medical laboratory network includes non-functional requirements for quality factors specifications. In the following table, you will find articles taken from the requirements document. For each section, fill in the name of the element that best corresponds to the requirement (choose only one factor for the requirements section). Use the following definitions of quality factors to fill in the table.

    • Reliability: Ability of a program to carry out all functions specified in a reference document, in an operating environment, without failure for a given amount of time.
    • Security: Quality attribute for software characterized by the absence of events that endanger the integrity of property or human lives during its operation.
    • Integrity: Level of protection of the system and data that it handles against unauthorized or malicious access.
    • Ergonomics: Ability of the system to be used with a minimum level of effort.
    • Efficacy: Ability of software to optimally use the available physical resources (memory space, central unit time).
    • Testability: Ability of software to be verified appropriately, while the system is in operation.
    • Maintainability: Ability of software to facilitate operations required to locate and correct an error when the system is in operation.
    • Flexibility: Ability of software to adapt to a change in specifications.
    • Reusability: Ability of a software component to be reused in different applications.
    • Portability: Ability of software to adapt to an environment different from that of the previous applications.
    • Compatibility: Quality of several software applications (or components) related to a given function, based on certain criteria, for example, standardization of data structures, internal communications.
      No. Description of the quality requirement Quality factor
      1. The probability that the “Super-lab” system software will fail during normal operating hours (9 A.M. to 4 P.M.) must be less than 0.5%.  
      2. The “Super-lab” system must transfer the laboratory analysis results to the patient file software.  
      3. The “Super-lab” system includes a function that prepares a detailed report of the test results throughout a hospital stay (a copy of this report will also be used for the family doctor). The execution time (preparation and printing) must be less than 60 seconds; the level of precision and completeness of the report must be 99% or greater.  
      4. The “Super-lab” software functionality is targeted for public hospital laboratories. It must allow for easy adaptation to the private laboratory market.  
      No. Description of the quality requirement Quality factor
      5. The training of a technician to use the software must be done in less than three days in order for the technician to reach the “C” skill level for using the software. This skill level must allow him to process more than 20 patient files in an hour.  
      6. The “Super-lab” stores access data. As well, it must report unsuccessful access attempts by unauthorized personnel. This report must include the following information: the identification of the access terminal network, the system code used, the date and time of the access attempt.  
      7. The “Super-lab” system includes a patient billing function for laboratory tests. This sub-system will be reused in the physiotherapy center software.  
      8. The “Super-lab” system will process all monthly reports for each hospital department, as listed in Appendix B of the contract.  
      9. The system must serve 12 workstations and 8 pieces of automatic test equipment from an AS250 centralized server, and a CS25 communications server, which will support 25 lines of communication. The system (software + hardware) must meet or exceed the availability specifications described in Appendix C.  
      10. The “Super-lab” software developed on Linux must also be able to work on Windows.  
  6. Calculate the MTBF for software that must be available during work hours, that is, 37.5 hours a week, and cannot fail more than 15 minutes per week.

  7. Describe the steps proposed by the IEEE 1061 standard to determine a good definition of non-functional requirements.

  8. Explain the two different perspectives, the internal and external point of view, for software maintainability, and give an example.

  9. Describe the three categories used to classify requirements.

  10. What is the difference between a business requirement and a functional requirement?

  11. Describe the steps recommended for defining non-functional requirements.

  12. Explain what is meant by bidirectional traceability of requirements.

Note