Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — Quality measurement framework

This document provides a framework for developing quality measurement. The contents of this document are as follows: — quality measurement reference model; — relationships among different types of quality measures; — guidelines for selecting quality measures; — guidelines for constructing quality measures; — guidelines for planning and performing measurements; — guidelines for the application of measurement results. It includes considerations for selecting quality measures and quality measure elements (Annex A), assessing the reliability of measurement and the validity of quality measures (Annex B), elements for documenting quality measures (Annex C), normalized measurement function for quality measures (Annex D) and the measurement information model in ISO/IEC/IEEE 15939 (Annex E). This document can be applied for designing, identifying, evaluating and executing the measurement model of system and software product quality, quality in use, data quality and IT service quality. This reference model can be used by developers, acquirers, quality assurance staff and independent evaluators—essentially by people responsible for specifying and evaluating the quality of information and communication technology (ICT) systems and services.

Ingénierie des systèmes et du logiciel — Exigences de qualité du produit logiciel et évaluation (SQuaRE) — Modèle de référence de mesure et guide

General Information

Status
Published
Publication Date
11-Jul-2019
Current Stage
6060 - International Standard published
Due Date
09-Dec-2019
Completion Date
12-Jul-2019
Ref Project

Relations

Buy Standard

Standard
ISO/IEC 25020:2019 - Systems and software engineering -- Systems and software Quality Requirements and Evaluation (SQuaRE) -- Quality measurement framework
English language
27 pages
sale 15% off
Preview
sale 15% off
Preview

Standards Content (Sample)

INTERNATIONAL ISO/IEC
STANDARD 25020
Second edition
2019-07
Systems and software engineering —
Systems and software Quality
Requirements and Evaluation
(SQuaRE) — Quality measurement
framework
Ingénierie des systèmes et du logiciel — Exigences de qualité du
produit logiciel et évaluation (SQuaRE) — Modèle de référence de
mesure et guide
Reference number
ISO/IEC 25020:2019(E)
©
ISO/IEC 2019

---------------------- Page: 1 ----------------------
ISO/IEC 25020:2019(E)

COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2019
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2019 – All rights reserved

---------------------- Page: 2 ----------------------
ISO/IEC 25020:2019(E)

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 5
5 Conformance . 5
6 Quality measurement . 5
6.1 Quality measurement reference model . 5
6.2 Different QMs and their relationships . 7
6.3 Selecting QMs .10
6.4 Constructing QMs .10
6.4.1 Identify QMs needed to be constructed .10
6.4.2 Description of the QM .11
6.4.3 Definitions of the QMEs .11
6.5 Plan and perform measurement .12
6.6 Application of the measurement results .13
Annex A (informative) Considerations for selecting QMs and QMEs .14
Annex B (informative) Assessing the reliability of measurement and the validity of QMs .16
Annex C (informative) Elements for documenting QMs .18
Annex D (informative) Normalized measurement function for QMs .21
Annex E (informative) Measurement information model in ISO/IEC/IEEE 15939 .24
Bibliography .27
© ISO/IEC 2019 – All rights reserved iii

---------------------- Page: 3 ----------------------
ISO/IEC 25020:2019(E)

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity
assessment, as well as information about ISO's adherence to the World Trade Organization (WTO)
principles in the Technical Barriers to Trade (TBT) see www .iso .org/iso/foreword .html.
This document was prepared by Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 7, Software and systems engineering.
This second edition cancels and replaces the first edition (ISO/IEC 25020:2007), which has been
technically revised.
The main changes compared to the previous edition are as follows:
— relationships among different types of quality measures have been added;
— application of measurement results and description of quality measure have been added;
— elements for documenting quality measures in Annex C have been supplemented and categorized;
— Annex D has been added showing a normalized measurement function for QMs;
— Annex E has been added showing the measurement information model in ISO/IEC/IEEE 15939;
— harmonized with ISO/IEC 25000:2014, ISO/IEC 25022:2016, ISO/IEC 25023:2016, ISO/IEC 25024:2015
and ISO/IEC/IEEE 15939:2017.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at https: //www .iso .org/members .html.
iv © ISO/IEC 2019 – All rights reserved

---------------------- Page: 4 ----------------------
ISO/IEC 25020:2019(E)

Introduction
0.1  General
This document is a part of the SQuaRE series of International Standards. It provides a framework
for measuring the quality characteristics and sub-characteristics (defined in ISO/IEC 2501n). This
document serves as a guideline for developing and selecting quality measures for quality in use
(in conjunction with ISO/IEC 25022), system and software product quality (in conjunction with
ISO/IEC 25023), data quality (in conjunction with ISO/IEC 25024) and IT service quality (in conjunction
1)
with ISO/IEC TS 25025 ).
0.2  Quality measurement division
This document is a part of ISO/IEC 2502n Quality Measurement Division of the SQuaRE series that
consists of the following International Standards:
— ISO/IEC 25020 — Quality measurement framework: provides a framework for developing quality
measurement;
— ISO/IEC 25021 — Quality measure elements: provides a format for specifying QMEs (Quality Measure
Elements) and a few examples of QMEs that can be used to construct software quality measures;
— ISO/IEC 25022 — Measurement of quality in use: provides measures, including associated
measurement functions for the quality characteristics in the quality in use model;
— ISO/IEC 25023 — Measurement of system and software product quality: provides measures,
including associated measurement functions and QMEs for the quality characteristics in the
product quality model;
— ISO/IEC 25024 — Measurement of data quality: provides measures, including associated measurement
functions and QMEs for the quality characteristics in the data quality model;
— ISO/IEC TS 25025 — Measurement of IT service quality: provides measures for the IT service
quality model.
Figure 1 shows the relationship between this document and other standards in the ISO/IEC 2502n
division.
Figure 1 — Structure of the Quality Measurement Division
1) To be developed.
© ISO/IEC 2019 – All rights reserved v

---------------------- Page: 5 ----------------------
ISO/IEC 25020:2019(E)

0.3  Outline and organization of the SQuaRE series
The SQuaRE series consists of five main divisions and one extension division. The outlines of each
division within the SQuaRE series are as follows.
— ISO/IEC 2500n - Quality Management Division. The standards comprising this division define
all common models, terms and definitions referred further by all other standards in the SQuaRE
series. The division also provides requirements and guidance for the planning and management of
a project.
— ISO/IEC 2501n - Quality Model Division. The standards comprising this division present quality
models for system/software products, quality in use and data. The IT service quality model is
published as a Technical Specification.
— ISO/IEC 2502n - Quality Measurement Division. The standards comprising this division include
a system/software product quality measurement reference model, definitions of quality measures,
and practical guidance for their application. This division presents QMs on internal and external
property of a system and software product, QMs for quality in use, QMs for data quality and QMs for
IT service. Quality measure elements forming the foundations of the quality measures are defined
and presented.
— ISO/IEC 2503n - Quality Requirements Division. The standards comprising this division help
specify quality requirements. These quality requirements can be used in the process of quality
requirements elicitation for a system/software product to be developed, designing a process for
achieving necessary quality, or as inputs for an evaluation process.
— ISO/IEC 2504n - Quality Evaluation Division. The standards comprising this division provide
requirements, recommendations and guidelines for system/software product evaluation, whether
performed by independent evaluators, acquirers or developers. The support for documenting a
quality measure as an Evaluation Module is presented as well.
— ISO/IEC 25050-25099 - SQuaRE Extension Division. These standards are reserved for SQuaRE
extension International Standards, which currently include ISO/IEC 25051 and ISO/IEC TR 25060
to ISO/IEC 25069.
vi © ISO/IEC 2019 – All rights reserved

---------------------- Page: 6 ----------------------
INTERNATIONAL STANDARD ISO/IEC 25020:2019(E)
Systems and software engineering — Systems and software
Quality Requirements and Evaluation (SQuaRE) — Quality
measurement framework
1 Scope
This document provides a framework for developing quality measurement.
The contents of this document are as follows:
— quality measurement reference model;
— relationships among different types of quality measures;
— guidelines for selecting quality measures;
— guidelines for constructing quality measures;
— guidelines for planning and performing measurements;
— guidelines for the application of measurement results.
It includes considerations for selecting quality measures and quality measure elements (Annex A),
assessing the reliability of measurement and the validity of quality measures (Annex B), elements for
documenting quality measures (Annex C), normalized measurement function for quality measures
(Annex D) and the measurement information model in ISO/IEC/IEEE 15939 (Annex E).
This document can be applied for designing, identifying, evaluating and executing the measurement
model of system and software product quality, quality in use, data quality and IT service quality.
This reference model can be used by developers, acquirers, quality assurance staff and independent
evaluators—essentially by people responsible for specifying and evaluating the quality of information
and communication technology (ICT) systems and services.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC/IEEE 15939, Systems and software engineering — Measurement process
ISO/IEC 25000, Systems and software engineering — Systems and software Quality Requirements and
Evaluation (SQuaRE) — Guide to SQuaRE
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 25000 and
ISO/IEC/IEEE 15939 and the following apply.
ISO and IEC maintain terminological databases for use in standardisation at the following addresses:
— ISO Online browsing platform: available at https: //www .iso .org/obp
— IEC Electropedia: available at http: //www .electropedia .org/
© ISO/IEC 2019 – All rights reserved 1

---------------------- Page: 7 ----------------------
ISO/IEC 25020:2019(E)

3.1
attribute
inherent property or characteristic of an entity that can be distinguished quantitatively or qualitatively
by human or automated means
Note 1 to entry: ISO 9000 distinguishes two types of attributes: a permanent characteristic existing inherently in
something; and an assigned characteristic of a product, process or system (e.g. the price of a product, the owner of
a product). The assigned characteristic is not an inherent quality characteristic of that product, process or system.
[SOURCE: ISO/IEC 25000:2014, 4.1, modified — Note 1 to entry has been removed; Note 2 to entry has
become Note 1 to entry.]
3.2
base measure
measure (3.6) defined in terms of an attribute (3.1) and the method for quantifying it
Note 1 to entry: A base measure is functionally independent of other measures.
Note 2 to entry: Based on the definition of “base quantity” in the International Vocabulary of Metrology – Basic
and General Concepts and Associated Terms, 2012.
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.3]
3.3
derived measure
measure (3.6) defined as a function of two or more values of base measures (3.2)
Note 1 to entry: Adapted from the definition of “derived quantity” in the International Vocabulary of Metrology –
Basic and General Concepts and Associated Terms, 2012.
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.8]
3.4
indicator
measure (3.6) that provides an estimate or evaluation of specified attributes (3.1) derived from a model
with respect to defined information needs (3.5)
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.10]
3.5
information need
insight necessary to manage objectives, goals, risks and problems
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.12]
3.6
measure, noun
variable to which a value is assigned as the result of measurement (3.8)
Note 1 to entry: The plural form “measures” is used to refer collectively to base measures (3.2), derived measures
(3.3) and indicators (3.4).
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.15]
3.7
measure, verb
make a measurement (3.8)
[SOURCE: ISO/IEC 25000:2014, 4.19]
2 © ISO/IEC 2019 – All rights reserved

---------------------- Page: 8 ----------------------
ISO/IEC 25020:2019(E)

3.8
measurement
set of operations having the objective of determining a value of a measure (3.6)
Note 1 to entry: Measurement can include assigning a qualitative category such as the language of a source
program (ADA, C, JAVA, etc.).
[SOURCE: ISO/IEC 25000:2014, 4.20]
3.9
measurement function
algorithm or calculation performed to combine two or more base measures (3.2)
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.20]
3.10
measurement method
logical sequence of operations, described generically, used in quantifying an attribute (3.1) with respect
to a specified scale
Note 1 to entry: The type of measurement method depends on the nature of the operations used to quantify an
attribute. Two types can be distinguished:
— subjective: quantification involving human judgment;
— objective: quantification based on numerical rules.
Note 2 to entry: Based on the definition of “method of measurement” in the International Vocabulary of Metrology
– Basic and General Concepts and Associated Terms, 2012.
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.21]
3.11
property to quantify
property of a target entity that is related to a quality measure element (3.14) and which can be quantified
by a measurement method (3.10)
Note 1 to entry: A software artifact is an example of a target entity.
[SOURCE: ISO/IEC 25023:2016, 4.7]
3.12
quality in use
degree to which a product or system can be used by specific users to meet their needs to achieve specific
goals with effectiveness, efficiency, freedom from risk and satisfaction in specific contexts of use
Note 1 to entry: Before the product is released, quality in use can be specified and measured (3.7) in a test
environment designed and used exclusively by the intended users for their goals and contexts of use, e.g. User
Acceptance Testing Environment.
[SOURCE: ISO/IEC 25000:2014, 4.24]
3.13
quality measure
QM
derived measure (3.3) that is defined as a measurement function (3.9) of two or more values of quality
measure elements (3.14)
[SOURCE: ISO/IEC 25021:2012, 4.13, modified — The abbreviated term “QM” has been added.]
© ISO/IEC 2019 – All rights reserved 3

---------------------- Page: 9 ----------------------
ISO/IEC 25020:2019(E)

3.14
quality measure element
QME
measure (3.6) defined in terms of a property and the measurement method (3.10) for quantifying it,
including optionally the transformation by a mathematical function
Note 1 to entry: The system or software quality characteristic or subcharacteristic of the entity is derived
afterwards by calculating a software quality measure (3.13).
[SOURCE: ISO/IEC 25000:2014, 4.26, modified — The abbreviated term “QME” has been added.]
3.15
quality measure on external property
QM on external property
measure (3.6) of the degree to which a system or software product enables its behaviour to satisfy
stated and implied needs for the system including the software to be used under specified conditions
Note 1 to entry: Attributes (3.1) of the behaviour can be measured (3.7), verified and/or validated by executing
the system or software product during testing and operation.
EXAMPLE The failure density against test cases found during testing is a quality measure on external
property related to the number of faults present in the computer system. The two measures are not necessarily
identical since testing may not find all faults, and a fault may give rise to apparently different failures in different
circumstances.
[SOURCE: ISO/IEC 25000:2014, 4.11, modified — The term has been changed from “external measure of
system or software quality" to "quality measure on external property”; "QM on external property" has
been added as an alternative; in Note 1 to entry, the word "measured" has been added; in EXAMPLE,
"number of failures" has been changed to "failure density against test cases".]
3.16
quality measure on internal property
QM on internal property
measure (3.6) of the degree to which a set of static attributes (3.1) of a software product satisfies stated
and implied needs for the software product to be used under specified conditions
Note 1 to entry: Static attributes include those that relate to the software architecture, structure and its
components, data structure and its formats, structure and appearance of graphical display on screen and menus
for users or recipients of service.
Note 2 to entry: Static attributes can be verified by review, inspection, simulation and/or automated tools.
Note 3 to entry: Quality measures on internal property are typically associated with quality requirements on
static properties and attributes that can be specified in or derived from requirements.
EXAMPLE Complexity measures and the number, severity, and failure frequency of faults found in a walk
through are typical quality measures on internal property made on the product itself.
[SOURCE: ISO/IEC 25000:2014, 4.16, modified — The term has been changed from “internal measure of
software quality" to "quality measure on internal property"; "QM on internal property" has been added
as an alternative; in Note 1 to entry, more information on static attributes has been added; Note 3 to
entry has been added.]
3.17
system and software product quality
product quality
capability of a system and/or software to satisfy stated and implied needs when used under specified
conditions
Note 1 to entry: Product quality model refers to the system and software product quality model defined in
ISO/IEC 25010.
4 © ISO/IEC 2019 – All rights reserved

---------------------- Page: 10 ----------------------
ISO/IEC 25020:2019(E)

4 Abbreviated terms
ICT Information and Communication Technology
QM-RM Quality Measurement Reference Model
QM Quality Measure
QME Quality Measure Element
5 Conformance
Any measurement process for system and software product quality and quality in use, data quality and
IT service quality that conforms to this document shall fulfil the requirements of Clause 6.
6 Quality measurement
6.1 Quality measurement reference model
The Quality Measurement Reference Model (QM-RM) describes the relationship between a quality
model and the construction of QMs from QMEs, as shown in Figure 2. The relationship constitutes
the reference model for the measurement of system and software product quality, quality in use, data
quality and IT service quality. The measurement information model presented in Annex E describes the
relationship between attributes and measurement.
© ISO/IEC 2019 – All rights reserved 5

---------------------- Page: 11 ----------------------
ISO/IEC 25020:2019(E)

NOTE Target entity can be a system, a software product, data or IT service.
Figure 2 — Quality Measurement Reference Model (QM-RM)
The quality of a system, software product, data or IT service is the degree to which it satisfies the stated
and implied needs of various stakeholders, and thus provides value. User needs for quality include
requirements for system quality in specific contexts of use. These stated and implied needs are represented
in the SQuaRE series of standards by quality models that categorise quality into characteristics, which
are further subdivided into sub-characteristics. Quality properties are measured using a measurement
method. A measurement method is a logical sequence of operations used to quantify a property against a
specified scale. The result of applying a measurement method is called a QME.
QMs are constructed by applying a measurement function to a set of QMEs. A measurement function is
an algorithm used to combine QMEs. The result of applying a measurement function is called a QM. In
this way, QMs serve as quantifications of quality characteristics (and sub-characteristics). More than
one QM may be used for measuring a quality characteristic (and sub-characteristics).
In the special case where the QME serves as a QM as well, the measurement function applied would
be the identity function. QMEs may either be base or derived measures. Annex B provides assessment
information for the validation and verification of the measure. QMEs are constructed based on the
guidance provided in ISO/IEC/IEEE 15939. Refer to ISO/IEC 25030 for guidance on selecting quality
characteristics and sub-characteristics of interest in conjunction with the specification of quality
requirements and ISO/IEC 25040 for guidance on using software QMs for software product evaluation.
6 © ISO/IEC 2019 – All rights reserved

---------------------- Page: 12 ----------------------
ISO/IEC 25020:2019(E)

6.2 Different QMs and their relationships
There are four quality models in the SQuaRE series:
a) the quality in use model in ISO/IEC 25010 that can be applied to software products, systems and IT
services;
b) the product quality model in ISO/IEC 25010 that can be applied to systems and software products;
c) the data quality model in ISO/IEC 25012 that can be applied to data within a computer system and
used by humans and systems;
d) the IT service quality model defined in ISO/IEC/TS 25011 that can be applied to IT services that
support the needs of an individual user or a business.
These models provide a set of quality characteristics and sub-characteristics, as well as their definitions.
The relationships among various QMs for different quality models from the SQuaRE series are shown in
Figure 3.
Figure 3 — Relationships among different QMs
© ISO/IEC 2019 – All rights reserved 7

---------------------- Page: 13 ----------------------
ISO/IEC 25020:2019(E)

QMs for quality in use are defined or selected to specify stakeholder requirements in a specific context
of use, when quality requirements are derived from user needs by analysing the concept of operation.
QMs for quality in use are to measure the extent to which a product meets the needs of specific
users with respect to their specific personal or business goals by means of quantifying outcomes of
interaction between a user and a system or of effects to stakeholders, including indirect users as well as
direct users. These measures can only be prepared in a realistic and operational system environment.
QMs on external and internal property of product are for the user (including executing testing
engineer) and the developer, respectively. There is no distinction between the two, even at the level of
characteristics and sub-characteristics. However, when users apply the QMs depending on the purpose
and stage of the software product life cycle, the QME and QM selected should be related and suitable
to either the user or the developer. QMs on external property are used to measure the quality of the
system and software product based on the behaviour of the system. QMs on external property are used
in the testing and operational stages of the product life cycle. QMs on internal property allow users to
measure the quality of intermediate deliverables or work products. Additionally, these measures may
be used with an analysis model to predict the quality of the final system and software product. This
allows users to detect system and software product quality issues and take corrective and preventive
actions during the early stages of the development life cycle.
Data quality measures can be transformed from quality in use, system and software product quality
requirements and measures. Then, these measures representing the targeted data quality requirements
are used to evaluate the data quality of system and software product, to verify, validate and improve
data and product phase-by-phase during design, implementation, testing or in use. QMs for data quality
are to measure data in the system and software product from two viewpoints, namely, “Inherent” and
“System-dependent”, to detect potential quality problems related to the data and database. These QMs
can be applied during the development, testing and operation stages. Data quality has a big influence on
quality in use, in particular for effectiveness, usefulness and risks management.
QMs for IT service quality quantify the degree to which the properties of an IT service can satisfy the
stated and implied needs of the IT service when used under specified conditions. IT service has its own
provision system. QMs for IT service quality typically measure interactions between the s
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.