Methods for Testing and Specification (MTS);

ETSI TR 103 386 V1.1.1 (2015-06)

Methods for Testing and Specification (MTS);

Deployment of Model-Based Automated Testing Infrastructure in a Cloud

TECHNICAL REPORT

ETSI TR 103 386 V1.1.1 (2015-06)

Reference

DTR/MTS-103386

Keywords

testing, cloud, infrastructure

ETSI

650 Route des Lucioles

F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16

Siret N° 348 623 562 00017 - NAF 742 C

Association à but non lucratif enregistrée à la

Sous-préfecture de Grasse (06) N° 7803/88

Important notice

The present document can be downloaded from:
http://www.etsi.org/standards-search

The present document may be made available in electronic versions and/or in print. The content of any electronic and/or print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat.

Users of the present document should be aware that the document may be subject to revision or change of status. Information on the current status of this and other ETSI documents is available at http://portal.etsi.org/tb/status/status.asp

If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx

Copyright Notification

No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm except as authorized by written permission of ETSI.

The content of the PDF version shall not be modified without the written authorization of ETSI.

The copyright and the foregoing restriction extend to reproduction in all media.

DECTTM, PLUGTESTSTM, UMTSTM and the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members.
3GPPTM and LTE™ are Trade Marks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.

Contents (style TT)

If you need to update the Table of Content you would need to first unlock it.
To unlock the Table of Contents: select the Table of Contents, click simultaneously: Ctrl + Shift + F11.
To update the Table of Contents: F9.
To lock it: select the Table of Contents and then click simultaneously: Ctrl + F11.

Contents (style TT) 3

Intellectual Property Rights 5

Foreword 5

Modal verbs terminology 5

Executive summary (style H1) 5

Introduction (style H1) 5

1 Scope (style H1) 6

2 References 6

2.1 Normative references 6

2.2 Informative references (style H2) 6

3 Definitions, symbols and abbreviations (style H1) 8

3.1 Definitions (style H2) 8

3.2 Symbols (style H2) 11

3.3 Abbreviations (style H2) 11

4 An integrated framework for testing automation on a cloud infrastructure 13

4.1 Roles, relationships and interactions among TaaS users 13

4.1.1 End user core services 14

5 End user use cases 16

5.1 Direct test execution use case 16

5.1.1 Direct test execution use case TaaS sequence diagram 17

5.2 Manual test design use case 19

5.2.1 Manual test design use case TaaS sequence diagram 20

5.3 Automated test design use case 21

5.3.1 Automated test design use case TaaS sequence diagram 22

6 Representation of test descriptions in TPaaS 23

6.1 MDSL conceptual model 23

6.1.1 Test Planning Concept 24

6.1.2 Test Analysis Concepts 24

6.1.3 Test Design Concepts 26

6.1.4 Test Case Concepts 26

6.1.5 Test Data Concepts 27

6.1.6 Test Derivation Concepts 28

6.1.7 Refined Test Design Concepts 30

6.1.8 Test Scheduling Concepts 31

6.2 Realisation as UML Profiles 33

6.2.1 Test Planning Concepts Implementation 33

6.2.3 Test Requirement Implementation 33

6.2.4 Test Object Implementation 33

6.2.5 Test Component Implementation 34

6.2.6 SUT Implementation 34

6.2.7 Test Configuration Implementation 34

6.2.8 Test Case Implementation 34

6.2.9 Precondition Implementation 34

6.2.10 Postcondition Implementation 34

6.2.11 Parameter Implementation 34

6.2.12 Stimulus Implementation 34

6.2.13 Response Implementation 34

6.2.14 Verdict Implementation 35

6.2.15 Test Design Model Implementation 35

6.2.16 TestData Implementation 35

6.2.17 DataPartition Implementation 35

6.2.18 TestDataValue Implementation 35

6.2.19 DataPool Implementation 35

6.2.20 Test Suite Implementation 36

6.2.21 Test Procedure Implementation 36

6.2.22 Scheduling Specification Implementation 36

6.3 Constraints on the MIDAS DSL 36

6.3.1 TestConfiguration/TestContext Constraints 36

6.3.2 TestCase Constraints 37

6.3.3 TestProcedure Constraints 39

6.4 MDSL Validator 40

6.5 TTCN-3 Generator 40

7 Deployment of the TPaaS on the public cloud infrastructure 42

7.1 Integration of test methods on the TPaaS platform 42

7.1.1 The Database structure for the MIDAS TPaaS 43

7.1.2 The storage file system for MIDAS TPaaS 44

7.2 Implemented facilities 45

7.2.1 Development Environment (devenv_vm) 45

7.2.2 Production Environment (prodenv_multivm) 47

Annex A: Direct Execution Use Case Example – IMS Comformance testing 49

A.1 First clause of the annex (style H1) 49

A.1.1 First subdivided clause of the annex (style H2) 49

Annex B: Manual test design example - SCM Pilot 50

B.1 First clause of the annex (style H1) 50

B.1.1 First subdivided clause of the annex (style H2) 50

Annex C: Automated test design example - e-Health Pilot 51

C.1 First clause of the annex 51

Annex E: Bibliography (style H9) 52

Annex F: Change History (style H9) 53

History (style H1) 54

A few examples: 54

Intellectual Property Rights

IPRs essential or potentially essential to the present document may have been declared to ETSI. The information pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found in ETSISR000314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (http://ipr.etsi.org).

Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSISR000314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present document.

Foreword

This Technical Report (TR) has been produced by ETSI Technical Committee Methods for Testing and Specification (MTS).

Modal verbs terminology

In the present document "shall", "shall not", "should", "should not", "may", "may not", "need", "need not", "will", "will not", "can" and "cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions).

"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.

Executive summary (style H1)

This unnumbered clause, if present, appears after the "Modal verbs terminology" and before the "Introduction". It is an optional informative element and shall not contain requirements.

The "Executive summary" is used, if required, to summarize the ETSI deliverable. It contains enough information for the readers to become acquainted with the full document without reading it. It is usually one page or shorter.

Introduction (style H1)

This unnumbered clause, if present, appears just before the "Scope". It is an optional informative element and shall not contain requirements.

1 Scope (style H1)

This clause numbered 1 shall start on a new page. More details can be found in clause 2.9 of the EDRs.

The Scope shall not contain requirements. Forms of expression such as the following should be used:

The present document will provide an overview ... of the for the deployment of MBT automated testing infrastructure as TPaaS (Test Platform as a Service); - for the deployment of (3rd-party) test methods for automated test suite generation, scheduling, execution, and test arbitration within TPaaS; - for the definition and use of Domain Specific Language (DSL) for developing SUT models and Test models that are compliant with the test methods used within TPaaS.

2 References

References are either specific (identified by date of publication and/or edition number or version number) or nonspecific. For specific references, only the cited version applies. For non-specific references, the latest version of the referenced document (including any amendments) applies.

Referenced documents which are not found to be publicly available in the expected location might be found at http://docbox.etsi.org/Reference.

NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee their long term validity.

2.1 Normative references

The following referenced documents are necessary for the application of the present document.

Not applicable.

2.2 Informative references (style H2)

Clause 2.2 shall only contain informative references, which are cited in the document itself.

NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee their long term validity.

The following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject area.

· Use the EX style, add the letter "i" (for informative) before the number (which shall be in square brackets) and separate this from the title with a tab (you may use sequence fields for automatically numbering references, see clause 6.9.2: "Sequence numbering") (see example).

EXAMPLE:

[i.1] 4 [tab] <Standard Organization acronym> <document number>: "<Title>".

[i.2] 4 [tab] <Standard Organization acronym> <document number>: "<Title>".

The following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject area.

[i.1] ETSI ES 202 951: "Methods for Testing and Specification (MTS); Model-Based Testing (MBT); Requirements for Modelling Notations".

[i.2] ETSI ES 203 119: "Methods for Testing and Specification (MTS); The Test Description Language (TDL); Specification of the Abstract Syntax and Associated Semantics".

[i.3] MIDAS Deliverable D2.2.WP2: "Architecture and specifications of the MIDAS framework and platform", 2013.

[i.4] MIDAS Deliverable D3.6.WP3: "Generating TTCN-3 from PIM UML SAUT Specifications", 2014.

[i.5] MIDAS Deliverable D6.1.WP6: " Analysis of required functionalities and available public Cloud services", 2014.

[i.6] MIDAS Deliverable D6.2.WP6: "Specification and design of the basic MIDAS platform as a service on the Cloud", 2014.

[i.7] MIDAS Deliverable D6.3.WP6: "The basic MIDAS platform and the integrated test evaluation, planning and scheduling macro-component", 2014.

[i.8] ISO 9126

[i.9] ISO 9000

[i.10] International Organisation for Standardisation (ISO): ISO/IEC 29119, Software Testing Standard, http://www.softwaretestingstandard.org

[i.11] UTP_1_2 (2013). UML testing profile (UTP) version 1.2. Tech. Rep. formal/2013-04-03, Object Management Group.

[i.12] International Software Testing Qualifications Board (ISTQB): ISTQB/GTB standard glossary for testing terms. http://www.software-tester.ch/PDF-Files/CT_Glossar_DE_EN_V21.pdf.

[i.13] Object Management Group (OMG): Business Motivation Model (BMM). http://www.omg.org/spec/BMM

[i.14] IEEE Std. 610-12

[i.15] NIST

3 Definitions, symbols and abbreviations (style H1)

Delete from the above heading the word(s) which is/are not applicable, (see clause 2.11 of EDRs).

Definitions and abbreviations extracted from ETSI deliverables can be useful when drafting documents and can be consulted via the Terms and Definitions Interactive Database (TEDDI) (http://webapp.etsi.org/Teddi/).

3.1 Definitions (style H2)

Clause numbering depends on applicability.

· A definition shall not take the form of, or contain, a requirement.

· The form of a definition shall be such that it can replace the term in context. Additional information shall be given only in the form of examples or notes (see below).

· The terms and definitions shall be presented in alphabetical order.

The following text block applies. More details can be found in clause 2.11.1 of the EDRs.

For the purposes of the present document, the [following] terms and definitions [given in ... and the following] apply:

Definition format

· Use the Normal style.

· The term shall be in bold, and shall start with a lower case letter (unless it is always rendered with a leading capital) followed by a colon, one space, and the definition starting with a lower case letter and no ending fullstop.

example 1: text used to clarify abstract rules by applying them literally

NOTE: This may contain additional information.

For the purposes of the present document, the following terms and definitions apply:

Accuracy: the capability of the software product to provide the right or agreed results or effects with the needed degree of precision. (ISO 9126)

Black-box testing: testing, either functional or non-functional, without reference to the internal structure of the component or system. (ISTQB Glossary)

Cloud computing: a computing model for enabling ubiquitous, convenient, on-demand network access to a shared pool of conﬁgurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. (NIST)

Cloud provider: is a (service) provider that offers customers storage or software services available via a private (private cloud) or public network (public cloud). The storage and software are available for access via the Internet and the cloud provider manages the infrastructure and platforms on which the applications run.

Coverage: the degree, expressed as a percentage, to which a specified coverage item has been exercised by a test suite. (ISTQB Glossary)

Equivalence partitioning: a black box test design technique in which test cases are designed to execute representatives from equivalence partitions. In principle test cases are designed to cover each partition at least once. (ISTQB Glossary)

Event: an observable action of a software that is characterized by its type and its target, e.g., a user interaction with a system with the type mouse click and the target Ok button.

Failure: deviation of the component or system from its expected delivery, service or result. (ISTQB Glossary)

Functional testing: testing based on an analysis of the specification of the functionality of a component or system. See also black-box testing. (ISTQB Glossary)

Fuzzing: see Fuzz testing