Cloud Application Management for Platforms Test Assertions Version 1.1
Working Draft 02
02 April 2013
Technical Committee:
OASIS Cloud Application Management for Platforms (CAMP) TC)
Chair:
Martin Chapman (), Oracle
Editors:
Jacques Durand (), Fujitsu Limited
Anish Karmarkar (), Oracle
Adrian Otto (), Rackspace Hosting, Inc.
Additional artifacts:
This prose specification is one component of a Work Product which also includes:
· ??
· Other parts (list titles and/or file names)
Related work:
This specification replaces or supersedes:
· Specifications replaced by this specification (hyperlink, if available)
This specification is related to:
· Related specifications (hyperlink, if available)
Abstract:
This document defines the Test Assertions for V1.1 of the CAMP (…) specification for Platform as a Service (PaaS). Test Assertions are testable or measurable expressions related to normative statements in a specification for evaluating if an implementation (or part of it) conforms to this specification. These Test Assertions are intended to be used for defining precisely the conditions required by a CAMP implementation in order to conform. They support the testing activity by representing a bridge between the normative statements in the specification and the executable test cases that are parts of a conformance test suite.
Status:
This Working Draft (WD) has been produced by one or more TC Members; it has not yet been voted on by the TC or approved as a Committee Draft (Committee Specification Draft or a Committee Note Draft). The OASIS document Approval Process begins officially with a TC vote to approve a WD as a Committee Draft. A TC may approve a Working Draft, revise it, and re-approve it any number of times as a Committee Draft.
Copyright © OASIS Open 2012. All Rights Reserved.
All capitalized terms in the following text have the meanings assigned to them in the OASIS Intellectual Property Rights Policy (the "OASIS IPR Policy"). The full Policy may be found at the OASIS website.
This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published, and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this section are included on all such copies and derivative works. However, this document itself may not be modified in any way, including by removing the copyright notice or references to OASIS, except as needed for the purpose of developing any document or deliverable produced by an OASIS Technical Committee (in which case the rules applicable to copyrights, as set forth in the OASIS IPR Policy, must be followed) or as required to translate it into languages other than English.
The limited permissions granted above are perpetual and will not be revoked by OASIS or its successors or assigns.
This document and the information contained herein is provided on an "AS IS" basis and OASIS DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY OWNERSHIP RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
STATUS:
- WD08
Table of Contents
1 Introduction 5
1.1 Terminology 5
1.2 Normative References 5
1.3 Non-Normative References 5
2 Test Assertions Design and Objectives 6
2.1 Design Principles 6
2.2 Coverage of the Specification 6
2.2.1 Categories of TAs 6
2.2.2 Anatomy of a Test Assertion 7
3 Test Assertions 8
3.1 Test Assertions for Model Serialization and Integrity 8
3.1.1 Basic Schema Compliance for Platform Resources, Provider Side 8
3.1.1.1 Platform Serialization (TA-MOD-1-1) 8
3.1.1.2 PlatformComponentTemplate Serialization (TA-MOD-1-2) 8
3.1.1.3 PlatformComponentCapability Serialization (TA-MOD-1-3) 9
3.1.2 Basic Schema Compliance for Deployed Package Resources 9
3.1.2.1 AssemblyTemplate Serialization (TA-MOD-2-1) 9
3.1.2.2 AssemblyComponentTemplate Serialization (TA-MOD-2-2) 9
3.1.2.3 ApplicationComponentTemplate, ComponentRequirement Serializations (TA-MOD-2-3) 10
3.1.3 Basic Schema Compliance for Application Resources 10
3.1.3.1 Assembly Serializations (TA-MOD-3-1) 11
3.1.3.2 ApplicationComponent Serialization (TA-MOD-3-2) 11
3.1.3.3 ApplicationComponent , PlatformComponent Serializations (TA-MOD-3-3) 11
3.1.4 Basic Schema Compliance for Platform Resources, Consumer Side 12
3.1.4.1 Requests to Platform (TA-MOD-4-n) 12
3.1.4.2 Parameter Use (TA-MOD-4-n) 12
3.1.5 Consistency of Serialization with Metadata 12
3.1.5.1 Required Attribute Enforcement (Provider) (TA-MOD-5-1) 12
3.1.5.2 Non-Mutability Enforcement (Provider) (TA-MOD-5-2) 13
3.1.5.3 Non-Mutability Enforcement (Consumer) (TA-MOD-5-3) 13
3.1.6 Metadata Integrity and Serialization 13
3.1.6.1 Parameters Definitions Serialization and Integrity (TA-MOD-6-1) 13
3.1.6.2 Specification Version value (TA-MOD-6-) 13
3.1.6.3 Specification Version consistency (TA-MOD-6-) 13
3.1.6.4 Implementation Version value (TA-MOD-6-) 13
3.1.6.5 Specification Version consistency (TA-MOD-6-) 13
3.1.6.6 Formats Serialization 13
3.1.6.7 Format Serialization and Referential Integrity 14
3.1.6.8 TypeDefinitions Serialization 14
3.1.6.9 TypeDefinition Serialization and Referential Integrity 14
3.1.6.10 Attribute Definition Serialization and Referential Integrity 14
3.1.6.11 Platform Endpoints 14
3.1.7 Model Integrity 14
3.1.7.1 ACT Not Shared (TA-MOD-7-1) 14
3.2 Protocol Test Assertions 15
3.2.1 Resource Creation and HTTP 15
3.2.1.1 POST to a Platform 15
3.2.1.2 POST to an AssemblyTemplate 15
3.2.1.3 POST to an ApplicationComponentTemplate 15
3.2.2 Resource Queries and HTTP 16
3.2.2.1 SelectAttr Positive Provider side (TA-PRO-2-1) 16
3.2.2.2 SelectAttr Positive Consumer side (TA-PRO-2-2) 16
3.2.2.3 SelectAttr Negative Case (TA-PRO-2-3) 17
3.2.3 Media Types Formats and Headers 18
3.2.4 Query Parameter Mechanism 18
3.2.5 Resource Updates 18
3.3 Resource State Changes and Lifecycle Test Assertions 18
3.3.1 Resource Creation and Consistency of Representations 18
3.3.2 Resource Integrity 19
3.3.2.1 Dependencies Resolution (TA-STA-2-1) 19
3.3.3 Resource Deletion 19
3.3.4 Component Dependencies Resolution 20
3.3.5 Resource Export 20
3.3.6 State Changes of an Application Instance 20
3.3.6.1 Running State of an Application Instance (TA-STA-6-1) 20
3.4 PDP Test Assertions 21
3.4.1 PDP overall Structure (Manifest?) 21
3.4.2 Deployment Plan Integrity 21
4 Mapping CAMP Conformance Profiles to Test Assertions 22
4.1.1 section title 22
4.1.1.1 section title is usually deepest for Table of Contents 22
5 # Conformance 23
Appendix A. Acknowledgments 24
Appendix B. Non-Normative Text 25
B.1 Subsidiary section 25
B.1.1 Sub-subsidiary section 25
Appendix C. Revision History 26
camp-ta-v1.1-wd01 Working Draft 01 03 December 2012
Standards Track Draft Copyright © OASIS Open 2012. All Rights Reserved. Page 6 of 26
1 Introduction
[All text is normative unless otherwise labeled]
1.1 Terminology
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119].
1.2 Normative References
[RFC2119] Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels”, BCP 14, RFC 2119, March 1997. http://www.ietf.org/rfc/rfc2119.txt.
Reference] ion]
1.3 Non-Normative References
Reference] ion]
NOTE: The proper format for citation of technical work produced by an OASIS TC (whether Standards Track or Non-Standards Track) is:
[Citation Label]
Work Product title (italicized). Approval date (DD Month YYYY). OASIS Stage Identifier and Revision Number (e.g., OASIS Committee Specification Draft 01). Principal URI (version-specific URI, e.g., with filename component: somespec-v1.0-csd01.html).
For example:
[OpenDoc-1.2] Open Document Format for Office Applications (OpenDocument) Version 1.2. 19 January 2011. OASIS Committee Specification Draft 07. http://docs.oasis-open.org/office/v1.2/csd07/OpenDocument-v1.2-csd07.html.
[CAP-1.2] Common Alerting Protocol Version 1.2. 01 July 2010. OASIS Standard. http://docs.oasis-open.org/emergency/cap/v1.2/CAP-v1.2-os.html.
2 Test Assertions Design and Objectives
2.1 Design Principles
- explain the general design of a test assertion, its structure.
- Suggest to use the TAG methodology (used in SCA and WS-I). See: http://docs.oasis-open.org/tag/model/v1.0/os/testassertionsmodel-1.0-os.pdf and the related primer: https://www.oasis-open.org/committees/download.php/44696/taguidelines-v1.0-wd02.pdf .
- Test environment assumptions: Describe the approach used: e.g. the test design assumes the CAMP Provider to be a “black box” that is only evaluated via message exchanges. So the actual test targets will be messages or sequences of messages (or separate artifacts? E.g. PDP packages? )
- Conformance targets: Behind these concrete test targets, the overall conformance target is primarily the CAMP Provider, and also the Consumer to a lesser extent (i.e. a Consumer may fail (?) to conform to CAMP when communicating with a Provider).
- An example(s) of TA.
2.2 Coverage of the Specification
- explain the intended coverage of the specification (MUST / SHOULD / MAY)
- categories of CAMP TAs, if any.
- What part of the specification, or what kind of statements are NOT covered so far.
2.2.1 Categories of TAs
Four general categories of test assertions can be distinguished:
· Model Serialization and Integrity TAs: addressing basic serialization of resources
o Provider side: Responses to simple requests (GET, no advanced queries). Metadata rules are taken into account. Integrity constraints are also addressed, including referential integrity.
o Consumer side: Serialization of resources passed by value.
· Protocol TAs: addressing protocol aspects, from HTTP use to query syntax and semantics.
o Provider side: HTTP code. Responses to advanced queries. Verifying that receiver understands the quesry semantics
o Consumer side: Verifies that the requests use well-formed requests (HTTP, parameters) and queries.
· Resource State Changes and Lifecycle Test Assertions TAs:
o Provider side: Verifying the Resource state and content has changed according to operations.
o Consumer side: Verifies that the requests use well-formed operations.
· PDP TAs.: (in case the PDP as a separate resource has its own requirements)
2.2.2 Anatomy of a Test Assertion
Example:
TA_Id: CAMP-TA-NNN
NormativeSource: “(section: §5.8)”
Target: A response message to a GET ApplicationComponentTemplate message.
Predicate: the target message content satisfies the JSON schema for ApplicationComponentTemplate resources.
PrescriptionLevel: mandatory
Tag: conformance=Provider
Notes:
1. the “NormativeSource” element is here simply referring to the section(s) in the specification that contain(s) the normative statements addressed by the test assertion”. (Sometimes it does more than make a reference, i.e. makes some “interpretation” of the original statement(s) that can be narrower, and that reflects more concretely the actual test expression).
2. The “Target” or test target is expressed in terms of observable/testable item: any form of implementation or part of it (artifact, processor, document, …) here a particular message pattern (e.g. as expected to be captured in a log) that would “trigger” the verification.
3. The “Predicate” contains the actual conformance logic. It is worded as a fact (not as a rule or requirement – no MUST/SHOULD) - It should be evaluated each time a sequence of message appears in the log that satisfies the message pattern described in the Target. Predicate=false means the implementation failed the normative requirement.
4. The PrescriptionLevel: indicates the strength of the normative statement (shall/should/may, or equivalent keywords). It can take values: mandatory/preferred/permitted, matching known keywords.
5. Test assertions may have “tags”, as accessory information. There is a tag here named conformance with value “Provider”. This means that the actual entity under evaluation for conformance by this assertion is the Provider (as opposed to the Consumer): the failure of the verification (i.e. predicate value=”false”) means a failure of the Provider to conform.
Instead of repeating the above test assertion for each resource type, one can define a generic test assertion, by defining a “variable” that can iterate over a set of values, see below.
3 Test Assertions
3.1 Test Assertions for Model Serialization and Integrity
Objectives:
- Verify that all serializations are well-formed, both in requests and responses.
- Includes consistency with metadata semantics (e.g. which attribute should be returned)
- Only covers cases of simple queries (no q parameters à protocol TAs)
3.1.1 Basic Schema Compliance for Platform Resources, Provider Side
NOTES:
- the TAs only require Platform accessibility – no PDP need be deployed.
- verify the serialization of responses to GET , as well as referential integrity from Platform.
3.1.1.1 Platform Serialization (TA-MOD-1-1)
TA_Id: TA-MOD-1-1
NormativeSource: “(section: §5.7 of WD08)”
Target: A response message to a GET Platform message.
Predicate: the target message content satisfies the JSON “schema” for Platform, and has a type attribute = “platform”.
PrescriptionLevel: mandatory
Tag: conformance=Provider
3.1.1.2 PlatformComponentTemplate Serialization (TA-MOD-1-2)
TA_Id: TA-MOD-1-2
NormativeSource: “(section: §5.12 of WD08)”
Target: A response message to a GET PlatformComponentTemplate message.
Prerequisite: the GET URI exists in Platform.platformComponentTemplates[] as result to a previous GET Platform URI.
Predicate: the target (response) message content satisfies the JSON schema for PlatformComponentTemplate , and has a type attribute = “platformComponentTemplate”.
PrescriptionLevel: mandatory
Tag: conformance=Provider
(if fail: either the resource serialization is wrong, or the resource does not exist while having a reference in the Platform)
3.1.1.3 PlatformComponentCapability Serialization (TA-MOD-1-3)
TA_Id: TA-MOD-1-3
NormativeSource: “(section: §5.14 of WD08)”
Target: A response message to a GET PlatformComponentCapability message.
Prerequisite: the GET URI exists in Platform.platformComponentCapabilities[] as result to a previous GET Platform URI.
Predicate: the target (response) message content satisfies the JSON schema for PlatformComponentCapability, and has a type attribute = “platformComponentCapability”.
PrescriptionLevel: mandatory
Tag: conformance=Provider
(if fail: either the resource serialization is wrong, or the resource does not exist while having a reference in the Platform)
3.1.2 Basic Schema Compliance for Deployed Package Resources
NOTES:
- the TAs apply when a PDP has been deployed.
- verify the serialization of responses to GET
3.1.2.1 AssemblyTemplate Serialization (TA-MOD-2-1)
TA_Id: TA-MOD-2-1
NormativeSource: “(section: §5.8 in WD08)”
Target: A response message to a GET AssemblyTemplate message.
Prerequisite: the GET URI exists in Platform.assemblyTemplates[] as result to a previous GET Platform URI.
Predicate: the target message content satisfies the JSON schema for AssemblyTemplate, and has a type attribute = “assemblyTemplate”.
PrescriptionLevel: mandatory
Tag: conformance=Provider
(if fail: either the resource serialization is wrong, or the resource does not exist while having a reference in the Platform)
3.1.2.2 AssemblyComponentTemplate Serialization (TA-MOD-2-2)
TA_Id: TA-MOD-2-2