Automatic Test Markup Language (ATML)

Automatic Test Markup Language (ATML) Summary

An overview of the IEEE ATML activity, from the July 2007 ATML Status Report:

ATML's mission is to define a collection of XML-based schemas that allows Automatic Test Equipment (ATE) and test information to be exchanged in a common format adhering to the XML standard. ATML defines a framework through which different architectures using XML can be implemented. It:

• defines the components from which users can build their architectures, whilst being interoperable with other compliant architectures
• shows examples of the net centric services by which this information can be exchanged across different Automatic Test System (ATS) platforms as part of a maintenance process
• defines the XML format that these elements

The ATML specifications will define: (1) how to define XML schemas that represent ATE and test information, and (2) a set of XML schemas supporting the exchange of specific ATE and test information. The ATML specifications will support:

• services that can be used for exchanging ATE and test information in a distributed net-centric environment
• services supporting the exchange of specific ATE and test information in specific common areas

The IEEE Standards Coordinating Committee 20 on Test and Diagnosis for Electronic Systems (SCC20) is the standards organisation through which the ATML components (i.e., schemas and documentation) will be published under various IEEE standards. The ATML organisation is an open, independent focus group contributed to by the ATE industry and government agencies to advance the common exchange of test information through the use of XML. The ATML group provides draft schemas and associated documents, examples, use cases, requirements and conducts trial use of any ATML components. Their findings are submitted to the various SCC20 subcommittees and working groups to advance the IEEE standards associated with the ATML components..."

Automatic Test Markup Language (ATML) Project Description

ATML Project Information:

Scope: ATML defines a standard exchange medium for sharing information between components of automatic test systems. This information includes test data, resource data, diagnostic data, and historic data. The exchange medium is defined using the extensible markup language (XML). This document specifies the architecture for the family of ATML standards.

Purpose: The purpose of ATML is to support test program, test asset and Unit Under Test (UUT) interoperablility within an automatic test environment. ATML accomplishes this through a standard medium for exchanging UUT, test and diagnostic information between components of the test system. The purpose of this document is to provide an overview of ATML goals as well as to provide guidance for usage of the ATML family of standards.

Justification: The ATML initiative is driven by a desire to standardize the XML format for use by various proprietary tools used within the automatic test industry. This will benefit both automatic (computer controlled) test equipment manufactures, maintainers and test system users in a broad range of industries including aerospace and government/military. By using a common format different tools and systems can exchange information, and form co-operative heterogeneous systems resulting in:

• Decreased test times
• Reduced incidents of Can Not Duplicate or No Fault Found
• Reduced Repair Cycle
• Formalized capture of historic data
• Improved Closed loop diagnostic systems

ATML defines a framework through which different architectures using XML can be implemented. It:

• Defines the components from which users can build their architectures, whilst being interoperable with other compliant architectures
• Shows examples of the net centric services by which this information can be exchanged across different ATS platforms as part of a maintenance process
• Defines the XML format that these elements will take

The ATML specifications will define:

• How to define XML schemas that represent ATE and test information
• A set of XML schemas supporting the exchange of specific ATE and test information
• Example services that can be used for exchanging ATE and test information in a distributed net-centric environment
• A set of services supporting the exchange of specific ATE and test information in specific common areas

Committee SCC20 and Relevant Subcommittees

Within IEEE SCC20 (IEEE Standards Coordinating Committee 20 on Test and Diagnosis for Electronic Systems), the Test Information Integration (TII) Subcommittee and Test and ATS Description Subcommittee are responsible for most of the work in the P1671 Project. See the IEEE Approval of P1671 (Standard Automatic Test Markup Language (ATML) for Exchanging Automatic Test Equipment and Test Information via XML). On 08-December-2004, the IEEE-SA Standards Board approved this project through 31-December-2008. References for SCC20 and the two subcommittees follow. Page 5 ("SCC20 Organisation for 2007") of the document ATML StatusIssue 11 supplies an organizational chart for SCC20 as of July 2007.

• IEEE Standards Coordinating Committee 20 on Test and Diagnosis for Electronic Systems (SCC20)
• IEEE SCC20 Test Information Integration (TII) Subcommittee. IEEE 1671 Overview and Architecture, with IEEE P1671.3 - IEEE P1671.6 (4x).
• IEEE SCC20 Test and ATS Description Subcommittee. IEEE P1671.1 and P1671.2.

Automatic Test Markup Language (ATML) Project General References

• ATML Home Page
• ATML Project Information
• ATML StatusIssue 11 (July 2007). Overview of the ATML activity in the ATML focus group and as part of the IEEE SCC20 subcommittees. [source .ppt]
• ATML Document Archive
• ATML Project document log
• ATML Project public discussions
• ATML Meeting Minutes
• ATML Members
• ATML Working Groups
• ATML Focus Group
• ATML Milestones and Meeting Schedule
• Complete ATML schedule and timescales

Automatic Test Markup Language (ATML) Overview Documents

• IEEE P1671 /D2. "Draft Trial-Use Standard for Automatic Test Markup Language (ATML) for Exchanging Automatic Test Equipment and Test Information via XML". ATML Introduction (non-normative). December 2005. 43 pages. Prepared by members of the Test Information Integration (TII) Subcommittee of the IEEE Standards Coordinating Committee 20 on Test and Diagnosis for Electronic Systems Copyright © 2005 by the Institute of Electrical and Electronics Engineers, Inc. "This introduction is not part of IEEE P1671 /D2, Draft Trial-Use Standard for Automatic Test Markup Language (ATML) for Exchanging Automatic Test Equipment and Test Information via XML)." See the .doc canonical source format.

  "The family of Automatic Test Markup Language (ATML) standards is being developed under the guidance of the Test Information Integration (TII) subcommittee of the IEEE Standards Coordinating Committee 20 (SCC20) to serve as standards for product test. The ATML family of standards specifies a comprehensive environment for integrating design data, test strategies and requirements, test procedures, test results management, and test system implementations. The family of ATML standards includes reference to IEEE Std. 1232 (AI-ESTATE), IEEE P1636 (SIMICA) and IEEE Std. 1641 (STD). These referenced IEEE standards are therefore part of the ATML family.

  Scope: ATML defines a standard exchange medium for sharing information between components of automatic test systems. This information includes test data, resource data, diagnostic data, and historic data. The exchange medium is defined using the Extensible Markup Language (XML). This document specifies the framework for the family of ATML standards.

  Purpose: The purpose of ATML is to support test program, test asset, and Unit Under Test (UUT) interoperability within an automatic test environment. ATML accomplishes this through a standard medium for exchanging UUT, test and diagnostic information between components of the test system. The purpose of this document is to provide an overview of ATML goals as well as to provide guidance for usage of the ATML family of standards..."

• ATML Framework Overview and Architecture.IEEE Std. 1671-2006. December 15, 2006. IEEE Trial-Use Standard for Automatic Test Markup Language (ATML) for Exchanging Automatic Test Equipment and Test Information via XML. Technical Contact: Chris C Gorringe (Phone:+44 1202 872800). (ISBN: 0-7381-5245-5, 2006; E-ISBN: 0-7381-5246-3; DOI: 10.1109/IEEESTD.2006.261413). History: PAR APP: Dec 08, 2004, BD APP: Sep 15, 2006. This later publication (available apparently only through purchase from IEEE) is part of the published Trial-Use Standard, and provides a later version of the ATML overview.

  Availability: "For non-technical questions, including pricing, availability and ordering, please contact IEEE Customer Service at 1-800-678-IEEE (in the U.S.and Canada); or 1-732-981-0060 (outside the U.S. and Canada); or send a detailed email message to customer-service@ieee.org. To purchase this standard, or for pricing and availability go to http://shop.ieee.org/ieeestore and type in the standard number."

  "ATML defines a standard exchange medium for sharing information between components of automatic test systems. This information includes test data, resource data, diagnostic data, and historic data. The exchange medium is defined using the Extensible Markup Language (XML). This document specifies the framework for the family of ATML standards. Project Purpose: The purpose of ATML is to support test program, test asset, and Unit Under Test (UUT) interoperability within an automatic test environment. ATML accomplishes this through a standard medium for exchanging UUT, test, and diagnostic information between components of the test system. The purpose of this document is to provide an overview of ATML goals as well as to provide guidance for usage of the ATML family of standards. Abstract: This trial-use standard specifies the framework for the family of ATML standards. ATML defines a standard exchange medium for sharing information between components of an Automatic Test System (ATS), using the Extensible Markup Language (XML).

• Automatic Test Markup Language <ATML/>. September 28, 2004. 16 pages. Issue 0.11. This document is an early version of the ATML overview. From the ATML web site Management Area. See the .doc canonical source format.

  "ATML is the standardization of test information to allow test program and test asset interoperability, and UUT test data including results and diagnostics to be interchanged between heterogeneous systems. [The goal is] to define a collection of XML schemas that allows ATE and test information to be exchanged in a common format adhering to the XML standard... External Interfaces represent the information that exchanges between two distinct subsystems. For ATML these subsystems are defined as: TestResults, Diagnostics, TestDescription, Instrument, TestConfiguration, UUTData, TestStation, and InterfaceAdapter. This information shall be representable in XML and defined through the use of XML Schema Document (XSD) conforming to the XMLSchema specification..."

Automatic Test Markup Language (ATML) Schemas

• ATML Download Site. Candidate Schemas (all 1671 Schemas). 14-August-2007 or later.

• Description of schema files from the README: The schemas published at this location are copies associated with the ATML trial-use standard at the time it was published. As such, updated schemas may be available more frequently than the standard. All updated schemas will have a new XML namespace and version to avoid invalidation of any XML instance documents. For a list of current development ATML schemas with their version, see [/ATML/]. For the complete archive of ATML schemas by version, see [Archive]

• Snapshots of example schemas from "current schemas" as of 2007-11:

  • Capabilities See also the Capabilities Working Group
  • Common. See also Common and Miscellaneous
  • HardwareCommon. See also Common and Miscellaneous
  • TestDescription. See also the Working Group
  • InstrumentDescription. See also the Working Group.
  • UUTDescription. See also the Working Group and UUT Data.
  • TestConfiguration. See also the Working Group.
  • TestEquipment. See also the Test Adaptor and Test Station, with TestAdapter directory.
  • TestResults. See also the Working Group
  • STDBSC
  • STDTSFLib
  • STDTSF

Current Project Authorization Request (PAR) Documents of SCC-20

Active PARs for P1671:

• IEEE-P1671: ATML Overview. "Standard Automatic Test Markup Language (ATML) for Exchanging Automatic Test Equipment and Test Information via XML." Life Cycle: Trial-Use.
• SASB/SCC20. P1671.1. Test Descriptions (Ion Neag). "Trial-Use Standard Automatic Test Markup Language (ATML) for Exchanging Automatic Test Equipment and Test Information via XML: Exchanging Test Descriptions."
• SASB/SCC20. P1671.2. Instrument Descriptions (Teresa Lopes). "Trial-Use Standard Automatic Test Markup Language (ATML) for Exchanging Automatic Test Equipment and Test Information via XML: Exchanging Instrument Descriptions."
• SASB/SCC20. P1671.3. Exchanging UUT Description Information (John Ralph). "Standard Automatic Test Markup Language (ATML) for Exchanging Automatic Test Information via XML (Extensible Markup Language): Exchanging UUT (Unit-Under-Test) Description Information."
• SASB/SCC20. P1671.4. Exchanging Test Configuration Data (Tim Davis). "Standard for Automatic Test Markup Language (ATML) for Exchanging Automatic Test Information via XML: Exchanging Test Configuration Information."
• SASB/SCC20. P1671.5. Exchanging Test Adapter Information (Tamara Einspanjer and Ron Taylor). "Standard for Automatic Test Markup Language (ATML) for Exchanging Automatic Test Information via XML: Exchanging Test Adapter Information." Test Station and Test Adapter both share the 'Test Equipment' XML Schema; the Test Equipment Schema is documented in the Test Station Standard. The Test Station Standard is therefore a normative reference to the Test Adapter Standard.
• SASB/SCC20. P1671.6. Exchanging Test Station Information (Tamara Einspanjer and Ron Taylor). "Standard for Automatic Test Markup Language (ATML) for Exchanging Automatic Test Information via XML: Exchanging Test Station Information."

Commentary on Automatic Test Markup Language (ATML)

• ATML: The Standard for Interfacing Test System Components Using XML. Tutorial from National Instruments Corporation. NI Developer Zone. May 11, 2007. "A new XML-based standard for ATE and test information data exchange, known as Automatic Test Markup Language (ATML), is emerging with widespread support among test and measurement industry leaders and major government programs alike. Led by the Naval Air Systems Command and ATE industry members, ATML is a cooperative effort to define a collection of XML schemas to represent test information, such as test programs, test asset interoperability, and unit under test (UUT) test data (including test results and diagnostics procedures)."

• "Implementing ATML in Distributed ATS for SG-III Prototype." By Chen Ming (et al). Plasma Science and Technology Volume 9, Number 2 (April 2007), 227-230. "With the forthcoming large-scale scientific experimental systems, we are looking for ways to construct an open, distributed architecture within the new and the existing automatic test systems. The new standard of Automatic Test Markup Language meets our demand for data exchange for this architecture through defining the test routines and resultant data in the XML format. This paper introduces the concept of ATML(Automatic Test Markup Language) and related standards, and the significance of these new standards for a distributed automatic test system. It also describes the implementation of ATML through the integration of this technology among the existing and new test systems."

• "ATML: A New Standard for ATE." By Ron Harrison (National Instruments). From EE-Evaluation Engineering (March 2005). "For many years, leading electronics manufacturers have searched for standards to improve-the way they share test-system and test-result information with the rest of the enterprise. Despite initial industry efforts to regulate test-system and execution communications through standards such as IPC 25xx, many test organizations already have or are considering developing their own proprietary XML-based data exchange standards to meet specific needs for sharing ATE and test information. However, this no longer may be necessary. A new XML-based standard for ATE and test information data exchange, known as Automated Test Markup Language (ATML), is emerging with widespread support among test and measurement industry leaders as well as major government programs..."

• "ATML in Manufacturing Test Systems Using National Instruments TestStand." By David Rosenthal (Instrumentation Engineering). "Automatic Test Markup Language (ATML) is a means to communicate test outcomes up and down the maintenance chain. The ATML family of standards was developed in the belief that a useful exchange of test information among product life-cycle phases would improve the inefficiencies of testing in both time and budget. In a NetCentric Environment, the promise of increased compatibility leverages usability, support and maintenance to improve the future productivity of Automatic Test System (ATS) investments."

• "Test Management Software." Product Announcement. From The Engineer Online (November 16, 2005). "... NI TestStand 3.5 also features an Automated Test Markup Language (ATML) reporting interface and integration. ATML is a cooperative industry effort to define a collection of XML schemas to represent test information, such as test programs, test asset interoperability and unit under test (UUT) test data, which includes test results and diagnostics procedures..."

• "IEEE Sets in Motion Dual ATML Test Standards." By Greg Reed. From Test & Measurement World (November 01, 2005). "Test engineers working on automotive and aerospace applications can expect a couple of new tools to arrive soon. The IEEE is developing two standards — IEEE P1671.1 and IEEE P1671.2 — that cover the use of automatic test markup language (ATML) for exchanging descriptions via XML technology on both the test performed and the instruments evaluated. Work on the standards is already well underway; final internal ballots are ready for 2006, and delivery is expected in 2007.

• Documents from IEEE. The online abstracts for the IEEE P1671 specifications and technical reports are freely available from the IEEE Xplore database [Results for "(atml) IN metadata" — your search matched 43 of 1692897 documents]; complete documents are available for a fee. Examples:

  • "ATML as a framework for reducing development time and maintenance of next generation test systems." By Santiago Delgado and Anand Jain. September 2007. From Autotestcon 2007.
  • "ATML: What 'it' is, what 'it' is not, and an example of how 'it' can be applied." By M. Seavey and T. Einspanjer. Paper presented at Autotestcon 2005.
  • "Implementing ATML into an existing software architecture." By S.J. O'Donnell and P. Richardson (Lockheed Martin STS, Orlando, FL, USA). Paper presented at Autotestcon 2004.

TaMIE Technical Committee Proposal Comment

Proposal Section 2a. Similar Work: Identification of similar or applicable work that is being done in other OASIS TCs or by other organizations... "ATML has been designed for exchange of diagnostic information, configuration data, and test instrument definitions between conforming software applications. It was not designed to support test scripting and execution. However, ATML representations could be complementary to executable test case scripts."

---

Event-Driven Test Scripting Language (eTSL)

eTSL Overview

Event-Driven Test Scripting Language. Working Draft 0.85. 7-November-2007. Edited by Jacques Durand (Fujitsu Computer Systems). Contributors: Hyunbo Cho (KorBIT), Michael Kass (NIST), Jungyub Woo (Samsung Electronic), Monica Martin (Sun), Serm Kulvataniou (NIST), Jaewook Kim (NIST), Pete Wenzel (Sun). Produced by members of the OASIS ebXML Implementation Interoperability and Conformance (IIC) Technical Committee. See the document reference page and soruce PDF.

The Event-driven Test Scripting Language (eTSL) is a model and language for eBusiness / eGovernment test suites, with a particular focus on communication events, versatile usage for the QA testing phase as well as the monitoring of deployed systems, and extensible design to leverage specialized validation processors as well as XML tools such as Xpath, XSLT, and Xquery. It builds on previous experiences in conformance and interoperability testing in ebXML, WS-I as well as on learning from eBusiness proof of concepts and testing environments done within the context of the OAGI-NIST testbed and the KorBIT testbed.

Test cases for e-Business / e-Government will be used to verify the conformance of a partner endpoint to the various standards and conventions that communicating partners have agreed on. These can be of different nature:

• Messaging infrastructure standards, ranging from transport level — e.g. HTTP, SMTP — to higher-level messaging protocols and quality of service (reliability, security), such as those defined as SOAP extensions.

• Message choreographies. These include business choreographies (e.g. Conforming to UMM business transaction patterns) as well as lower level exchange patterns that are often necessary to establish some quality of service requirements and related context (reliability, trust, security).

• Business document standards, industry-specific such as XML schemas, or horizontal (e.g. Modeling practice). These include syntactic as well as semantic conformance. Also "metadata documents" such as agreements, are included in this category.

There have been conformance and interoperability testing tools and initiatives for each one of these categories, but there is currently no support for testing an integration of the above. WS-I profiles address mainly the integration of standards within the messaging infrastructure layer, and remain horizontal in nature (industry-independent).

In practice, a deployed e-Business system will profile such an integration, involving all above layers in addition to industry-specific profiles. Relying on testing each layer independently will not be sufficient to test this integration. Testing for the conformance (or interoperability) of the sum is more than the sum of testing for the conformance of its parts.

For example, a user community may decide that a particular messaging QoS (reliability and security levels) should be used for specific business transactions but not for others. Such requirements assume an ability to generate and verify test cases that mix low-level message controls and business-level transactions aspects (e.g. involving the use of specific business documents in payloads.) Or, some business transaction patterns must bind to lower-level protocols in a very specific way, e.g. regarding the use of the HTTP back-channel, depending on how soon a business response is expected, which is a business attribute...

The objective of this document is to define a model for such integrated testing, and a scripting representation for the related test cases. It builds on previous experiences in conformance and interoperability testing in ebXML, WS-I, as well as on learning from past e-Business proof of concepts for business collaboration done with OAGI material, in particular within the context of the OAG-NIST testbed and semantic document validation....

TaMIE Technical Committee Proposal Comment

1c. Scope: ... deserves prime attention from the TC assuming their IP status is compatible with the IPR mode of the TC: The Event-Driven Test Scripting Language (eTSL) Version 0.85..." See also TaMIE TC proposal section "2g. List of Contributions: The following work, already submitted in October 2006 to the OASIS ebXML Implementation, Interoperability and Conformance TC, is also contributed to this TC: eTSL draft V0.85, originally derived from the TestFramework 1.1 produced OASIS IIC, 2004-2005."

---

JXUnit: General Software Test Environments and Scripting

JXUnit Overview

Summary from the article by Ervin Varga:

"JUnit is a framework for writing unit tests. Numerous extensions are available for different purposes; one of them is a JXUnit framework. JXUnit is a directory-driven test scripting system. A complete test suite is basically comprised from various test suites/cases located in different parts of the whole folder tree in question. Essentially, the terms test suite and test case are only conceptual in JXUnit, since, no explicit differentiation exists on the framework level between them. Nevertheless, in the rest of this paper we will use a term test suite for a test case holding other tests. A folder where they are situated names all test cases.

Using a folder tree for specifying a test suite has a clear advantage over a hard-coded variant, in sense, new suites can be easily constructed by just reorganizing folders (no code changes, no recompilations). Thus, JXUnit effectively utilizes the file system for this particular purpose. Furthermore, as JXUnit builds upon JUnit, tests can be run using the standard TestRunners (textual, AWT or SWING) found in the JUnit framework.

Nonetheless, the Quick framework is the cornerstone of JXUnit's powerfulness and flexibility. Quick is a Java based XML data binding framework, more precisely, a framework for advanced data transformations and mappings. Quick uses QJML for expressing a binding schema. This schema serves as a bridge between XML and Java object representations. By using Quick, one can easily create a Java object from data encapsulated inside an XML document, and vice versa, without ever touching the source code of the class the target Java object belongs to...

JXUnit and JUnit References

• JXUnit: Building Suites of Test Data with XML. Building Suites of Test Data with XML. Release 2.0.0. 18-April-2001.
• jxu QJML Schema
• "JXUnit Primer Part 1: Overview of the JXUnit Framework." By Ervin Varga.
• "JXUnit Primer Part 2: Extending the JXUnit Framework." By Ervin Varga.
• JXUnit Primer Part 3: Reworking the JXUnit Framework." By Ervin Varga.
• JXUnit Forum
• JXUnit Home Page
• JUnit - SourceForge
• JUnit.org

TaMIE Technical Committee Proposal Comment

Proposal Section 2a. Similar Work: Identification of similar or applicable work that is being done in other OASIS TCs or by other organizations... "JXUnit is an XML-based test-scripting system built on top of the JUnit Java classes. It is a data-driven testing system in which input to the system under test and expected output are specified and then the actual output is evaluated. There is no built-in support for B2B testing, in particular the communication and the event-driven tests. However, as a general, test-scripting platform that relies on a common programming language, JXUnit and JUnit could be used as an implementation platform for essentially any test framework including the one proposed here."

---

OASIS Conformance Testing and Certification Framework

White Paper: Conformance Testing and Certification Framework. By Lynne Rosenthal, Mark Skall [WWW], and Lisa Carnahan. Version 1.1. June 25, 2001. 21 pages. Submitted by William Gebert on January 23, 2002. [source PDF]

This work was produced by members of the OASIS Conformance TC which was chartered to "define and develop conformance requirements, guidelines, white papers, and methodologies to enable the development of testable specifications and conforming implementations."

Conformance Testing and Certification Framework Overview

The document "presents general concepts, components, and issues related to establishing a conformance testing program. Although originally submitted to ebXML to stimulate discussion on testing and certification and to serve as a basis upon which ebXML test programs could be developed, it is not exclusive to ebXML. This document represents a work that has been submitted to ebXML as a white paper with its purpose to stimulate discussion and possibly serve as a base document upon which ebXML testing programs could be based...

The objective of this white paper is to present an overview of a conformance and certification testing infrastructure that could be adapted for ebXML and would provide implementations of ebXML specifications the ability to verify that they comply with the specifications. Conforming implementations are a necessary prerequisite for achieving interoperability among implementations. To achieve this objective, we will describe the components necessary to enable conformance testing as well as the activities, roles and responsibilities of the organizations needed to conduct a conformance testing and certification program.

The paper presents "general concepts, components, and issues related to establishing and administering a conformance testing and certification program. It will identify the types of activities, responsibilities, services, and documentation required to conduct conformance testing. The paper addresses both formal validation and certification of products, as well as self-testing. Self-testing provides implementers the ability to perform self-tests at their locations as quality assurance checks on their own implementations of the ebXML specifications or in preparation for formal validation. Testing of the ebXML specifications will not be specifically addressed. However, a testing plan will be provideed for testing registry implementations to the ebXML Registry specification...

Background: The ebXML specification, like all standards, is not an end in itself but a means to an end. Standards are worthless if not implemented and meaningless if not implemented in a consistent and correct manner. The goal is to obtain implementations of the standard that correctly perform the functionality specified in the standard. Conformance testing helps to achieve correct implementation. It provides a way to determine whether or not these implementations conform to the standards in question. Determining whether an implementation faithfully implements a specification is essential to creating robust, interoperable solutions. Conformance testing performed by implementers early-on in the development process can help them to find and correct errors before the software reaches the marketplace, and users rely on the software. Moreover, conformance testing provides software developers and users assurance and confidence that the conforming product behaves as expected, performs functions in a known manner, or possesses a prescribed interface or format..."

TaMIE Technical Committee Proposal Comment

Proposal Section 1c. Scope, "... input material to this TC, which will deserve prime attention from the TC assuming their IP status is compatible with the IPR mode of the TC... Conformance testing and Certification Framework (OASIS, Conformance TC, June 2001)."

---

OASIS ebXML IIC Test Framework

IIC Test Framework: Bibliographic Information

ebXML Test Framework. OASIS Committee Specification. Version 1.1. October 11, 2004. Draft. Document identifier: ebxml-iic-test-framework-11. Edited by Michael Kass (NIST) and Jacques Durand (Fujitsu). Contributors: Monica Martin (Sun Microsystems), Jacques Durand (Fujitsu), Christopher Frank (SEEBURGER), Serm Kulvatunyou (NIST), Tim Sakach (Drake Certivo, Inc), Hyunbo Cho (Postech), Han Kim Ngo (NIST), and Pete Wenzel (SeeBeyond). 214 pages. Produced by members of the OASIS ebXML Implementation, Interoperability and Conformance Technical Committee, and available from the TC document repository. See also the .doc format, extracted from the ZIP package [source ZIP].

The document was submitted by Mr Michael Kass on Friday, January 07, 2005 as IIC ebXML Test Framework V1.1 Technical Committee Specification. Description: "Version 1.1 of the OASIS IIC ebXML Test Framework Specification, incorporating new features that enable more complex scripting, targeted at performance and interoperability testing of business process driven ebXML web services.

IIC Test Framework: Overview

Abstract: "This document specifies ebXML interoperability testing specification for the eBusiness community."

Summary: This document describes a test framework for automatically running test suites for — but not limited to — ebXML specifications. The framework includes an architecture design based on components that can be combined and distributed in different ways, to accommodate different test harnesses. It also includes an extensible test scripting language for coding test suites in an executable way. It can accommodate third-party plug-ins, that would perform advanced verifications for example on message material (e.g. semantic verification using rule engine), or that would help build testing material (e.g. digital signature). This specification is organized around the following topics: (1) Test Framework Components; (2) Test Suite Representation using XML; (3) Test Execution using Workflow Principles; (4) Test Scenarios and Test Framework Profiles; (5) Sample Instance files of Test Material; (6) Normative Schemas for Test Material.

The target audience for this specification is: (1) The community of software developers who implement and/or deploy the ebXML Messaging Service (ebMS) or use other ebXML technologies such a s Registry/Repository (RegRep), Collaboration Profile Protocol/Agreement (CPPA) or Business Process Specification Schema (BPSS) and (2) The testing or verification authority, which will implement and deploy conformance or interoperability testing for ebXML implementations.

Objectives: The OASIS IIC ebXML Test Framework is intended to support conformance and interoperability testing for ebXML (as well as other eBusiness) specifications. It describes a testbed architecture and its software components, how these can be combined to create a test harness for various types of testing. It also describes the test material to be processed by this architecture, a mark-up language and format for representing test requirements, and test suites (a set of Test Cases). The Test Framework described here has been designed to achieve the following objectives...

The Test Framework is a foundation for testing all ebXML architectural components such as Messaging, Registry, BPSS, CPA, and Core Components . Although designed to support testing implementations of current and future ebXML specifications , the Test Framework is flexible enough to permit testing beyond ebXML message format, to include any message envelope and payload testing of XML-based e-Business messaging services. Additionally, the IIC Test Framework can be employed for XML-based A2A ( Application to Application) conformance and interoperability testing.

Regardless of the type of testing that is employed however, all testing MUST follow the same procedural steps, and employ the same XML format as defined by the XML schemas defined in the Appendix of this specification. By following the formalized guidelines in this specification, conformance and interoperability test suites can be used by any IIC Test Framework Specification compliant implementation.

The harnessing of an ebXML implementation (or possibly several implementations, in case of interoperability testing) with the Test Framework requires a moderate effort. It generally requires some interfacing work specific to an implementation, in the case no standard interface (API) has been specified. For example, the Test Service (a component of the Test Framework) defines Actions that will need to be called by a particular MSH implementation for ebXML Messaging Services conformance testing...

Operating the Test Framework — or one of the test harnesses that can be derived from it — in order to execute a test suite, does not require advanced expertise in the framework internals, once the test suites have been designed. The tests should be easy to operate and to repeat with moderate effort or overhead, by users of the ebXML implementation(s) and IT staff responsible for maintaining the B2B infrastructure, without expertise in testing activity. Users can define new Test Suites and Test Cases to be run on the framework. For this, they will script their tests using the proposed test suite definition language or mark-up (XML-based) for Test Cases.

A Test Suite (either for conformance or for interoperability) can be run entirely and validated from one component of the framework: the Test Driver. This means that all test outputs will be generated — and test conditions verified — by one component, even if the test harness involves several (possibly remote)components of the framework. The verification of each Test Case is done by the Test Driver at run-time, as soon as the Test Case execution is completed. The outcome of the verification can be obtained as the Test Suite has completed, and a verification report is generated...

General Methodology: This specification only addresses the technical aspect of ebXML testing, and this section describes the portion of testing methodology that relates directly to the usage of the Test Framework. A more general test program for ebXML, describing a comprehensive methodology oriented toward certification, is promoted by the OASIS Conformance TC and is described in Conformance Testing and Certification Framework (L. Rosenthal, M. Skall, L. Carnahan; April 2001, NIST). When conformance certification is the objective, the ebXML Test Framework should be used in a way that is compliant with a conformance certification model as described in Conformance Testing and Certification Model for Software Specifications (L. Carnahan, L. Rosenthal, M. Skall; ISACC 1998 Conference).

TaMIE Technical Committee Proposal Comment

From Proposal Section 1c. Scope, input material to this TC, which will deserve prime attention from the TC assuming their IP status is compatible with the IPR mode of the TC: OASIS IIC Test Framework 1.1"

See also Section 2a. Similar Work, Identification of similar or applicable work that is being done in other OASIS TCs or by other organizations: "The OASIS ebXML IIC Test Framework (TF v1.1): This framework was designed specifically to test the conformance and interoperability of the software implementing the ebXML Messaging Specification. The framework defines components and harnesses (configurations) necessary to simulate and control the environment for conformance and interoperability tests. A shortcoming of the IIC TF is that it has been designed for ebXML messaging 2.0, and provides weak extensibility options, both for external events and for specialized evaluations. The IIC TF does not support other suites of B2B integration standards such as other SOAP profiles integrating several Web Services standards. It also does not support ebMS 3.0 and electronic documents testing."

---

RosettaNet Self-Test Kit (RNSTK)

RosettaNet Self-Test Kit Overview

The TaMIE TC proposers identified the RosettaNet Self-Test Kit as a technology potentially relevant to the work of the proposed TC, citing a thesis written by Albin Kjellin.

"RosettaNet is a globally supported standards organization that provides unmatched leadership in promoting collaborative commerce. RosettaNet aims to align the business processes of supply chain partners. This goal is achieved by the creation of RosettaNet Partner Interface Processes (PIPs) and it helps to define business processes between trading partners. Each PIP defines how two specific processes, running in two different partners' organizations, will be standardized and interfaced across the entire supply chain. PIP includes all business logic, message flow, and message contents to enable alignment of the two processes. RosettaNet has delineated the scope of supply chain processes for which it will design PIPs. This scope is divided into several clusters, segments, or groups of core business processes and within each segment are individual PIPs. The clusters and segments serve as a mechanism to group all supply chain processes into a manageable framework. When partners are ready to implement PIPs, they should not be bound by the rigidity of this framework; they should select from all segments the subset of PIPs required to address specific business interface scenarios..."

The RosettaNet Self-Test Kit includes self-service tools and utilities that provide users with the capability to test their software solutions for compliance with the RosettaNet Implementation Framework (RNIF) and Partner Interface Processes (PIP) specifications.

Whether you are a solution provider looking to launch your RosettaNet enabled products into the market, or a supply-chain company about to implement a RosettaNet e-Business process with your trading partner; you will need to test your product or solution for:

• Compliance with the RosettaNet Implementation Framework specifications for packaging, security and transport
• Content verification of the payload
• Choreography of Business Messages exchanged for each specific PIP implemented
• Ability to handle exceptions

The RosettaNet Self Test Kit includes self-service tools and utilities to give you full control of this testing process to:

• Configure the Test Scenarios: customize the testing to be performed for the specific PIPs you are implementing through an easy to use browser interface
• Automate the Execution of the Test Sequences
• Present the Test Results: results are gathered in real time during the execution of the test sequence and are available to you in a human-readable as well as machine-readable form

A Study of the RosettaNet Business-To-Business Communication Standard

The RosettaNet Standard and Compliant Platforms. A Study of the RosettaNet Business-To-Business Communication Standard and its Role in Business and Technical Integration Platforms. By Albin Kjellin. Masters Thesis. Lund University Department of Communication Systems, 2005. 51 pages. See the reference to the Masters Thesis work supervised by Per Johan Runeson [thesis, cache version].

Abstract: Modern technology and the Internet in particular have revolutionized the way business is conducted. The need is extensive for collaborations between and amongst business and information systems. Electronically based trading has proven to provide a number of advantages such as increased speed, reliability and efficiency. The term business process has been extended to include collaboration between trading partners apposed to just describing the internal information flow. To be able to integrate organizations IT systems with each other, a global framework for e-business has to be established. There have been many initiatives from global organizations and the latest one is the RosettaNet standard.

The most central conception of the RosettaNet standard is the Partner Interface Process (PIP) and it describes the choreography of a business message exchange between two business partners. The PIPs are in turn controlled by the RosettaNet Implementation Framework (RNIF) that specifies the more grammatical aspects of the business process. The context of a business message is specified in two dictionaries, one technical and one business oriented. The simplest way to describe the RosettaNet standard is to look at it as a language where the dictionaries provides the words, the RNIF the grammar and the PIPs describe the dialog.

The purpose of this project is to examine two RosettaNet compliant platforms, IBM WebSphere Partner Gateway and BEA WebLogic, to determine pros and cons in terms of validation level, complexity and connectivity options. The testing of these two platforms has been conducted using a test client provided by the RosettaNet consortium. The client application is called RosettaNet Self Test Kit (RNSTK) and is used to validate compliance to the RosettaNet standard in terms of structure and syntax. Both WebLogic and WebSphere Integration Connect have been tested using this application. They have also been tested against each other to determine the level of validation. IBM WebSphere Integration Connect is by far the application that has the most complete support for the RosettaNet standard. Messages that pass the validation in both RNSTK and WebLogic generate an exception in WebSphere Integration Connect.

One part of this project was to modify the RNSTK to provide extra functionality. After the modification the RNSTK can receive any type of RosettaNet message and provide a response without needing prior configuration for that particular test scenario...

Self-Test Kit: References

References:

• RosettaNet web site
• Self-Test Kit: 10-Step Demonstration "Take a quick, 10-step tour of the self-test application using a PIP test example. eBusinessReady will provide the current Self-Test Kit version upon registration of software compliance testing."
• Self-Test Kit FAQ Document. This document provides answers questions on use of the Self-Test Kit (STK), how to download test scripts and the scope of the RNIF 2.0 and PIP software compliance tests.
• RosettaNet Developer Tools Download
• Section 4.3 'STK' in the Thesis by Albin Kjellin.
• RosettaNet Store Kit Description
• "RNIF Compliance Test Final Report: WebSphere Partner Gateway Advanced Edition V6.0. "Drummond Group Inc. is pleased to announce that WebSphere Partner Gateway Advanced Edition V6.0 and WebSphere Partner Gateway Enterprise Edition V6.0 have completed all requirements and successfully passed all tests to prove compliance to the RosettaNet RNIF 2.0 specification on the basis of interaction with the RosettaNet Self Test Kit (STK) over the RosettaNet RNIF test suite. To fully understand what completing the test means regarding the use of the product in production, please read this document carefully."
• Tutorial: Building RosettaNet Solutions. From BEA. "The RosettaNet W3C XSD schemas for selected PIPs are included in the RosettaNet Self-Test Kit (STK) which can be downloaded from the Developer Tools area on the RosettaNet Ready Web site... If the schema for the PIP you are using is not available on this web site, it is possible to implement RosettaNet solutions in WebLogic integration using RosettaNet message definitions specified as DTD files. However, it is recommended to use W3C XSD files instead, since many of the WebLogic Integration tools such as the XQuery mapping tools (used to define data transformations) only support XSD schemas..."

TaMIE Technical Committee Proposal Comment

Proposal Section 2a. Similar Work: Identification of similar or applicable work that is being done in other OASIS TCs or by other organizations... "RNSTK is a test system provided by the RosettaNet consortium. The system allows software solutions to perform self-tests for compliance to business collaboration scenarios and the RosettaNet Integration Framework (RNIF) specification. The RNSTK tests an integrated system implementing particular B2B collaboration scenarios, yet is tightly dependent on RNIF messaging. RNSTK test suites are encoded using the RNIF specification itself. Due to these reasons, the RNSTK cannot be used to test other suites of B2B standards including the Document Type Definition semantics. Another weakness is that its architecture only supports conformance test configurations."

---

Testing and Test Control Notation Version 3 (TTCN-3)

TTCN-3 Overview

TTCN-3 "is a product of the ETSI Technical Committee MTS (Methods for Testing and Specification). It includes a programming language that has been used for more than 15 years in standardization as well as industry. TTCN-3 is specifically designed for testing and certification, constantly developed and maintained at ETSI by a team of leading testing experts from industry, research institutes, and academia, It parovides a testing technology that applies to a variety of application domains and types of testing. TTCN-3 has been proven to work in very large and complex industrial tests, e.g., of 3G network elements... TTCN-3 has expanded into new kinds of testingm including software module testing, layer/unit testing, integration testing, and distributed systems testing..."

TTCN-3 (Testing and Test Control Notation) "is an internationally standardized testing language for formally defining test scenarios and their implementation. It is designed purely for testing. Design principles: Use one test technology for different tests: distributed, platform-independent testing, tntegrated graphical test development, -documentation and -analysis; adaptable, open test environment. Use one test technology for both distributed IT and Telco systems. TTCN-3 provides for different kinds of testing: Functional testing, Conformance testing, Scalability testing... It covers a large range within development cycle, from unit to integration testing. Triple C: (1) Configuration: Dynamic concurrent test configurations with test components; (2) Communication: Various communication mechanisms — synchronous and asynchronous; (3) Control: Test case execution and selection mechanisms. It features a well-defined syntax, static and operational semantics, different presentation formats, a module concept, extensibility via attributes, external function, external data. It is harmonized with ASN.1 (ETSI ES 201 873-7 Integration of ASN.1) and XML (Methods for Testing and Specification (MTS). The Testing and Test Control Notation version 3; Part 9: Using XML with TTCN-3 TTCN-3 and the Use of XML)."

TTCN-3 and Web Service Test Frameworks

• Web Service Test Framework with TTCN-3. Master's Thesis. By Stefan Troschütz. ISSN: 1612-6793. Reference: ZFI-BM-2007-14. June 15, 2007. 143 pages. Software Engineering for Distributed Systems Group, Institute for Informatics (Zentrum für Informatik). Georg-August-University Göttingen.

  "Web services are standards-based software systems designed to facilitate interoperable application-to-application integration over a network. The broadening adoption of Web services and especially their use for business purposes or critical applications introduce a growing need for efficient testing approaches that allow assuring the correctness and interoperability of Web services.

  This thesis presents a framework for the testing of Web services with the standardized test specification and implementation language TTCN-3. Foremost, the mapping of a Web service description to a TTCN-3 abstract test suite, which facilitates basic testing of the Web service, is discussed in detail. A time-saving automation of the proposed mapping implemented as a Java console application is introduced afterwards. Finally, the enhancement of TTworkbench Basic, a TTCN-3 test development and execution environment, for Web service testing is presented. The implemented extension enables the execution of a TTCN-3 abstract test suite derived from a Web service description. In addition, it provides dialog-based wizards for using the automation of the mapping from within TTworkbench or defining new, more complex test cases...

  Detailed rules are been presented for the mapping of a Web service description given by means of a WSDL document to a TTCN-3 abstract test suite that allows basic testing of the Web service. In addition to the specification of mapping rules for the individual major WSDL elements, the mapping between TTCN-3 and XML Schema, which forms the default type system of WSDL descriptions, is examined. The discussion of the proposed mapping between WSDL and TTCN-3 is exemplified through the derivation of a TTCN-3 abstract test suite from the WSDL description of an exemplary 'Movie Database' Web service.

  The proposed mapping between WSDL and TTCN-3 can be done manually but it is also well capable of being automated. Because an automated mapping is less error-prone and most notably less time-consuming, the WSDL2TTCN utility was implemented in the context of this thesis. The architecture, two implementation details, the invocation, and the available command line options of the Java console application have been presented. Furthermore, an option has been discussed that allows changing the mapping between WSDL and TTCN-3 in certain points in order to generate a TTCN-3 abstract test suite fully compatible to TTworkbench Basic.

• Automated Testing of XML/SOAP based Web Services. By Ina Schieferdecker (FOKUS, Berlin, Germany) and Bernard Stepien (University of Ottawa, Canada). Abstract: "Web services provide seamless connections from one software application to another over private intranets and the Internet. The major communication protocol used is SOAP being mainly XML over HTTP. The exchanged data follow precise format rules in the form of XML Document Type Definitions or more recently the proposed XML Schemas. Web service testing considers functionality and load aspects to check how a Web service performs for single clients and scales as the number of clients accessing it increases. This paper discusses the automated testing of Web services by use of the Testing and Test Control Notation TTCN-3. A mapping between XML data descriptions to TTCN-3 data is presented to enable the automated derivation of test data. This is the basis for functional and load tests of XML interfaces in TTCN-3. The paper describes the mapping rules and prototypical tools for the development and execution of TTCN-3 tests for XML/SOAP based Web services.

• Testing Service Oriented Architecture Based Web Applications." By Bernard Stepien, Liam Peyton, Pulei Xiong, and Pierre Seguin. Compare TTCN-3 Examples — "The SOAP XML based service TTCN-3 example consist in an abstract test suite that enables to test the responses of a weather forecast service to requests of weather conditions for various cities. It illustrates the TTCN-3 features of typing, data templates definitions, data template parametrization, the use of timers, the mismatch of responses, the re-use of data template definitions..."

TTCN-3 References

• TTCN-3 home page
• TTCN-3 Standard Suite
• Archives of the list 'ttcn3@list.etsi.org'
• The TTCN-3 Language: An Introduction. By Theofanis Vassiliou-Gioles (CEO Testing Technologies, Berlin).
• Testing and Test Control Notation
• TTCN-3 in a Nutshell. By Bernard Stepien
• The Evolution of TTCN
• An XML to TTCN-3 Mapping
• Bernard Stepien Papers. Telecommunications Software Engineering Research Group, Department of Computer Science, University of Ottawa. For example: (1) Formal Testing of Web Content using TTCN-3; (2) An Efficient Formal Testing Approach for Web Service with TTCN-3.

TaMIE Technical Committee Proposal Comment

Proposal Section 2a. Similar Work: Identification of similar or applicable work that is being done in other OASIS TCs or by other organizations... "TTCN-3 (Testing and Test Control Notation Version 3, ETSI, June 2001). TTCN-3 is a powerful, programmatically complete procedural language with specialized constructs for defining test procedures, test verdicts, matching mechanisms for evaluation, timer handling and distributed test components. Due to its original focus on telecommunication systems, it also has the ability to specify encoding information, to communicate both synchronously and asynchronously, and to perform passive monitoring. These capabilities are all essential for testing in message-based B2B integration. Relative to previous test frameworks, TTCN-3 is more powerful and flexible. However, TTCN-3 notation can be difficult for a business or domain expert to use. TTCN-3 has not been designed to delegate some functions to external modules and it is weak when it comes to validating business transaction events and XML electronic documents, which are essential for B2B integration testing."

---

W3C QA Framework: Specification Guidelines

QA Framework: Specification Guidelines Overview

QA Framework: Specification Guidelines. W3C Recommendation. 17-August-2005. This version: Latest version URI: http://www.w3.org/TR/qaframe-spec/. Previous version URI: http://www.w3.org/TR/2005/PR-qaframe-spec-20050629/. Edited by Karl Dubost (W3C), Lynne Rosenthal (NIST), Dominique Hazaël-Massieux (W3C), and Lofton Henderson (CGM Open). Contributions by: Tim Boland (NIST), Jeremy Caroll (HP), Patrick Curran (Sun Microsystems), Dimitris Dimitriadis (Ontologicon), Karl Dubost (W3C), Dominique Hazaël-Massieux (W3C), Lofton Henderson (CGM Open), Björn Höhrmann, Richard T. Kennedy (Boeing), Susan Lesch (W3C), David Marston (IBM Research), Lynne Rosenthal (NIST), Mark Skall (NIST), Andrew Thackrah (Open Group), Olivier Théreaux (W3C).

Summary: "The goal of this document is to help W3C editors write better specifications, by making a specification easier to interpret without ambiguity and clearer as to what is required in order to conform. It focuses on how to define and specify conformance. It also addresses how a specification might allow variation among conforming implementations. The document presents guidelines or requirements, supplemented with good practices, examples and techniques...

This document is a guide for editors of W3C specifications. It provides guidelines for improving conformance-related characteristics. In that respect, this document differs from other W3C process and publication-related documents. It addresses the most basic and critical topics with respect to conformance, including how to convey what is required for an implementation in order to conform and how to allow variation among conforming implementations.

The term specification is used as defined in ISO/IEC Guide 2:2004. Standardization and related activities -- General vocabulary as meaning a document that prescribes requirements to be fulfilled by a product, process or service. Specifications can be defined in one document or as a coherent set of several documents, and can import requirements of other specifications with normative references...

W3C Quality Assurance Activity References

• QA Specification Guidelines: Implementation Conformance Statement. "This document is an appendix to QA Framework: Specification Guidelines. It provides a tabular checklist of all requirements and good practices from the specification guidelines, sorted by topics."
• Variability in Specifications. W3C Working Group Note. 31-August-2005. "A companion document of QA Specification Guidelines which analyzes how design decisions of a specification's conformance model may affect its implementability and the interoperability of its implementations."
• W3C QA Matrix. The specifications in the Matrix are at least at Last Call stage, except if they are working on a Test Suite at Working Draft stage.
• QA Framework Primer
• The QA Handbook
• Test Assertion Guide. Editor's Draft. 28-April-2006.
• Conformance Testing and Certification Model for W3C Specifications
• Test Development FAQ Document
• W3C Quality Assurance Library
• W3C Quality Assurance Glossary
• W3C QA Tools
• W3C Quality Assurance Interest Group
• W3C QA Interest Group Charter (QA IG)
• W3C QA Working Group (QAWG) Home Page
• W3C QA Working Group Charter (QAWG)
• W3C Quality Assurance Activity Statement
• Mail Archives for 'www-qa@w3.org' — for discussion on topics related to W3C's Quality assurance, and reviews of the work done W3C's QA WG.

TaMIE Technical Committee Proposal Comment

From Proposal Section 1c. Scope, "input material to this TC, which will deserve prime attention from the TC assuming their IP status is compatible with the IPR mode of the TC: QA Framework: Specification Guidelines (W3C, November 2004)"

---

WS-I Test Assertion Documents/Tools

WS-I Testing Tools Working Group

The Testing Tools Working Group of the Web Services Interoperability Organization (WS-I) develops the documentation and processes for WS-I test tools development, and develops or supervises the development of materials that test Web service implementations for conformance with WS-I profiles.

WS-I Testing Tools are used to determine whether the messages exchanged with a Web service conform to WS-I guidelines. These tools monitor the messages and analyze the resulting log to identify any known interoperability issues. These testing capabilities are important for developers to ensure that their implementations comply with the current interoperability guidelines for the use of Web services specifications. Tests are self administered and aimed at uncovering unconventional usage or errors in specification implementations, thus improving interoperability between applications and across platforms. To date, WS-I has developed tests for developers to verify their conformance with the Basic Profile 1.0, and work on the other WS-I profiles is underway.

Working Group Charter: Web Services Profile Testing Tools and Material. Edited by Tom Glover (IBM), Christopher Kurt (Microsoft), Jeff Mischkinsky (Oracle), Jacques Durand (Fujitsu), Narendra Patil (Optimyz Software, Inc), Steve Holbrook (IBM). Also available in .mht and .doc formats.

The Working Group was chartered to "develop WS-I Test Material and Tools that will facilitate the verification of conformance to WS-I profiles. The artifacts monitored and analyzed by the tools include message material, Web Service description material and registry material. For each WS-I profile, the working group will define test assertions as a testable interpretation of the profile definitions, gathered in Test Assertions documents.

The working group will define or supervise the definition of test case material necessary to verify that the tools behave as expected, and that multiple WS-I tool implementations function in a consistent manner. Tools produced will attempt to implement testing of all assertions within Test Assertion documents in support of the use of these tools in the evaluation of profile compliance...

WS-I Test Assertion Document

• Basic Security Profile [1.0] Test Assertions Version 1.0. BSP 1.0 TAD Version 1.0. Working Group Draft. 2006-06-13, uploaded 17-October-2006. Edited by Ram Poornalingam (Microsoft Corporation), Ed Johns (Microsoft Corporation), Govind Ramanathan (Microsoft Corporation), Shrikant Wagh (Optimyz Software, Inc), David Lauzon (IBM Corporation), Craig Chaney (IBM Corporation). Other contributors are noted by Keith Stobie (Microsoft Corporation) and Martin Gudgin (Microsoft Corporation). Also available in XML format.

  The Test Assertion Document is the formal specification of the tests performed by the WS-I tools as specified in the interoperability Profiles. It is used by the Analyzer tool to identify interoperability issues. The analyzer specification [WS-I Analyzer Tool Functional Specification] contains a detailed explanation of all of the fields listed in this document. This is [as of 2007-11] the current Working Group Draft TAD for the Basic Security Profile.

• Attachments Profile [1.0] (with Basic Profile [1.1] and Simple Soap Binding Profile [1.0]) Test Assertions Version 1.0. WS-I Final Material. June 13, 2005. AP 1.0. SSBP 1.0. TAD Version 1.0. Edited by David Lauzon (IBM Corporation) and Shrikant Wagh (Optimyz Software, Inc). Other contributors noted from Simeon Greene (Oracle Corporation), Narendra Patil (Optimyz Software, Inc.), and Keith Stobie (Microsoft Corporation).

  "This document contains the test assertions for the WS-I Attachments Profile 1.0 combined with the test assertions for the WS-I Simple Soap Binding Profile 1.0 and WS-I Basic Profile 1.1. These test assertions are used by the analyzer testing tool to determine if a Web service is conformant to the Attachments Profile 1.0 in conjunction with the Simple Soap Binding Profile 1.0 and the Basic Profile 1.1."

• Basic Profile 1.0 Test Assertions Version 1.1. Final Material. 2005-01-12. Edited by Jacques Durand (Fujitsu), Simeon Greene (Oracle Corporation), Peter Brittenham (IBM Corporation), Keith Stobie (Microsoft Corporation), David Lauzon (IBM Corporation), Lucien Kleijkers (Microsoft Corporation). Other contributors are noted as Ed Johns (Microsoft Corporation), Narendra Patil (Optimyz), Ajay Honnur (BEA), Rami Jaamour (Parasoft), and Shrikant Wagh (Optimyz). Also available in XML format.

  "The Test Assertion Document is the formal specification of the tests performed by the WS-I tools as specified in the interoperability Profiles. It is used by the Analyzer tool to identify interoperability issues." WS-I Basic Profile 1.0 defines a set of non-proprietary Web services specifications, along with clarifications and amendments to those specifications which promote interoperability.

• Basic Profile 1.1 Test Assertions Version 1.1. Final Material. 2005-01-12. Edited by Simeon Greene (Oracle Corporation), David Lauzon (IBM Corporation), and Keith Stobie (Microsoft Corporation), Other contributors are noted by Narendra Patil (Optimyz), Ed Johns (Microsoft Corporation), Ajay Honnur (BEA), Rami Jaamour (Parasoft), Shrikant Wagh (Optimyz). Also available in XML format.

  The Test Assertion Document is the formal specification of the tests performed by the WS-I tools as specified in the interoperability Profiles. It is used by the Analyzer tool [WS-I Analyzer Tool Functional Specification] to identify interoperability issues...

• Simple Soap Binding Profile [1.0] (with Basic Profile [1.1]) Test Assertions Version 1.0. Final Material. 2005-01-12. SSBP 1.0. BP 1.1. TAD Version 1.0. Edited by Simeon Greene (Oracle Corporation), David Lauzon (IBM Corporation), Keith Stobie (Microsoft Corporation), Ed Johns (Microsoft Corporation), and Shrikant Wagh (Optimyz Software, Inc). Other contributors are noted from Ajay Honnur (BEA), Rami Jaamour (Parasoft), Narendra Patil (Optimyz Software, Inc). Also available in XML format.

  This document contains the test assertions for the WS-I Simple SOAP Binding Profile 1.0 combined with the test assertions for the WS-I Basic Profile definition. These test assertions are used by the analyzer testing tool to determine if a Web service is conformant to the Simple Soap Binding Profile 1.0 in conjunction with the Basic Profile 1.1.

• WS-I Profile Test Assertion Document. Basic Profile Test Assertions, version 1.0.1.7.1. January 2004. "The Test Assertion Document is the formal specification of the tests performed by the WS-I tools as specified in the interoperability Profiles. It is consumed directly by the Analyzer to identify interoperability issues. This is the final TAD for version 1 of the WS-I Test Tools. A candidate element is one that is to be verified for conformance. The binding of the tModel if <wsi-analyzerConfig:uddiReference> is given or the <wsi-analyzerConfig:wsdlElement> in the configuration file of the Analyzer define a candidate element for verification. A verification on an element also implies that the same verification is made for all the elements that it uses. That is, the elements it uses also become candidate elements. Verification it based on transitivity rules, applied recursively, for (1) WSDL element references and (2) UDDI references..."

TaMIE Technical Committee Proposal Comment

Proposal Section 2a. Similar Work: Identification of similar or applicable work that is being done in other OASIS TCs or by other organizations... Test Assertion Documents for WS-I profiles (WS-I, 2003-2005)."

See also Proposal Section 2a. Identification of similar or applicable work that is being done in other OASIS TCs or by other organizations: "Web Services Interoperability (WS-I) Test Tools (for Basic Profile 1.1). These tools do not allow for the scripting of a test suite; instead they apply a battery of predefined tests to any Web service material provided as input. The objective is to verify the conformance of this material to one of the WS-I profiles. The tools only passively analyze message traffic and have no ability to control the systems under test. Nevertheless, this capture and analysis architecture is un-intrusive, simple, and supports deferred testing. The analyzer tool of WS-I cannot be used as a general test engine, being tied to profile definitions; but, it can be leveraged as a specialized module by a more general monitoring capability, when the conformance of message material to WS-I profiles is required."

---