The Cover PagesThe OASIS Cover Pages: The Online Resource for Markup Language Technologies
Advanced Search
Site Map
CP RSS Channel
Contact Us
Sponsoring CP
About Our Sponsors

Cover Stories
Articles & Papers
Press Releases

XML Query

XML Applications
General Apps
Government Apps
Academic Apps

Technology and Society
Tech Topics
Related Standards
Created: September 12, 2005.
News: Cover StoriesPrevious News ItemNext News Item

W3C Publishes QA Handbook and Related Quality Assurance Specifications.


On September 06, 2005, W3C announced the publication of The QA Handbook, developed by members of the Quality Assurance Working Group as the latest of the W3C QA Framework documents. While some features of specification QA presented in the document are unique to W3C's technical process and software automation tools, several of the resources referenced by The QA Handbook have broad applicability to formal specification development in any similar standards organization.

Release of the The QA Handbook follows publication of the QA Framework Specification Guidelines, QA Framework Primer, and Variability in Specifications Note in August 2005. The Handbook and related resources have been produced through the W3C Quality Assurance Activity and its maintenance program.

The QA Handbook documents a set of good practices that helps Working Groups improve their deliverables and keep their schedules. It is a non-normative handbook describing "processes and operational aspects of certain quality assurance practices of W3C's Working Groups, with particular focus on testability and test topics. It is intended for Working Group chairs and team contacts. It aims to help them to avoid known pitfalls and benefit from experiences gathered from the W3C Working Groups themselves. It provides techniques, tools, and templates that should facilitate and accelerate their work. Supported by real-world stories and examples, The QA Handbook offers a practical guide to applying good practices and quality assurance techniques to WG activities, especially developing Recommendations and test materials."

Published as a W3C Recommendation on August 17, 2005, the QA Framework: Specification Guidelines document is designed to help W3C editors write better specifications by making a specification easier to interpret without ambiguity and clearer as to what is required in order to conform. It focuses on how to define and specify conformance and addresses how a specification might allow variation among conforming implementations. The document presents guidelines or requirements, supplemented with good practices, examples and techniques. W3C urges the Guidelines be applied to technical work from very beginning, with attention to a conformance model, normative language usage, writing with test assertions, optional featuresand extensibility, as well as profiles, levels, and conformance claims. These Guidelines are to be followed along with the W3C "pubrules" and W3C Manual of Style to support testable language, clarity, conciseness."

A QA Framework document on the Variability in Specifications was published on August 31, 2005. This document details some of the most important conformance-related concepts evoked in the QA Specification Guidelines. It analyzes how design decisions of a specification's conformance model may affect its implementability and the interoperability of its implementations. To do so, it introduces the concept of variability (how much implementations conforming to a given specification may vary among themselves) and presents a set of well-known dimensions of variability. Seven dimensions are presented in a sequence from most to least independence from other design factors. The document's goal is to raise awareness of the potential cost that some benign-looking decisions may have on interoperability and to provide guidance on how to avoid these pitfalls by better understanding the mechanisms affected by variability."

A companion QA Framework Primer updated in August 2005 gives an introduction to the QA Framework at W3C. The Primer provides a roadmap to the QA Framework document and orients the reader by presenting the QA Framework from three different prespectives — a document view, a role-based view, and a Working Group milestones view.

The Matrix of W3C Specifications document, produced using XML, RDF, XSLT, and XHTML, is also published as part of the QA Framework. The Matrix of W3C specifications displays a table for deliverables at least at Last Call stage, except if the team is working on a Test Suite at Working Draft stage. The default view is sorted by title; a view sorted by status is also available. For each specification in the Matrix, the table indicates the URI, the specification maturity level, and symbols indicating associated QA tools. A validator icon is shown if the specification has a validator, a W3C Test suite icon if the specification has a Test Suite located on the W3C server, a generic/remote Test Suite icon if the specification has such a Test Suite, and a Conformance section icon if the specification has a conformance section.

The Matrix of W3C Specifications is also part of the W3C TR Automation Project. In this automation project, the use of Semantic Web tools and technologies has allowed W3C to streamline the publication paper trail of W3C Technical Reports by maintaining an RDF-formalized index of specifications, and to create a number of tools using (meta)data extracted from XHTML into various databases.

The Matrix shows related documents with hyperlinks if they are available: Errata, Implementation Reports, Primer, Requirements Documents, Use Cases, Accessibility Support Document, Authoring Techniques Document, etc. The displayed XHTML output has been produced with an XSL stylesheet; the Matrix represents an XHTML ouput of an RDF maintained list of QA related information merged with the RDF version of the TR page. Production details are described in the Manual and in the document "Why the QA Matrix in RDF?"

W3C's QA Activity was charged with a fourfold mission: (1) "to improve the quality of W3C specifications through guidelines and reviews of specifications at critical stages of their development; (2) to promote wide deployment and proper implementation of formal specifications through articles, tutorials, and validation services; (3) to communicate the value of test suites and help Working Groups produce quality test suites; (4) to design effective processes that, if followed, will help working groups achieve QA goals."

Bibliogaphic Information

  • The QA Handbook. Edited by Lofton Henderson. W3C Working Group Note. 6-September-2005. Version URL: Latest version URL: Previous version URL:

    Acknowledgments to the following QA Working Group and Interest Group participants have contributed significantly to The QA Handbook document: Daniel Dardailler (W3C), Dimitris Dimitriadis (Ontologicon), Karl Dubost (W3C), Dominique Hazaël-Massieux (W3C), Lofton Henderson (CGM Open), David Marston (Lotus Development Corp), Patrick Curran (Sun Microsystems), Lynne Rosenthal (NIST), Mark Skall (NIST), Andrew Thackrah (Open Group), Olivier Thereaux (W3C), Jeremy Carroll (Hewlett-Packard), Richard Kennedy (The Boeing Company), and Tim Boland (NIST).

  • QA Framework: Specification Guidelines. W3C Recommendation. 17-August-2005. Edited by Karl Dubost (W3C), Lynne Rosenthal (NIST), Dominique Hazaël-Massieux (W3C), and Lofton Henderson (CGM Open). Version URL: Latest version URL: Previous version URL:

  • Variability in Specifications. W3C Working Group Note. 31-August-2005. Edited by Dominique Hazaël-Massieux (W3C) and Lynne Rosenthal (NIST). Version URL: Latest version URL: Previous version URL:

  • QA Framework Primer. Edited by Lofton Henderson (CGM Open), Lynne Rosenthal (NIST), Mark Skall (NIST). Last Modified: 24-August-2005 or later.

  • Test Metadata. W3C Working Group Note. 14-September-2005. Edited by Patrick Curran (Sun Microsystems) and Karl Dubost (W3C). Produced by members of the the W3C Quality Assurance Working Group.

    "To be truly useful, a test suite should consist of more than a simple collection of tests. Additional information is typically required to help users understand how to execute the tests and how to interpret the results. Much of this information should be provided in the test suite documentation but some is more appropriately provided in the form of metadata about the tests themselves. Well-defined metadata can help in: (1) tracking tests during the development and review process; (2) filtering tests according to a variety of criteria — for example, whether or not they are applicable for a particular profile or optional feature; (3) identifying the area of the specification that is tested by the tests; (4) constructing a test harness to automatically execute the tests; (5) formatting test results so that they are easily understood. Most test suites provided by W3C Working Groups make use of some form of metadata. However, the extent of metadata usage and the forms and syntax in which metadata elements are defined varies widely from Group to Group. This document defines a minimal set of metadata elements that have proved useful in practice and attempts to standardize their names, syntax, and usage. If the use of standard metadata elements is adopted within the W3C it is likely that standardized tools will be developed to facilitate the tasks listed above."

  • Test Development FAQ. Updated 2005/06/23 or later. "The Test FAQ provides introductory information about the purpose of testing, how to get started, and what the testing process involves. This FAQ primarily documents what is already considered good testing practice or the norm, but it also includes a number of advanced testing goals that have not yet been fully achieved by any Working Group. The Test FAQ document is addressed to those who develop tests or organize testing efforts. It should also be useful to those who develop specifications or who run tests. It stresses early planning for testing, defines what makes a good test and examines test reporting, publishing and packaging of tests..."

  • Conformance and Quality Assurance: The Matrix of W3C Specifications. The Matrix of W3C Specifications, gathers and formalizes QA efforts for W3C specifications. For each specification entry, a symbol indicates if the specification has a conformance clause and if there are conformance tools or test suites associated with the specification. As of September 12, 2005, the Matrix contained 79 Recommendations, 15 Candidate Recommendations, 2 Proposed Recommendations, and 30 Last Call Working Drafts. Produced using XML, RDF, XSLT, and XHTML as part of the W3C TR Automation Project.

  • QA Wiki Space. Includes various pages on developing a test suite, including QA Tools.

Related News: EARL 1.0 Schema Working Draft

On 2005-09-12, W3C announced that the Evaluation and Repair Tools Working Group had released an initial Working Draft of the Evaluation and Report Language (EARL) 1.0 Schema.

EARL goals are defined in a document "Requirements for the Evaluation and Report Language (EARL) 1.0": "[the document] describes the requirements for the scope, design and features of the Evaluation and Report Language (EARL) 1.0. The Evaluation and Report Language is a standardized format to express test results. The primary motivation for developing this language is to facilitate the exchange of test results between Web accessibility evaluation tools in a vendor neutral and platform independent format. It will also provide reusable vocabulary for generic Web quality assurance and validation purposes."

Evaluation and Report Language (EARL) 1.0 Schema. W3C Working Draft. 09-September-2005. Edited by Charles McCathieNevile (Opera Software) and Shadi Abou-Zahra (World Wide Web Consortium).

EARL 1.0 Schema Abstract: "This document describes the formal schema of the Evaluation and Report Language (EARL) 1.0. The Evaluation and Report Language is a standardized format to express test results. The primary motivation for developing this language is to facilitate the exchange of test results between Web accessibility evaluation tools in a vendor neutral and platform independent format. It also provides reusable vocabulary for generic Web quality assurance and validation purposes. While this document focuses on the technical details of the specification, a companion document describes the motivations for EARL and provides a tutorial introduction to its use..."

Principal References

Hosted By
OASIS - Organization for the Advancement of Structured Information Standards

Sponsored By

IBM Corporation
ISIS Papyrus
Microsoft Corporation
Oracle Corporation


XML Daily Newslink
Receive daily news updates from Managing Editor, Robin Cover.

 Newsletter Subscription
 Newsletter Archives
Bottom Globe Image

Document URI:  —  Legal stuff
Robin Cover, Editor: