W3C has published three Recommendations which improve Web services performance by standardizing the transmission of large binary data. The specifications have been produced by the XML Protocol Working Group as part of the W3C Web Services Activity.

Using an XML Schema datatype, the XML-binary Optimized Packaging (XOP) specification defines an efficient serialization of XML element content based upon XML Infosets. A SOAP Optimized Serialization Use Cases and Requirements document created by the XMLP Working Group motivates this work; it provides use cases and specifies the requirements for optimizing the processing and serialization of SOAP messages.

The SOAP Message Transmission Optimization Mechanism (MTOM) provides optimization for hop-by-hop exchanges between SOAP nodes using a XOP-based selective encoding. The concrete implementation in the MTOM Recommendation thus relies on the XML-binary Optimized Packaging format for carrying SOAP messages. It "describes an abstract feature and a concrete implementation of it for optimizing the transmission and/or wire format of SOAP messages."

The Resource Representation SOAP Header Block specification "describes the semantics and serialization of a SOAP header block for carrying resource representations in SOAP messages." This RRSHB functionality "allows SOAP message recipients to access cached representations of external resources. This is important, as there may be times when there are either limits to bandwidth or access of files. It gives the recipient the option of using either the original file that may be identified by a URI, or to use a cached copy that accompanies the actual SOAP message. Used with MTOM, it enhance greatly the speed and of processing as the external data is already present when the recipient is starting processing the message."

According to the W3C announcement, "one of the biggest technical and performance issues for Web services occurs when a user or application is handling large binary files. Encoding binary data as XML produces huge files, which absorbs bandwidth and measurably slows down applications. For some devices, it slows down so much that the performance is considered unacceptable."

Also, Web Services applications "have the primary goal of sharing and using data between applications. This includes an increasingly diverse set of media formats and devices, including large schematics and other graphical files. Examples are as intricate as sharing architectural blueprints between multiple parties, or as simple as sending a photo from a digital camera directly to a printer."

The three W3C Recommendations just published "work with SOAP 1.2 to address the specific issue of improving Web services performance by providing standard methods and mechanisms for transmitting large binary data."

Bibliographic Information

• XML-binary Optimized Packaging. W3C Recommendation. 25-January-2005. Edited by Martin Gudgin (Microsoft), Noah Mendelsohn (IBM), Mark Nottingham (BEA), and Hervé Ruellan (Canon). Version URL: http://www.w3.org/TR/2005/REC-xop10-20050125/. Latest version URL: http://www.w3.org/TR/xop10/. Previous version URL: http://www.w3.org/TR/2004/PR-xop10-20041116/.

  Acknowledgements: "This specification is the work of the W3C XML Protocol Working Group. Participants in the Working Group are, at the time of writing, and by alphabetical order: David Fallside (IBM), Tony Graham (Sun Microsystems), Martin Gudgin (Microsoft Corporation, formerly of DevelopMentor), Marc Hadley (Sun Microsystems), Gerd Hoelzing (SAP AG), John Ibbotson (IBM), Anish Karmarkar (Oracle), Suresh Kodichath (IONA Technologies), Yves Lafon (W3C), Michael Mahan (Nokia), Noah Mendelsohn (IBM, formerly of Lotus Development), Jeff Mischkinsky (Oracle), Jean-Jacques Moreau (Canon), Mark Nottingham (BEA Systems, formerly of Akamai Technologies), David Orchard (BEA Systems, formerly of Jamcracker), Herve Ruellan (Canon), Jeff Schlimmer (Microsoft Corporation), Pete Wenzel (SeeBeyond), Volker Wiechers (SAP AG)."

• SOAP Message Transmission Optimization Mechanism. W3C Recommendation. 25-January-2005. Edited by Martin Gudgin (Microsoft), Noah Mendelsohn (IBM), Mark Nottingham (BEA), and Hervé Ruellan (Canon). Version URL: http://www.w3.org/TR/2005/REC-soap12-mtom-20050125/. Latest version URL: http://www.w3.org/TR/soap12-mtom/. Previous version URL: http://www.w3.org/TR/2004/PR-soap12-mtom-20041116/.

• Resource Representation SOAP Header Block. W3C Recommendation. 25-January-2005. Edited by Anish Karmarkar (Oracle Corp), Martin Gudgin (Microsoft), and Yves Lafon (W3C). Version URL: http://www.w3.org/TR/2005/REC-soap12-rep-20050125/. Latest version URL: http://www.w3.org/TR/soap12-rep/. Previous version URL: http://www.w3.org/TR/2004/PR-soap12-rep-20041116/.

About XML-binary Optimized Packaging

"A XOP package is created by placing a serialization of the XML Infoset inside of an extensible packaging format (such a MIME Multipart/Related, RFC 2387). Then, selected portions of its content that are base64-encoded binary data are extracted and re-encoded (i.e., the data is decoded from base64) and placed into the package. The locations of those selected portions are marked in the XML with a special element that links to the packaged data using URIs.

In a number of important XOP applications, binary data need never be encoded in base64 form. If the data to be included is already available as a binary octet stream, then either an application or other software acting on its behalf can directly copy that data into a XOP package, at the same time preparing suitable linking elements for use in the root part; when parsing a XOP package, the binary data can be made available directly to applications, or, if appropriate, the base64 binary character representation can be computed from the binary data.

However, at the conceptual level, this binary data can be thought of as being base64-encoded in the XML Document. As this conceptual form might be needed during some processing of the XML Document (e.g., for signing the XML document), it is necessary to have a one to one correspondence between XML Infosets and XOP Packages. Therefore, the conceptual representation of such binary data is as if it were base64-encoded, using the canonical lexical form of XML Schema base64Binary datatype; see XML Schema Part 2: Datatypes Second Edition 3.2.16, base64Binary. In the reverse direction, XOP is capable of optimizing only base64-encoded Infoset data that is in the canonical lexical form.

Only element content can be optimized; attributes, non-base64-compatible character data, and data not in the canonical representation of the base64Binary datatype cannot be successfully optimized by XOP..." [from the XOP spec Introduction]

About SOAP Message Transmission Optimization Mechanism

The first part of the MTOM Recommendation "describes an abstract feature for optimizing the transmission and/or wire format of a SOAP message (SOAP Version 1.2 Part 1: Messaging Framework) by selectively encoding portions of the message, while still presenting an XML Infoset to the SOAP application.

Use of the Abstract SOAP Transmission Optimization Feature is a hop-by-hop contract between a SOAP node and the next SOAP node in the SOAP message path, providing no mandatory convention for optimization of SOAP transmission through intermediaries. The feature does provide optional means by which binding implementations MAY choose to facilitate the efficient pass-through of optimized data contained within headers or bodies relayed by an intermediary. Additional specifications might also be written to provide for other optimized multi-hop capabilities, perhaps building on the mechanisms provided herein.

The second part of the Recommendation describes an Optimized MIME Multipart/Related Serialization of SOAP Messages implementing the Abstract SOAP Transmission Optimization Feature in a binding independent way. This implementation relies on the XML-binary Optimized Packaging (XOP) format.

The third part of the MTOM Recommendation uses this Optimized MIME Multipart/Related Serialization of SOAP Messages for describing an implementation of the Abstract Transmission Optimization Feature for the SOAP 1.2 HTTP binding. See SOAP Version 1.2 Part 2: Adjuncts, 7 - 'SOAP HTTP Binding'..." [from the MTOM REC Introduction]

About Resource Representation SOAP Header Block

As defined in this W3C Recommendation, the Representation header block "is designed to allow applications to carry a representation of a Web resource in a SOAP message. Applications of this header include cases where the receiver has limited ability to get the representation using other means, for example because of access restrictions or because the overhead would be unacceptable. The Representation header block is also useful when multiple references to the same resource are required but duplication of the resource is undesirable. See UC2 and SOAP Optimized Serialization Use Cases and Requirements (UC6) for details.

The meaning of the Representation header block, when present in a SOAP message, is to make available the contained representation of the resource it carries to the processing SOAP node. The SOAP node may use this representation when dereferencing the URI of the resource instead of making a network request to obtain a representation of the resource. Note that implementations may need to process a Representation header block before processing other header blocks that require dereferencing of a URI whose representation is carried in the Representation header block.

Multiple occurrences of the Representation header block may be present in the same SOAP Message to carry representation of multiple Web resources or multiple representations of the same Web resource. Several occurrences of the Representation header block having the same value for the role and resource attribute information item may be present in the same SOAP Message. Such Representation header blocks should not have the same metadata (such as media-type). If such Representation header blocks have the same metadata then any one of them may be used.

URIs that are character for character identical must be considered equal when using a representation header to resolve a web reference; URIs that are considered equal according to the URI scheme of the URI should be considered equal..." [from the RRSHB spec Introduction]

W3C XML Protocol Working Group

"Today, the principal use of the World Wide Web is for interactive access to documents and applications. In almost all cases, such access is by human users, typically working through Web browsers, audio players, or other interactive front-end systems. The Web can grow significantly in power and scope if it is extended to support communication between applications, from one program to another. The purpose of this Working Group is to create a simple foundation to support the needs of such communicating applications...

A broad range of applications will eventually be interconnected through the Web. The initial focus of this Working Group is to create simple protocols that can be ubiquitously deployed and easily programmed through scripting languages, XML tools, interactive Web development tools, etc. The goal is a layered system which will directly meet the needs of applications with simple interfaces (e.g., getStockQuote, validateCreditCard), and which can be incrementally extended to provide the security, scalability, and robustness required for more complex application interfaces.

Experience with SOAP, XML-RPC, WebBroker, etc. suggests that simple XML-based messaging and remote procedure call (RPC) systems, layered on standard Web transports such as HTTP and SMTP, can effectively meet these requirements. Specifically, the XML Protocol Working Group is chartered to design the following four components:

• An envelope for encapsulating XML data to be transferred in an interoperable manner that allows for distributed extensibility and evolvability as well as intermediaries.
• A convention for the content of the envelope when used for RPC (Remote Procedure Call) applications. The protocol aspects of this should be coordinated closely with the IETF and make an effort to leverage any work they are doing, see below for details.
• A mechanism for serializing data representing non-syntactic data models such as object graphs and directed labeled graphs, based on the datatypes of XML Schema.
• A mechanism for using HTTP transport in the context of an XML Protocol. This does not mean that HTTP is the only transport mechanism that can be used for the technologies developed, nor that support for HTTP transport is mandatory. This component merely addresses the fact that HTTP transport is expected to be widely used, and so should be addressed by this Working Group...

Furthermore, the following two general requirements must be met by the work produced by this Working Group:

• The envelope and the serialization mechanisms developed by the Working Group may not preclude any programming model nor assume any particular mode of communication between peers.
• Focus must be put on simplicity and modularity and must support the kind of extensibility actually seen on the Web. In particular, it must support distributed extensibility where the communicating parties do not have a priori knowledge of each other.

The Working Group shall start by developing a requirements document, and then evaluate the technical solutions proposed in the SOAP/1.1 submission against these requirements. If in this process the Working Group finds solutions that are agreed to be improvements over solutions suggested by SOAP 1.1, those improved solutions should be used..." [from the WG Charter]