Guidelines For The Use of XML in IETF Protocols

The eXtensible Markup Language (XML) is a framework for structuring data. While it evolved from the Standard Generalized Markup Language (SGML) -- a markup language primarily focused on structuring documents -- XML has evolved to be a widely-used mechanism for representing structured data in protocol exchanges. See for an introduction to XML.

Many Internet protocol designers are considering using XML and XML fragments within the context of existing and new Internet protocols. This document is intended as a guide to XML usage and as IETF policy for standards track documents. Experienced XML practitioners will likely already be familiar with the background material here, but the guidelines are intended to be appropriate for those readers as well.

This document is intended to give guidelines for the use of XML content within a larger protocol. There are a number of protocol frameworks already in use or under development which focus entirely on "XML protocol": the exclusive use of XML as the data representation in the protocol. For example, the World Wide Web Consortium (W3C) is developing an XML Protocol framework based on the Simple Object Access Protocol (SOAP) . The applicability of those protocols is not part of the scope of this document. In addition, there are higher-level representation frameworks, based on XML, that have been designed as carriers of certain classes of information; for example, the Resource Description Framework (RDF) is an XML-based representation for logical assertions. This document does not provide guidelines for the use of such frameworks.

Originally published in February 1998 , XML's popularity has led to several additions to the base specification. Although these additions are designed to be consistent with version 1.0 of XML, they have varying levels of stability, consensus, and implementation. Accordingly, this document identifies the major evolutionary features of XML and makes suggestions as to the circumstances in which each feature should be used.

There are many XML support groups, some devoted to the entire XML industry (e.g., http://xml.org/), some devoted to developers (http://xmlhack.com/), some devoted to the business applications of XML (e.g., http://oasis-open.org/), and many, many groups devoted to the use of XML in a particular context. It is beyond the scope of this document to provide a comprehensive list of referrals. Interested readers are directed to the three links above as starting points, as well as their favorite Internet search engine. (TBD: pointers to other best practice and design guidelines, such as http://www.xfront.com/BestPracticesHomepage.html and http://www.goland.org/xmlschema.htm)

XML is a tool that provides a means towards an end. Choosing the right tool for a given task is an essential part of ensuring that the task can be completed in a satisfactory manner. This section describes factors to be aware of when considering XML as a tool for use in IETF protocols: XML is a meta-markup language that can be used to define markup languages for specific domains and problem spaces. XML provides both logical structure and physical structure to describe data. Data framing is built-in. XML includes features to support internationalization and localization. XML is extensible. New tags (and thus new protocol elements) can be defined without requiring changes to XML itself. XML is still evolving. The formal specifications are still being influenced and updated as use experience is gained and applied. XML is text-based, so XML fragments are easily created, edited, and managed using common utilities. Further, being text-based means it more readily supports incremental development, debugging, and logging. XML is verbose when compared with many other data encapsulation languages. A representation with element extensibility and human readability typically requires more bits when compared to one optimized for efficient machine processing. XML implementations are still relatively new. As designers and implementers gain experience, it is not uncommon to find defects in early and current products. XML support is available in a large number of software development utilities, available in both open source and proprietary products. XML processing speed can be an issue in some environments. XML processing can be slower because XML data streams may be larger than other representations, and the use of general purpose XML parsers will add a software layer with its own performance costs.

This document focuses on guidelines for the use of XML, but it's useful to consider why one would use XML as opposed to some other mechanism. This section considers some other commonly used representation mechanisms and compares XML to those alternatives. For example, Abstract Syntax Notation 1 (ASN.1) along with the corresponding Basic Encoding Rules (BER) are part of the OSI communication protocol suite, and have been used in many subsequent communications standards (e.g., the ANSI Information Retrieval protocol and the Simple Network Management Protocol (SNMP) ). The eXternal Data Representation (XDR) and variations of it have been used in many other distributed network applications (e.g., the Network File System protocol ). With ASN.1, data types are explicit in the representation, while with XDR, the data types of components are described externally as part of an interface specification. Many other protocols use data structures directly (without data encapsulation) by describing the data structure with Backus Normal Form (BNF) ; many IETF protocols use an Augmented Backus-Naur Form (ABNF) . The Simple Mail Transfer Protocol is an example of a protocol specified using ABNF. Representation methods differ from XML in several important ways: Specification encoding: XML schema are themselves represented in XML, and the specification itself can be written using arbitrary characters from the language. The specification of representations in other systems (ASN.1, XDR, ABNF) are generally in ASCII text. Text Encoding and character sets: the character encoding used to represent a formal specification. XML defines a consistent character model based on ISO 10646 , with a base that supports at least UTF-8 and UTF-16 , and allows for other encodings. While ASN.1 and XDR may carry strings in any encoding, there is no common mechanism for defining character encodings within them. Typically, ABNF definitions tend to be defined in terms of octets or characters in ASCII. Data Encoding: XML is based on a character model. XML Schema includes mechanisms for representing some datatypes (integer, date, array, etc.) but other binary datatypes are encoded in Base64 . ASN.1 and XDR have rich mechanisms for encoding a wide variety of datatypes. Extensibility: XML has a rich extensibility model: XML representations can frequently be versioned independently. Many XML representations can be extended by adding tokens to the XML namespace (if done compatibly); other extensions can be added by adding to the namespace. ASN.1 is similarly extensible through the use of Object Identifiers (OIDs). XDR representations tend to not be independently extensible by different parties because the framing and datatypes are implicit and not self-describing. The extensibility of BNF-based protocol elements needs to be explicitly planned. Legibility of protocol elements: As noted above, XML is text-based, and thus carries the advantages (and disadvantages) of text-based protocol elements. Typically this is shared with (A)BNF-defined protocol elements. ASN.1 and XDR use binary encodings which are not visible. ASN.1, XDR, and BNF are described here as examples of alternatives to XML for use in IETF protocols. There are other alternatives, but a complete enumeration of all possible alternatives is beyond the scope of this document.

This section notes several aspects of XML and makes recommendations for use. Since the 1998 publication of XML version 1 , an editorial second edition was published in 2000; this section refers to the second edition.

An XML declaration (defined in section 2.8 of ) is a small header at the beginning of an XML data stream that indicates the XML version and the character encoding used. For example, <?xml version="1.0" encoding="UTF-8"?> specifies the use of XML version 1 and UTF-8 character encoding. Protocol specifications must be clear about use of XML declarations. In some cases, the XML used is a small fragment in a larger context, where the XML version and character encoding are specified externally. In those cases, the XML declaration might add extra overhead. In other cases, the XML is a larger component which may find its way alone as an external entity body, transported as a MIME message. In those cases, the XML declaration is an important marker and useful for reliability and extensibility. In general, an XML protocol element should either disallow XML declarations ("MUST NOT be used") or require one ("MUST have"). A design which allows but does not require an XML declaration leads to unreliable implementations. When in doubt, require an XML declaration.

An XML processing instruction (defined in section 2.6 of ) is a component of an XML document that signals extra "out of band" information to the receiver; a common use of XML processing instructions are for document applications. For example, the XML2RFC application used to generate this document and described in supports a "table of contents" processing instruction: <?rfc toc="yes"?> Again, protocol specifications must be clear about whether -- and if so, what kind of -- XML processing instructions are allowed. However, XML processing instructions appear to have rare applicability to XML fragments embedded in Internet protocols, and it is recommended that their use be explicitly disallowed ("MUST NOT use"). In cases where XML processing instructions are allowed, the nature of the allowable processing instructions should be specified explicitly.

A well-formed XML instance is one in which all character and markup data conforms to a specific set of structural rules defined in section 2.1 of . An XML instance that is not well-formed is not really XML; well-formedness is the basis for syntactic compatibility with XML. Without well-formedness, most of the advantages of using XML disappear. For this reason, it is imperative that protocol specifications REQUIRE that XML instances be well-formed.

Beyond well-formedness there are additional mechanisms that define a set of structural and data format constraints. Two mechanisms are commonly used to define grammars for classes of XML documents: Document Type Definition (DTD) (defined in section 2.8 of ) and XML Schema (defined in and ). DTDs are an older technology that has been found to have drawbacks, particularly in the features provided for extensibility and data typing. XML Schema was designed to address many DTD shortcomings. For example, with a DTD a validating parser can confirm that an element contains character data, but with XML Schema a validating parser can also confirm that the value of an element matches a particular regular expression. XML Schema provides powerful features to define a complete and precise specification of allowable protocol syntax and data type definitions. In order to obtain the advantages of XML as a data structure specification system, protocol specifications should supply an XML Schema and insist that XML instances MUST be valid according to that schema.

XML namespaces, defined in , provide a means of assigning markup to a specific vocabulary. If two elements or attributes from different vocabularies have the same name, they can be distinguished unambiguously if they belong to different namespaces. Additionally, namespaces provide significant support for protocol extensibility as they can be defined, reused, and processed dynamically. Markup vocabulary collisions are very possible when namespaces are not used to separate and uniquely identify vocabularies. Protocol definitions should use existing XML namespaces where appropriate. When new namespaces are needed, the namespace name (a URI) should be defined within the RFC itself, and the IETF URN namespace described in should be used to designate the namespace; for example: abc:xmlns="urn:ietf:params:xml:ns:abc"

XML provides much flexibility in allowing a designer to use either elements or element attributes to carry data. Element attributes are generally intended to contain meta-data that describes the value of the element, and as such they are subject to the following restrictions: Attributes are unordered, and Attribute values can only contain simple XML data types. Consider the following example that describes an IP address using a "type" attribute to describe the address value:

<address type="ipv4">10.1.2.3</address> XML allows the same information to be encapsulated using a <type> element instead of a "type" attribute:

<address> <type>ipv4</type> <value>10.1.2.3</value> </address> The preferred form is used in the first example, where the "type" attribute is used to describe the value of the <address> element. Consistent use of elements and element attributes is a characteristic of a sound design. Protocols are strongly urged to use elements as the primary XML data encapsulation structure. Attributes used in protocol elements should contain only meta-data that describes the value of the enclosing element.

XML is defined as a character stream rather than a stream of octets, and such there are no ways of embedding binary data directly within an XML data stream. XML Schema defines one encoding of binary data using Base64 and another using hexadecimal digits. Other protocols transmit binary data using some other communication channel, and include, in the XML data itself, a reference (using the anyURI data type). Protocols that need a container that can hold both structural data and large quantities of binary data should consider carefully whether XML is appropriate, since the Base64 and hex encodings are inefficient. Otherwise, protocols should use the mechanisms of XML Schema to represent binary data; the Base64 encoding is best for larger quantities of data, while the hex encoding will work for short bit strings.

This section describes internationalization considerations for the use of XML to represent data in IETF protocols. Readers should be familiar with IETF policy on the use of character sets and languages as described in RFC 2277 .

XML provides native support for encoding information using the Unicode character set and its more compact representations including UTF-8 and UTF-16 . Other encodings are also supported and can be specified using an "encoding" attribute in a document's XML declaration. It is strongly recommended that UTF-8 be mandated for protocols that represent data using XML. Guidelines for the use of XML declarations can be found in . If an XML declaration is omitted, it is strongly urged to require use of a consistent character set, and to require UTF-8 as the most appropriate character set. If an XML declaration is allowed, it is again strongly urged to require use of a consistent character set, to require UTF-8 as the most appropriate character set, and to recommend inclusion of an "encoding" attribute that explicitly notes use of UTF-8 encoding.

Text encapsulated in XML can be represented in many different human languages, and it is often useful to explicitly identify the language used to present the text. XML version 1 defines a special attribute in the "xml" namespace, xml:lang, that can be used to specify the language used to represent data in an XML document. The xml:lang attribute and the values it can assume are defined in section 2.12 of . It is strongly recommended that protocols representing data in a human language mandate use of an xml:lang attribute if the XML instance might be interpreted in language-dependent contexts.

There are standard mechanisms in the typography of some human languages that can be difficult to represent using merely XML character string data types. For example, pronunciation clues can be provided using Ruby annotation , and embedding controls (such as those described in section 3.4 of ) or an XHTML "dir" attribute can be used to note the proper display direction for bidirectional text. It is strongly recommended that protocols representing data in a human language reuse existing mechanisms as needed to ensure proper display of human-legible text.

This section does not contain any specific directives for IANA. However, when XML is used in an IETF protocol there are multiple factors that might require IANA action, including: XML media types. Some protocols have protocol elements that are MIME bodies, and allow MIME labeling. In cases where a MIME label is used to identify a protocol element the MIME labeling policies defined in RFC 3023 should be followed and an XML declaration should be present. The "application/xml" media type is most appropriate for general XML; if a new media type is expected, it should be registered. URI registration. There is an ongoing effort to create a URN namespace explicitly for defining URIs for namespace names and other URI-designated protocol elements for use within IETF standards track documents; it might also establish IETF policy for such use.

Being text-based, protocols built with XML face significant threats, including unintended disclosure, modification, and replay. Simple passive attacks, such as packet sniffing, allow an attacker to capture and view information intended for someone else. Captured data can be modified and replayed to the original intended recipient, with the recipient having no way to know that the information has been compromised, detect modifications, be assured of the sender's identity, or to confirm which protocol instance is legitimate. Several security service options are available mitigate these risks. Though XML does not include any built-in security services, other protocols and protocol layers provide services that can be used to protect XML protocols. XML encryption provides privacy services to prevent unintended disclosure. Canonical XML XML digital signatures provide integrity services to detect modification and authentication services to confirm the identity of the data source. Other IETF security protocols (e.g., the Transport Layer Security (TLS) protocol ) are also available to protect data and service endpoints as appropriate. Given the lack of security services in XML, it is imperative that protocol specifications REQUIRE additional security services to counter common threats and attacks; the specific required services will depend on the protocol's threat model.

The authors would like to thank the following people who have provided significant contributions to the development of this document: Josh Cohen and Andrew Newton.