[This local archive copy is from the official and canonical URL, http://www.codebits.com/wddx/white_paper.cfm, 1998-09-28; please refer to the canonical source document if possible.]
Explosive adoption of the Web as a business computing platform continues at a furious pace. Every day, tens of thousands of companies and organizations are deploying more and more substantial portions of their core operations on the Web. Internally, every organization has seen the mushrooming growth of Web servers that act as content and publishing vehicles for information exchange and collaboration. Over a 100 million end-users, who average 14 hours of Web use a week, are reshaping the very fabric of how companies engage society and the economy.
The opportunity to recast society and the world-wide economy sits before us, and in fact, is unfolding now through the ongoing innovative adoption of the Web by cutting-edge companies and consumers. No one doubts the economic revolution underway on the Internet.
Fewer people, however, understand the implications of the infrastructure and platforms that are unfolding at an equally unparalleled pace. Organizations worldwide are rushing to deploy content and services on the Web. These mostly one-off applications are deployed on Intranets for worker collaboration, or business system automation, on Extranets and the Internet as frameworks to serve partners and customers. What companies don't realize in their efforts is that for every Web server they deploy, and every database or application connected to this Web, they are increasing the overall value of the global network.
Metcalf's law suggests that the value of a network is a direct function of the number of nodes connected to that network. An apt observation, though rarely considered in Web deployment considerations, this is at the center of the global opportunity created by the Web. For in every Web server and application deployed, every other node on the network now has the potential to interact with and leverage that data and service.
The value of the Internet is a direct function of the actually deployed data and applications on the network, and the leverage of that data to serve the networks of relationships that the Web enables. At a concrete level, this means that a company's public web site is now a portal into its critical customer service and inventory management systems; that its suppliers and customers can now leverage that information in a way that was never possible before global connectivity and the ubiquitous, open platform provided by the Web.
And this is at the crux of the problem in how organizations are approaching the opportunity of the Web today. Most organizations view the Web and the servers they deploy from as stand-alone systems aimed at serving a fixed customer or employee base. The data they deploy is for the use and consumption of end-users, strictly. In this manner, they view their 'node' on the network as having fixed value for a given user base, as opposed to adding to the overall value of the network. Data and content and services on the Web are structured to serve fixed user-bases, not the rest of the network and other applications.
A good example is the FedEx web site. This site has been perceived and heralded as one of the first sites to embrace the Web as a commerce medium. At first a very simple application, the FedEx site showed that you could provide innovative customer service by providing customers real-time access to shipping information through a Web browser. No doubt, a huge improvement over first-generation publishing oriented sites, this approach still falls victim to the 'Web as a single node' problem identified above where the function of the Web site is to serve end-users alone, and in particular through a fixed and static application built into the Web site.
Embracing the Web suggested by Metcalf's law might have led (and still might lead!) FedEx to approach the problem in a different fashion. Let's take the example further to illustrate.
While clearly the FedEx customers include consumers and businesses receiving packages (and hence the need for a web-based customer service application), their true customers (i.e. those that are paying for FedEx service) are corporations that are sending packages through FedEx. In fact, the vast majority of their business is from corporations who pay for FedEx to pick-up and deliver packages. These customers, however, are not interested in an online service that allows their customers to track packages. While that approach is interesting and provides overall value, these companies could more fully benefit if their own order processing and fulfillment systems had a more direct way to interact with the FedEx systems, automating package pricing, pick-up requests, and billing.
This next-generation approach embraces Metcalf's law, and moves a Web site from being a strictly end-user application to being a back-end service that custom business applications residing anywhere on the network can leverage. This is what enables the value of the network to increase as nodes are added. Now, an online retailer providing an e-commerce service to customers can leverage the FedEx website as a distributed information service that compliments and expands the capability of their own node on the network.
Cooperating nodes on the Web weave a fabric of information, the value of which far exceeds the value of the information contained in any single node. Once this is understood, it becomes clear that the hundreds of thousands of web sites (Intranet/Internet/Extranet) deployed by companies worldwide are actually the fabric of a much broader new Internet economy, where companies create rich, distributed applications built on new value and service networks.
The Web infrastructure in place today was clearly not built to support this type of broadly distributed information and commercial exchange. The vast majority of Web sites today remain hard-coded, static content sites, with no semblance of structured data or interfaces that could enable other nodes on the network to leverage that data's value.
Furthermore, this lack of structure in and interfaces with the data and applications on the Web today has resulted in increasing complexity in the presentation of content in Web browsers. In some ways a paradox, while the capabilities of Web browsers have grown dramatically in the past two years, the approach taken to managing and deploying web content has not kept up with these capabilities. Updating site layout and interactive behavior has become a mammoth task for installed static Web sites, as there is no logical separation of data or content from the actual presentation of that content to end-users.
Browsers have become true, interactive platforms, evolving into more of an operating system than a publishing medium. As nodes on the network, their ability to work with structured data continues to increase, much in the same way that Web servers have evolved into Web application servers. Yet, even as Web application servers emerge and play a stronger role, their ability to provide data and services to increasingly capable Web clients is hampered by the lack of a standard way to exchange structured data between Web application servers and Web clients.
While significant progress has been made by moving from static sites to dynamic pages that drive JavaScript-enabled clients, this model still does not enable corporations to expose their data and services more generically to other applications on the network. Browser applications cannot easily interact with proprietary server applications, and server applications cannot easily leverage data and services running on other Web-server based systems.
All of this shapes the crisis of web applications we are facing today. We have an installed, rapidly growing network of nodes with increasing sophistication and capabilities, yet we are unable to leverage that rich information source because no framework exists to easily share data among these systems.
Organically evolving over the past five years, the Web platform has moved from being a mere document sharing system to being a distributed, global application platform for running core business systems in organizations worldwide. Any approach taken to address the emerging crisis of web applications must embrace this emergent platform, accepting its core components and capabilities, as well as its limitations. While it is tempting to look for a replacement technology built from scratch that does not have the Web=92s weaknesses, it is far more practical to look for an approach that evolves and embraces, but does not attempt to discard, the established Web platform.
The Web is by nature heterogeneous. It consists of a myriad of browsers and servers deployed on any number of operating systems and vendor implementations. As an application platform, it weaves together many types of content and programming languages. No single vendor controls it, and applications deployed on it are assumed to have few boundaries. (As an achievement in social anarchy, it appears to have no peers.)
Yet, the Web as an application platform has at last started to take a common and predictable shape. By surveying the landscape of Web applications (i.e. any Web site that uses client-side or server-side scripting and application logic), a consistent model of applications has emerged. Most corporations have embraced and are actively building applications using this model.
Dubbed 'Page-based applications', these applications use server-side data access and application logic to dynamically build pages that contain HTML, JavaScript and Cascading Style Sheets (CSS) to create dynamic user interfaces. On the server, Web applications are predominantly created using programming environments such as Perl, ColdFusion and Active Server Pages. (At times you'll find server-side applications written using Java or C++, though these are limited due to the complexity of development and the poor integration of the languages with the page and content centric nature of Web applications.)
On the client, the vast majority of applications are implemented using a combination of HTML 3.2 and JavaScript 1.1. Most applications are deployed to that subset of functionality which is supported both by Netscape and Microsoft browsers. Increasingly, applications are using JavaScript 1.2 and CSS, though there is still low adoption of these given browser compatibility issues. A very small percentage of applications use client-side Java because of its poor platform compatibility and significant performance and stability problems. An even smaller percentage of applications use client-side ActiveX, given the single-platform focus of this technology and its poor integration with HTML.
It might be noted that the ubiquitous Web application platform is little more than the evolution of a range of Web scripting and content languages that share some common properties: 1) they are straightforward and easy to use and learn, 2) they are mostly typeless, ASCII-based environments, and 3) they seamlessly integrate into the existing HTTP-based infrastructure used by Web browsers and servers. In short, the established Web application platform is a pragmatist=92s platform, not a single homogeneous technology and architecture.
Web applications are built with this architecture and these technologies because they are immediately available and they work within the severe constraints of the Internet: high-latency networks, low-bandwidth speeds, a huge range of client and server operating systems, and the ability to be leveraged across broad, globally distributed networks. Having emerged from a document publishing system (HTML/HTTP 1.0), the Web has had to cope with a huge variety of shifts, all of which have continued to accommodate the underlying vision of the Web as an application platform.
Attempts to impose "richer" architectures on the Web that purported to solve major architectural limitations have consistently failed. Whether Plug-Ins, ActiveX or Java as client-side executable platforms, or Shockwave and VRML, which imposed huge client-side overhead while offering questionable concrete value, these systems have failed to achieve broad adoption because of poor integration with the Web platform as it currently works.
Indeed, pragmatism has won and will continue to win the Web platform war.
As noted earlier, the Web is today undergoing another major shift and evolution. The shift is from viewing the installed base of browsers and servers as stand alone nodes, to viewing this broad network as a set of distributed services that can be leveraged by each other. In particular, the dramatic increase in the capabilities of Web browsers, through JavaScript and Dynamic HTML (DHTML), combined with the surging importance of Web application servers has finally woken the industry and customers to the fact that a sea-change is occurring in how much this platform can accomplish.
The two major shifts that are driving the next phase of the Web platform evolution are the increasing role of DHTML and Web clients for handling applications, and the changing role of Web servers to being distributed application servers that expose content and programmatic interfaces to other servers on the network.
After a number of false-starts with Plug-Ins, ActiveX and Java, the browser platform vendors have finally settled into Dynamic HTML (the combination of HTML, CSS and JavaScript) as the core model for client-side applications in the Web environment. With this standardization, developers are now realizing that substantial benefits can be had if data processing and application behavior is offloaded from servers to these more intelligent and interactive clients, moving browsers into a role and capacity that is likely to surpass Windows and like APIs for graphical user interfaces.
Traditional capabilities found in legacy client/server platforms, such as databinding and rich, interactive forms behavior, is now quite possible in browser clients. This capability has excited software ISVs and corporate developers, as finally they can fully embrace the Web platform as the new model of software development and distribution.
With this shift, the overall end-user experience and the role that desktop computers can play in the Web platform and global network is increasing dramatically.
Even with this shift towards richer client capabilities, the pragmatist web platform is also evolving to support a set of services on the Web server that drive both the client behavior and distributed, server-to-server capabilities.
Web application servers have emerged as the foundation of how companies build valuable applications and sites on the Web. As the host of application logic, user and personalization data, and most importantly, data and back-end systems integration, these application servers are the critical counterpart to the growing role of web clients.
In fact, the capabilities of web clients and their ability to participate as true 'citizens of the web community,=92 will increasingly rely on the ability of Web application servers to interact and provide structured data to those clients.
And, as companies move to use the Web as a platform to drive business-to-business commerce and collaboration, the ability of application servers to communicate and share data with each other across the Internet and Extranets is becoming a core requirement.
As identified above, two major problems exist for this next evolution of the Web: 1) Rich web-clients and web application servers have no structured way to exchange data, and 2) web application servers lack any common means to exchange data between servers over HTTP. The lack of a structured data exchange format is the missing link in this emergent platform.
Using our earlier FedEx example, we can see how this problem manifests itself. In the end-user example, it would be extremely costly for FedEx to have its server dynamically generate a set of pricing options data for a given package and send that data to the client, where the browser would allow the user to choose between varying options, examining different prices based on configuration options. Assuming that the FedEx system was implemented using server-side Perl, ColdFusion or C/C++, these server-based systems have no easy way to move the data about pricing options to the JavaScript based clients. In short, no standard mechanism exists to move data objects between these disparate, yet ubiquitous, web programming environments. Hence, increasing the value of the Web end-user experience has been hampered by this missing link.
On the server-side, the problem is magnified. Imagine that FedEx wanted to expose a set of URLs on its server to other corporations who are deploying e-commerce sites and servers on the Web. These other corporations want to be able to query the FedEx server for real-time pricing and package shipment status information over HTTP. FedEx can=92t assume that these incoming queries are coming from any given platform; different corporations are deploying e-commerce servers built on Perl, Java, ColdFusion, ASP, and JavaScript using Web application servers. Second, even if FedEx knew what kind of application was requesting its data, it would have no standard and predictable way to exchange that data. Again, the ability of FedEx to construct a next-generation business model that leverages the new Internet economy has been hampered by this missing link in the Web platform.
Over the past year, the Web industry has begun to usher in a new model for exchanging data over the Web. The Extensible Markup Language, or XML, has emerged as a foundation for expressing and exchanging data of almost any form over the native, HTTP-based Web environment.
XML promises to fill the missing link in the Web platform's future by providing a common glue for data exchange between disparate clients and servers on the Internet. XML embraces the uniqueness of the Web platform; it is based on ASCII, can be easily transported over HTTP, is network, language and platform independent, and is flexible and extensible such that it can scale to solve nearly any data exchange problem.
Two critical problems that XML potentially solves are 1) providing a framework to separate data from presentation or layout, and 2) providing a grammar that will allow Web application servers to share and exchange data with other servers across the network. More generally, XML promises to define common information formats across nearly every major computing endeavor.
As a language environment, XML embraces on the notion that computer languages must evolve to be both human readable and writable, as well as machine readable and writable. XML enables declarative and human understandable data and languages, opening doors to richer forms of abstraction in computer programming and components.
With the surging interest in XML, there is abundant speculation as to what its role will be in the emerging Web and computing landscape. Over the past year, we've begun to see a number of views emerge on the critical role that XML will play in transforming the Internet and data exchange standards.
One major view is that XML and the varying forms of XML data and vocabularies that emerge will act as the future of how almost all data is represented and programmed. In this view, XML data becomes a logical and physical storage format for data, replacing even SQL servers as the primary way to represent and access data. Developers define data structures (relational, hierarchical, networked, object-based, etc.) that are then represented in XML formats. Accessing and manipulating that data then requires programming what is referred to as a Document Object Model, or DOM. The DOM is the generic API for accessing and manipulating XML data, much like SQL and ODBC are the generic means for accessing relational data today.
In this new XML world, developers no longer think in terms of tables and joins, and SQL, they think in terms of XML vocabularies and DOM programming. We can see early examples of this with Microsoft's IE4 XML-databinding features. With this, a developer would use a server page, created with ASP or ColdFusion, for example, to query a standard database and dynamically build an XML format for that data, returning it to the client where that XML structure would be 'bound' to an HTML table or other page elements. In this world, the developer must create an XML format for their existing relational data. Instead of just passing the client a recordset, it would create tags and tag attributes containing data that maps to the relational structure.
Corresponding to this view is the idea that vendor consortia would work together to define common XML formats for cross-organizational problems. For instance, health care and insurance companies would work together to define a Patient Markup Language that defines the data and structures associated with a patient, allowing patient data to move easily between relevant business systems on the network. It might even assume that two companies working together to exchange invoice data would establish an XML format to exchange structured invoice data, shielding each other=92s systems from proprietary formats or platforms.
Another major view -- and not necessarily a competing one -- is that XML provides an underlying glue to application-to-application data exchange, in essence become a Web-native middleware environment for distributed applications. However, in this view XML would be more of a transport protocol for object data than an environment of specific formats for given problem domains.
For example, using XML as middleware would allow a client program --- say a JavaScript browser -- to invoke a server application -- say a ColdFusion page -- and request a set of data or recordsets. ColdFusion would generate relevant recordsets, transform into XML, and pass back to the client, where the client would transform that data back into a native JavaScript format. This would allow cross-language, network-based data exchange. The utility of XML as a data exchange protocol example to the server, where a ColdFusion server could invoke a service on another remote server, say running Java servlets or Perl, and request a set of data -- the other server environment would generate data and transform it into this XML format and pass it to ColdFusion, which would then translate into native ColdFusion data objects.
In the 'XML as middleware' model, the use and visibility of XML becomes transparent to the developer. Instead of defining custom XML formats and programming an XML DOM, the developer thinks and programs in the same high-level constructs of her native development language -- objects, arrays, recordsets, string variables, etc.
As one might have noted, the differences between these two models are potentially substantial. The crucial difference is in what the model imposes on the developer and corporation in terms of new data formats, architecture and programming approaches. The purist approach to XML Data assumes that corporations and developers will rapidly move to store and expose all of their content and data in custom XML-based formats. Secondarily, it assumes that the primary data access and manipulation language will shift from object and SQL based to DOM based programming.
This shift is both major and potentially unnecessary for the substantial majority of applications. SQL-based vocabularies for data access and management have been adopted widely because of SQL=92s simplicity and declarative nature. Likewise, relational database design fundamentals are broadly understood and easily created using products like Access and database CASE tools such as Power Designer.
Corporations have made billions of dollars of investments in the relational database model, and have built programming arsenals to support handling these systems. It is extremely unlikely that there will be a total shift away from them overnight, given both the current skills and infrastructure investment and the lack of replacement skills and infrastructure.
Furthermore, it assumes that developers will want to shift from a data structure model based on simple constructs such as variables, arrays and recordsets, to one based on the DOM, and that they are willing and able to impose a custom translation layer -- one that requires a new model of thinking about and representing data -- between their existing native database systems (most likely SQL-based) and other client and server systems.
Finally, the Purist XML Data approach is overly optimistic about different vendor-neutral standards and bodies' ability to define shared XML formats for data exchange. While clearly a requirement in the long-run, corporations wanting to do business-to-business integration over the Internet will need a more accessible and rapid approach to structured data exchange. For example, an HMO seeking to share data with regional health care providers over the Web can't wait to define a shared XML vocabulary, let alone wait for health care standards bodies to define XML formats for patient and billing information.
Clearly, both the XML Data Purist and XML Middleware approaches will both play important roles, though it seems that existing models and approaches to data storage, access and programming, combined with the unique platform requirements of the Web will drive companies towards a more pragmatist approach that shields developers from a large base of required knowledge of underlying XML data structures.
The real world of Web development and Web applications demands a pragmatist=92s approach to XML; an approach that can open up the range of opportunities that XML affords without imposing the intellectual and technical shift that a purist approach would require.
Returning to our earlier discussion of the evolving Web platform landscape, it is clear that an XML-based architecture is required to take browser-based applications to another level, and more importantly to begin globally exposing Web application servers as general distributed services and interfaces to the rest of the Web.
XML carries the same syntax and language advantages that HTML and CFML bring to the Web platform: high-level, declarative, human readable and human writable languages. As a system for encapsulating data and logic, and bringing abstraction into human computer programming, XML portends to be a breakthrough. With this in mind, it is clear that Web developers are seeking the use of XML for client and server-side component encapsulation and browser extensibility.
With this approach, XML-based tags and components encapsulate ranges of client and server side scripts, layout code, and data into intelligent Web components. This would allow a development team to create XML components that combine CSS, HTML and JavaScript which are in turn used by page designers who are thinking and working at a higher-level of abstraction. XML becomes the language and syntax glue that makes this possible.
Allaire has been pioneering this approach for the past two years through the use of ColdFusion Extensions, or CFXs. These XML-based components can encapsulate data, logic, and layout, whether on the client or server-side of the Web platform environment. Over 500 third-party tags are available which embrace this architecture. More recently, Microsoft has introduced XML Behaviors in IE5, and Netscape plans to introduce XML-based ActionSheets. Both of these provide a client-side mechanism for using XML-based components to encapsulate browser behavior. In addition, simply binding XML tags to CSS layout definitions will provide publishing-oriented developers with a cleaner, more reusable architecture.
Clearly, XML as a Web-centric language and component architecture will continue to evolve and grow, as pragmatist approaches continues to win the Web platform war.
Even as XML blazes a path as a language and component architecture, this still does not explicitly address the broader and more important need of finding a model and use for XML as a distributed data infrastructure. What would a pragmatist approach to XML data exchange look like?
First, a pragmatist approach to XML data exchange would need to acknowledge and embrace the real-world environments driving the Web platform. In particular, the approach would need to easily accommodate and interoperate within key programming environments in use on Web client and server platforms. These include ECMAScript (a.k.a. JavaScript), ColdFusion, Perl, ASP, and Java, and at times even Windows clients implemented using ActiveX.
Second, it would need to maintain the core data storage and manipulation environments driving corporate applications today. What this means in particular is that it would need to operate in the world of relational databases -- SQL, recordsets -- and the core data structures used in the above languages -- associative arrays, lists, recordsets, strings, etc.
Third, it would need to be transparent to its users. Developers wanting to exchange structured data across the Web, whether from servers to browsers and back, or from servers to other servers, should not have to think about XML parsing, data access and programming. Instead, the developers should work with data in their language and platform environments, assuming transparent exchange across the network.
Finally, it would need to embrace a simple design that worked across a lowest common denominator of data structures used in the Web platform. Relating to the first goal of interoperating between key Web languages, a lowest common denominator approach would realize that the majority of Web programming and applications takes place in higher-level scripting environments such as Perl, ColdFusion and JavaScript, not lower-level object-oriented languages such as Java and C++. It is critical that data exchange on the Web is modeled on the same principals that have made these languages successful -- loose typing, interpreted, relatively easy to learn, ASCII and HTTP friendly, etc.
Clearly there are models and applications where a less pragmatist approach is required. In particular, it will become important to develop and support XML vocabularies for common data formats and industries. These custom XML vocabularies should be used on an as needed basis, as opposed to being the core of how people take advantage of HTTP-based structured data exchange.
For instance, XML vocabularies to represent EDI related data could become important for certain classes of business-to-business data exchange, and having common formats for these matters would be beneficial. Or, for specific classes of information storage and retrieval where marked-up data becomes important for shared search capabilities, one can imagine successful common XML formats.
However, for distributed application-to-application data exchange and communication -- XML middleware -- it's clear that the use and visibility of XML should be both transparent to and integrated with basic programming data structures used in key web application environments.
A second outstanding issue is the desire and need to use XML middleware to supplement, if not supplant, traditional distributed object protocols such as DCOM and Corba. Clearly, for binary oriented object protocols and languages (such as Java and C++) there will be benefits to using XML middleware for distributed object programming. Efforts such as Microsoft's SOAP, or DataChannel's WebBroker will be important in this regard.
However, one should note that these solutions fail to directly address the more pragmatic need facing the actual environments driving the Web platform. Most distributed Web applications are a) page-based and not binary or object based, b) are built on non-object-oriented languages such as Perl, JavaScript and ColdFusion, and c) require a data exchange model that is more stateless in nature -- e.g. passing data back and forth between clients and servers, as opposed to using an RPC-style distributed programming layer.
As a leader in the Web application development platform arena, Allaire is addressing the need for an XML-based distributed computing model with the introduction of the Web Distributed Data Exchange (WDDX). WDDX is an open, freely available XML vocabulary that enables distributed data exchange across the dominant platforms that are driving the Web today.
Grounded in many of the beliefs and assumptions outlined above, WDDX seeks to provide a lowest-common denominator approach to distributed data on the Web using XML. WDDX embraces the actual Web platform environments -- JavaScript, Perl, ASP/COM, ColdFusion, and Java -- through a language neutral data exchange protocol that is virtually transparent to implement, and does not require developers to know anything about or implement XML programming.
WDDX uses XML to store and exchange data across the Web using HTTP. Different languages that implement WDDX need only implement simple serialization and deserialization stubs that transform language native data objects into WDDX, and vice-versa. This process provides a universal data format that is common to dominant Web programming languages. Because it is based on XML, this data structure can be easily transferred and accessed over HTTP, though WDDX can be shared and used in any text-based environment, such as flat-files, a relational database, etc.
WDDX embraces a common subset of data structures used in contemporary Web programming environments. In particular, WDDX supports the exchange of structures (a.k.a. associative arrays, data dictionaries, or hash tables), arrays, recordsets, lists, strings, date/times, and boolean values. WDDX is based on the same ISO standards as XML, ensuring international time/date, numeric and currency compatibility.
WDDX also relies on dynamic expression evaluation and data introspection capabilities in languages such as JavaScript, Perl, and ColdFusion to facilitate transparency in data exchange.
A simple example of WDDX in action will help clarify how it works. Imagine wanting to query a database on a server and move the recordset data to a browser client, where using JavaScript and DHTML you would render and iterate over the recordset.
First, using ColdFusion you would select a set of data:
<CFQUERY Name="Employees" Datasource="EmployeeDB"> Select FirstName, LastName, Email From Employees </CFQUERY>
Now, using a simple WDDX transformation routine you can move that data from being a native ColdFusion query object on the server into being a native JavaScript object on the client.
<CFWDDX Action = "CFML2JS" Input = "Employees" TopLevelVariable = "EmployeeData" >
Using HTTP to transfer this content back to the browser, the data object is automatically available to JavaScript as soon as the page loads. All of the records from the query are available as part of a native JavaScript array. That could be rendered, manipulated, and then transferred back to the server, using a JavaScript routine to transform back into WDDX and then on the server into native ColdFusion objects.
Behind the scenes, the WDDX implementation serialized the ColdFusion recordset into a WDDX packet stored in an XML format. The formatted WDDX data is included below:
<wddxPacket version='0.9'>
<header></header>
<data>
<recordset rowCount='15' fieldNames='FIRSTNAME,LASTNAME,EMAIL'>
<field name='FIRSTNAME'>
<string>Carolynn</string>
<string>Dave</string>
<string>Linda</string>
<string>Aaron</string>
<string>Peter</string>
<string>Linda</string>
<string>Peter</string>
<string>Richard</string>
<string>Jenna</string>
<string>Erica</string>
<string>Francisco</string>
<string>Michelle</string>
<string>Dominique</string>
<string>Walter</string>
<string>David</string>
</field>
<field name='LASTNAME'>
<string>Peterson</string>
<string>Heartsdale</string>
<string>Stewart</string>
<string>Smith</string>
<string>Barken</string>
<string>Jennings</string>
<string>Jacobson</string>
<string>Frankin</string>
<string>Smith</string>
<string>Manley</string>
<string>Cabrerra</string>
<string>Leary</string>
<string>Branden</string>
<string>Reardon</string>
<string>Barnes</string>
</field>
<field name='EMAIL'>
<string>CPETERSON</string>
<string>FHEARTSDALE</string>
<string>LSTEWART</string>
<string>ASMITH</string>
<string>PBARKEN</string>
<string>LJENNINGS</string>
<string>PJACOBSON</string>
<string>RFRANKLIN</string>
<string>JSMITH</string>
<string>EMANLEY</string>
<string>FCABRERRA</string>
<string>MLEARY</string>
<string>DBRANDEN</string>
<string>WREADON</string>
<string>DBARNES</string>
</field>
</recordset>
</data>
</wddxPacket>
The WDDX packet was then deserialzed into this JavaScript expression:
EmployeeData=new WddxRecordset(); _t2=new Array(); _t2[0]="Carolynn";_t2[1]="Dave";_t2[2]="Linda"; _t2[3]="Aaron";_t2[4]="Peter";_t2[5]="Linda"; _t2[6]="Peter";_t2[7]="Richard";_t2[8]="Jenna"; _t2[9]="Erica";_t2[10]="Francisco";_t2[11]="Michelle"; _t2[12]="Dominique";_t2[13]="Walter";_t2[14]="David"; EmployeeData.firstname=_t2; _t2=new Array(); _t2[0]="Peterson";_t2[1]="Heartsdale";_t2[2]="Stewart"; _t2[3]="Smith";_t2[4]="Barken";_t2[5]="Jennings"; _t2[6]="Jacobson";_t2[7]="Frankin";_t2[8]="Smith"; _t2[9]="Manley";_t2[10]="Cabrerra";_t2[11]="Leary"; _t2[12]="Branden";_t2[13]="Reardon";_t2[14]="Barnes"; EmployeeData.lastname=_t2; _t2=new Array();_t2[0]="CPETERSON";_t2[1]="FHEARTSDALE"; _t2[2]="LSTEWART";_t2[3]="ASMITH";_t2[4]="PBARKEN"; _t2[5]="LJENNINGS";_t2[6]="PJACOBSON";_t2[7]="RFRANKLIN"; _t2[8]="JSMITH";_t2[9]="EMANLEY";_t2[10]="FCABRERRA"; _t2[11]="MLEARY";_t2[12]="DBRANDEN";_t2[13]="WREADON"; _t2[14]="DBARNES"; EmployeeData.email=_t2; _t0=null;_t1=null;_t2=null;
The JavaScript expression was then dynamically evaluated into a JavaScript object named 'EmployeeData'. The developer could then operate against that recordset in JavaScript natively. All of this was accomplished with a single tag in the server page.
The behind the scenes aspects to this are very important. While apparently a complex process, it has been reduced to a single, transparent action taken on the client or server. XML storage, parsing and interpretation is hidden entirely from the developer, as they work at the familiar programmatic level in their application data. By declaring the CFML-to-JavaScript transformation on the server, the developer knows that the data object will automatically be available on the client when the page loads.
WDDX has the potential to open up and ease the development of complex, distributed Web applications. With its XML foundation, WDDX has broad applications in areas such as client-side databinding, server-to-server distributed data, intelligent web agents, and in enhancing DHTML user-interface controls.
A common method of accessing and using data in Windows and client/server database applications, databinding has been difficult, if not impossible, in the Web application world. Mostly because Web applications use dynamic pages and HTML over a stateless infrastructure (HTTP), it has been difficult to provide live databound user interfaces in page-based applications.
With the advent of JavaScript 1.x and richer client-side data handling, it become clear that the DHTML model combined with some form of databinding would be very desirable, and would go a long way in bringing Web applications into the mainstream of corporate applications.
The advantages of databinding are substantial -- it allows for more response user-interfaces without frequent and slow server roundtrips, and provides application developers with high-level data access functions on the client-side.
IE4 introduced XML-databinding as a first step in this direction. Aside from the fact that it is a proprietary implementation, it also suffers from the inadequacies discussed around the Purist XML Data worldview. IE4 XML-databinding requires that developers transform their data structures into native XML native structure, modeling their data in custom XML vocabularies, and having familiarity with parsing and scripting an XML DOM to be truly productive.
As a pragmatists approach to XML data, WDDX introduces a transparent means to databinding in browser-based environments. Using WDDX, a developer can retrieve a recordset on a server using Perl, CF, ASP, or from any datasource those systems can access data from, transform it into WDDX, and make it available as native client-side data objects.
Because JavaScript does not natively have the concept of a recordset object, Allaire introduced a JavaScript-based recordset object construct that is included as part of the freely available 'wddx.js' package. This allows the developer to iterate over columns and column names, add and remove cells and rows, etc.
A basic DHTML databinding example would work as follows:
Overall, this approach to databinding is more straightforward to implement, supports a broader base of browser and OS platforms, and does not require the developer to change their fundamental data access and manipulation approach used in their Web programming.
Fundamentally similar to the problem of browser-based databinding, WDDX potentially enables richer, more predictable use of DHTML controls in Web applications. DHTML controls are JavaScript/CSS/HTML abstractions that provide richer user-interface behaviors to Web applications. Examples could include menuing and navigation controls, such as tree controls, multi-form inputs, like wizards, and more complex DHTML applications such as interactive games.
To date, creating reusable DHTML controls has been difficult both because the code metaphor to encapsulate DHTML has been either proprietary or non-existent, and because the inputs for DHTML controls are unstructured and unpredictable. On the encapsulation side, XML-based component approaches such as IE5 Behaviors and ColdFusion CFXs go a long way to improving encapsulation and reusability in controls. However, providing standard inputs into controls is problematic given complexities in how Web applications are deployed.
For instance, it would be extremely convenient for developers of JavaScript or DHTML controls to redistribute those controls with a required set of inputs. The challenge, however, comes from the fact that 'inputs' into DHTML controls come from dynamic page servers and server-based scripting environments. Hence, there needs to be a common standard for server-based environments to expose and provide structured data to DHTML controls with common inputs.
WDDX provides an architecture that can combine the benefits of DHTML with any Web application server environment. For instance, a DHTML menu control could expect a set of WDDX compliant inputs, such as a set of named lists or arrays, which would be dynamically created and delivered from the server using ColdFusion, ASP or Perl.
As discussed earlier, one of the more exciting aspects of the emerging Web landscape is leveraging server-to-server data exchange to create distributed Web applications that span across the Internet and Extranets. Whether for replicating data, centralizing data serving, or doing business-to-business exchange, XML-based server-to-server data exchange will be critical to the next generation of web applications.
WDDX works as transparently with server-based communications as with server to browser communications. Using HTTP as the communications layer, either and of a data exchange need only have the WDDX stub to map the data into the native environment. What this means is that an application could leverage data and services running on remote servers using any Web platform. For instance, an application written in Perl could invoke a ColdFusion server of HTTP and have a recordset returned as WDDX, and then locally work with that data as if the query happened on the local server.
This model of computing opens up a huge new range of business applications. For example, Web content providers can now expose 'intelligent datasource URLs' that expose WDDX interfaces to dynamic applications running anywhere on the network. For instance, 'Weather.com' could expose a set of URLs that returned a WDDX packet with a structure of weather data based on a custom search. The 'client' application could then use that data locally as native language objects.
Early efforts using ColdFusion CFXs and HTTP to build intelligent agents are great examples of this; there's no doubt, however, that WDDX will take these efforts to another level. In fact, it may very well be the case that CFXs that wrap complex HTTP, WDDX and DHTML logic will be the primary way that the general Web development community benefits from this infrastructure, as most developers will not be working at the layer of distributed data programming and component creation.
Business models will emerge to support distributed, aggregated data providers who expose URLs to Web agents that invoke those services from any Web programming environment. The model extends logically into Affiliate and Syndication networks emerging on the Web, where the value-added data, products and information of a Web site are made available to 'Site Partners' via affiliate programs. These site partners leverage distributed data, and even invoke remote product orders and promotions, via these syndication networks. A framework as cross-platform and cross-language as WDDX and XML is a requirement for such a use of the Web to flower.
Allaire is releasing the WDDX DTD (Appendix I) and related language implementations freely to the Web community. As part of this, Allaire and industry partners are involved in a range of efforts to create native implementations on popular Web development and deployment platforms. The following section provides brief overviews of the work that has already been done, and references to other detailed information on WDDX implementations.
With ColdFusion Server 4.0, we've introduced substantial support for WDDX 1.0. At the center of the support is a new tag, <CFWDDX>, which provides support for WDDX serialization and deserialization. With this tag, developers can pass any ColdFusion variable, query, or structure into WDDX, and likewise can receive any WDDX packet and transform it into local ColdFusion variables.
In addition to WDDX-to-CFML and CFML-to-WDDX, ColdFusion 4 also provides a direct translation between CFML and JavaScript. With this, you can easily and transparently transform any ColdFusion variables and queries into the equivalent JavaScript objects on the client.
One minor and additional note is required on the implementation. Currently, in the ColdFusion-to-JavaScript transformation, ColdFusion deserializes the WDDX packet into a JavaScript expression, which is then passed back to the client where it is evaluated into a live JavaScript object. This two-part server process a) eliminates additional processing requirements on the client, and b) makes the overall processing less cumbersome to implement. However, using a client-side XML parser (JavaScript, Java or ActiveX-based), a developer could easily parse a straight WDDX packet and transform that data into a dynamic JavaScript expression, much like what is done on the server.
In addition to these core features, ColdFusion 4 also provides a set of other features that strongly complement the base WDDX functionality. Of particular note is CFML 4.0's new support for Structures. Structures are a new data type akin to Associative Arrays in JavaScript, Dictionary Objects in ASP, and Hash Tables in Perl. This new data structure is fundamental to WDDX, and makes cross-language platform data exchange more powerful for ColdFusion. Structures can be created out of WDDX packets, or through the use of a new set of Struct manipulation functions.
ColdFusion 4 also includes rich functionality for server-to-server data exchange via HTTP. The CFHTTP tag, introduced with ColdFusion 3.x, provides an easy mechanism to pass WDDX data between ColdFusion and other servers. Finally, using standard ColdFusion connectivity capabilities it becomes extremely simple to move data from any ODBC, Oracle or Sybase database, LDAP directories and any COM or CORBA object into WDDX/XML to pass to other servers or to browsers for local processing. Through WDDX, CF4 enables XML to become the lingua franca of your Web data.
With ubiquitous adoption via major Web browsers, JavaScript is both an ideal and fundamental platform in the adoption of WDDX and XML. As the user interface portal for the Web, most data will ultimately surface up to the JavaScript browser. And, with the standardization of ECMAScript as the dominant client-side programming language for the Web, we can expect to see this language environment widely implemented in emerging device categories such as Web phones, PDAs, Web TVs and other lightweight devices.
Currently, JavaScript support for WDDX is based on the 'WDDX.JS' package, available freely from Allaire, and also distributed with ColdFusion 4, as well as via additional server-side support provide in both the CFWDDX tag and via the WDDX COM objects available from Allaire (See Note above on CFML2JS implementation).
WDDX.JS provides two major functions today. First, it provides a set of methods for serializing existing JavaScript arrays and objects into WDDX. Once serialized, the developer can use this WDDX packet however they want, though most common will be to post it to a server as a hidden form field or URL token.
Second, the package provides a set of objects for using recordsets in JavaScript. Since JavaScript itself does not provide any recordset creation or manipulation functionality, and since this is key to browser-based databinding, the package implements that support natively. The recordsets it creates and uses will transparently work with WDDX.
Currently, however, WDDX.JS does not implement or support Deserialization from WDDX to JavaScript. As noted above, this is automatically implemented and supported in the key server-side frameworks, ColdFusion and ASP, which has both its advantages and disadvantages. It is our intention for WDDX.JS to support native deserialization, and there are efforts underway to implement this using XParse, a JavaScript 1.1 XML parser available from www.jeremie.com.
Another popular development platform for Web applications, and more generally for Windows applications, is COM and its Web cousin, Active Server Pages or ASP. Both for integrating legacy Windows applications with Web application servers, and for building 100% Web-based applications with ASP, it's important to have this set of nodes on the network be conversant in WDDX.
The COM implementation of WDDX offers very similar services to those provided in ColdFusion. Through a set of COM objects, developers can easily Serialize and Deserialize WDDX to native COM objects. The package supports automatic translation of the following native COM data types: Boolean, Float, DATE, BSTR, and Variant Array. These map to WDDX Boolean, Float, DateTime, String, and Array strucutres. In addition, we've provided two additional data structures that are encapsulated in the provided COM package. These data strucutres map to WDDX Structures, and WDDX Recordsets.
What this means is that ASP can act as a dynamic data server for JavaScript clients that use WDDX for DHTML databinding. ADO recordsets generated by ASP can map to WDDX Recordsets and then be transparently used on the client using JavaScript recordsets. Likewise, ASP servers can now easily pass data to other application servers, including ColdFusion or other ASP servers.
In addition, the COM package provides a set of interfaces for deserializing WDDX into JavaScript expressions, which can then be dynamically evaluated on the browser, akin to the CFML2JS implementation discussed above. This allows ASP to automatically handle the ASP2JS implementation without requiring XML parsing on the client.
Equally important, however, is the fact that the WDDX.COM implementation can easily be used within non-Web environments; in particular, any application environment that supports COM, which now includes Visual Basic, Delphi, Java, and C/C++. This capabilities enables Windows and Java clients to use HTTP to integrate distributed data, and to communicate with Web application servers easily without changing or compromising the language and programming approach used in those environments.
As an example, Allaire has embraced WDDX as a framework for bridging our own Windows client-side applications with our Web application server. In particular, the Remote Debugger introduced in ColdFusion Studio 4.0 uses WDDX as the distributed data protocol for data exchange between our Windows client (written in Delphi and C++) and the ColdFusion Application Server (using C++ and CFML on the server). In this environment, when Studio runs a page from the client, the page is passed to the server for processing, including a WDDX packet that describes the breakpoints for the page. The server executes the ColdFusion page until a breakpoint, and then returns the state of the ColdFusion engine (variables, tag stack, watches) as a WDDX package, which is then rendered into the Debugger environment. In Delphi, the environment is calling against a COM interface.
Going forward, the COM implementation of WDDX requires additional enhancements. In particular, enhancements are required in the areas of binding native COM-centric data access models to WDDX recordsets. We anticipate improvements and work surrounding the automatic binding of ADO recordsets to WDDX recordsets, as opposed to the two-pass process that is required to move ADO to WDDX today. Likewise, with ASP specific abstractions such as the Dictionary Object, it would be nice to easily map them to WDDX Structures.
The other two major Web development and deployment platforms in use today are Perl and Java. While primarily in use on servers, Perl and Java are used by hundreds of thousands of developers and a still growing installed base of applications. As key nodes on the network of value that the Web has created, WDDX interfaces for these languages are key to the overall success of distributed Web applications.
Currently, there are independent efforts underway to implement WDDX as native Perl modules and Java classes. These packages will enable transparent use of WDDX in the same fashion outlined for both ColdFusion and COM/ASP. Key services provided by these platforms will include core serialization and deserialization to/from WDDX, dynamic JavaScript expression creation, for server-based applications generating client-side JavaScript objects, and a set of custom objects for mapping to WDDX Structures and Recordsets. Like COM and ADO, the Java implementation could also implement some additional degree of transparency between JDBC result-sets and WDDX Recordsets. Likewise with Perl Hash Tables and WDDX Structures.
As implementations details and documentation for these platforms becomes available, we will in turn make it available via the Allaire Website.
<!--
************************************************************************
WDDXData DTD:
Author: Simeon Simeonov (simeons@allaire.com)
$Header: $
Copyright (c) 1998 Allaire Corp. http://www.allaire.com
-->
<!--
************************************************************************
Introductory Notes:
What is WDDX:
WDDX stands for Web Distributed Data eXchange. WDDX is a
mechanism for exchanging complex data structures between programming
languages. It has been designed with web applications in mind. WDDX
consists of a language-independent representation of instantiated data
based on XML 1.0 (which is defined using this DTD) and a set of
serializer/deserializer modules for every language/technology that uses
WDDX.
The WDDX DTD:
The WDDX DTD can be used to validate WDDX packets. Packets are
representations of instantiated data structures in programming
languages. The following is an example of a WDDX packet:
<!DOCTYPE wddxPacket SYSTEM "wddx.dtd">
<wddxPacket version='0.9'>
<header/>
<data>
<struct>
<var name='s'>
<string>a string</string>
</var>
<var name='n'>
<number>-12.456</number>
</var>
<var name='d'>
<dateTime>1998-06-12T04:32:12</dateTime>
</var>
<var name='b'>
<boolean value='true'/>
</var>
<var name='a'>
<array length='2'>
<number>10</number>
<string>second element</string>
</array>
</var>
<var name='obj'>
<struct>
<var name='s'>
<string>a string</string>
</var>
<var name='n'>
<number>-12.456</number>
</var>
</struct>
</var>
</struct><
</data>
</wddxPacket>
It defines a root level object which is a structure (also known
as an associative array) of six properties:
- s which is the string 'a string',
- n which is the number -12.456,
- d which is the date-time value June 12, 1998 4:32:12am,
- b which is the boolean value true,
- a which is an array of two elements (10 and 'second element'),
and
- obj which is a structure with two properties s and n
WDDX supports the following basic data types:
boolean (true/false), number, date-time, and string.
Numbers are internally represented with floating point numbers.
Because of differences between WDDX-enabled languages, the range of
numbers has been restricted to 3.4E+/-38. The precision has been
restricted to 7 digits after the decimal point. These requirements are
consistent with a 4-byte floating point representation.
Date-time values are encoded according to the full form of
ISO8601. Timezone information will be successfully parsed and used to
convert to a local data-time value.
Strings can be of arbitrary length and must not contain embedded
nulls.
WDDX supports the following complex data types:
arrays, structures, and recordsets.
Arrays are integer-indexed collections of objects of arbitrary
type. The starting index value is usually 0 with the notable exception
of CFML whose arrays have an initial index value of 1. Because of
these differences working with array indices can lead to non-portable
data.
Structures are string-indexed collections of object of arbitrary
type. In many languages they are known as associative arrays. To satisfy
the variable naming rules of a variety of languages, property names must
satisfy the following regular expression [_A-Za-z][_0-9A-Za-z]*. That
is, they must start with an underscore or a letter and be followed by
any number of underscores, letters, and numbers.
WDDX provides no notion of a null object. Null objects should be
serialized to empty strings.
WDDX serializes data using a model of pure aggregation. It has no
mechanism for handling object references. Aliased references will
result in multiple object instances being deserialized.
<!ELEMENT wddxPacket (header data)>
<!ATTLIST version CDATA #FIXED "0.9">
<!ELEMENT header (comment?)>
<!ATTLIST header>
<!ELEMENT comment (#PCDATA)>
<!ATTLIST comment>
<!ENTITY % dataTypes "boolean | number | dateTime | string | array | struct" >
<!ELEMENT data ( ( %dataTypes; )* ) >
<!ELEMENT data (%dataTypes;)>
<!ATTLIST data>
<!ELEMENT boolean EMPTY>
<!ATTLIST boolean value ( true | false ) #REQUIRED>
<!ELEMENT string (#PCDATA)>
<!ATTLIST string>
<!ELEMENT number (#PCDATA)>
<!ATTLIST number>
<!ELEMENT dateTime (#PCDATA)>
<!ATTLIST dateTime>
<!ELEMENT array (%data;*)>
<!ATTLIST array length CDATA #REQUIRED>
<!ELEMENT struct (var*)>
<!ATTLIST struct>
<!ELEMENT var (%data;)>
<!ATTLIST name CDATA #REQUIRED>