| Last modified: May 17, 2005 |
| XML and Query Languages |
Recent XML Query Specifications
[February 04, 2000] The W3C XML Query Working Group published a working draft document XML Query Requirements on January 31, 2000.
[September 14, 1999] Preliminary information on the W3C XML Query Working Group is available from the W3C XML Activity Page.
The W3C QL'98 - Query Languages Workshop represents one of several efforts to coordinate design activities that focus upon "querying" XML documents, document collections, and document webs. Features already designed within XSL, XPointer, XLink, DOM, DSSSL, and related specifications provide mechanisms for specifying locations/addresses, for tree traversal, and so forth. A current challenge, arguably, is to unify some of these expression/querying sub-languages as a basis for building generalized query facilities that are applicable to a broad range of requirements within different user communities. No formal XML 'query language activity' was established by the W3C when the W3C XML Working Groups were re-chartered in late 1998, but several have predicted that the W3C will eventually charter a new (XML) Query Language activity and/or working group.
[May 06, 1999] In connection with the publication of a working draft version of the XSL Transformations (XSLT) Specification (W3C Working Draft 21-April-1999) it was formally announced that the W3C XSL WG and the XML Linking WG "have agreed to unify XSLT expressions and XPointers. A common core semantic model for querying has been agreed upon, and this draft follows this model (see 6.1 Location Paths). However, further changes particularily in the syntax will probably be necessary. . ."
[April 14, 1999] Sharon Adler, as XSL Co-chair, posted a message to the xsl-list indicating that W3C working groups are indeed moving toward 'XPointer and XSL pattern unification'. "To all of you who wish to see this unification work go forward, I am writing this to let you all know that the work is underway. Last month both working groups voted to pursue this effort even if there was an impact to the schedule. Therefore, we will take a delay to our PR schedule. . ."
XML/XSL/TEI Framework
Extensible Style Language (XSL) - XSL select patterns are used to test for matching elements/nodes
XML Linking Language - Both the XML Linking Language (XLink) and (more particularly) the XML Pointer Language (XPointer) -- have relevance to 'query/retrieval'.
W3C XML Query Working Group
- XML Query Project (XQuery)
- Mail Archives for the W3C public list 'www-ql@w3.org'. A public mailing list on query languages, including (but not limited to) discussion on the XML-Query project. Subscribed users may post to this list.
- Mail Archives for W3C public list 'public-qt-comments@w3.org'. This QT (Query and Transform) list is for public feedback on the following W3C specifications published by the XML Query and XSL Working Groups: XQuery 1.0, XSLT 2.0, XPath 2.0, XQuery 1.0 and XPath 2.0 Data Model, XQuery 1.0 and XPath 2.0 Functions and Operators Version 1.0.
- Contact: Massimo Marchiori (W3C Contact for XML Query)
[December 04, 2000] W3C XML Query Working Group Publishes XML Query Algebra Working Draft. The first W3C public working draft for The XML Query Algebra has been released for review. Reference: W3C Working Draft 04-December-2000, latest draft http://www.w3.org/TR/query-algebra/, edited by Peter Fankhauser (GMD-IPSI), Mary Fernández (AT&T Labs - Research), Ashok Malhotra (IBM), Michael Rys (Microsoft), Jérôme Siméon (Bell Labs, Lucent Technologies), and Philip Wadler (Avaya Communication). The document "introduces the XML Query Algebra as a formal basis for an XML query language." The development work "builds on long standing traditions in the database community. In particular, we have been inspired by systems such as SQL, OQL, and nested relational algebra (NRA). We have also been inspired by systems such as Quilt, UnQL, XDuce, XML-QL, XPath, XQL, and YaTL. We give citations for all these systems below. In the database world, it is common to translate a query language into an algebra; this happens in SQL, OQL, and NRA, among others. The purpose of the algebra is twofold. First, the algebra is used to give a semantics for the query language, so the operations of the algebra should be well-defined. Second, the algebra is used to support query optimization, so the algebra should possess a rich set of laws. Our algebra is powerful enough to capture the semantics of many XML query languages, and the laws we give include analogues of most of the laws of relational algebra. It is also common for a query language to exploit schemas or types; this happens in SQL, OQL, and NRA, among others. The purpose of types is twofold. Types can be used to detect certain kinds of errors at compile time and to support query optimization. DTDs and XML Schema can be thought of as providing something like types for XML. The XML Query algebra uses a simple type system that captures the essence of XML Schema Structures. The type system is close to that used in XDuce. On this basis, the XML Query algebra is statically typed. This allows to determine and check the output type of a query on documents conforming to an input type at compile time rather than at runtime. Compare this to the situation with an untyped or dynamically typed query language, where each individual output has to be validated against a schema at runtime, and there is no guarantuee that this check will always succeed..." A tutorial introduction in the WD 'The Algebra by Example' introduces the main features of the algebra, using familiar examples based on accessing a database of books. In Appendix A 'The XML Query Data Model', the authors present a formal mapping relating the algebra to the XML Query Data Model. [cache]
[August 15, 2000] The W3C XML Query Working Group has published a revised working draft specification for XML Query Requirements. Reference: W3C Working Draft 15-August-2000, edited by Don Chamberlin (IBM Almaden Research Center), Peter Fankhauser (GMD-IPSI), Massimo Marchiori (W3C/MIT/UNIVE), and Jonathan Robie (Software AG). The document "specifies goals, requirements, and usage scenarios for the W3C XML Query data model, algebra, and query language." The goal of the XML Query Working Group is "to produce a data model for XML documents, a set of query operators on that data model, and a query language based on these query operators. The data model will be based on the W3C XML Infoset, and will include support for Namespaces. Queries operate on single documents or fixed collections of documents. They can select whole documents or subtrees of documents that match conditions defined on document content and structure, and can construct new documents based on what is selected." The working draft outlines several usage scenarios which are "intended to be used as design cases during the development of XML Query, and should be reviewed when critical decisions are made. These usage scenarios should also prove useful in helping non-members of the XML Query Working Group understand the intent and goals of the project: (1) Human-readable documents: Perform queries on structured documents and collections of documents, such as technical manuals, to retrieve individual documents, to generate tables of contents, to search for information in structures found within a document, or to generate new documents as the result of a query. (2) Data-oriented documents: Perform queries on the XML representation of database data, object data, or other traditional data sources to extract data from these sources, to transform data into new XML representations, or to integrate data from multiple heterogeneous data sources. The XML representation of data sources may be either physical or virtual; that is, data may be physically encoded in XML, or an XML representation of the data may be produced. (3) Mixed-model documents: Perform both document-oriented and data-oriented queries on documents with embedded data, such as catalogs, patient health records, employment records, or business analysis documents. (4) Administrative data: Perform queries on configuration files, user profiles, or administrative logs represented in XML. (5) Filtering streams Perform queries on streams of XML data to process the data in a manner analogous to UNIX filters. This might be used to process logs of email messages, network packets, stock market data, newswire feeds, EDI, or weather data to filter and route messages represented in XML, to extract data from XML streams, or to transform data in XML streams. (6) Document Object Model (DOM) Perform queries on DOM structures to return sets of nodes that meet the specified criteria. (7) Native XML repositories and web servers Perform queries on collections of documents managed by native XML repositories or web servers. (8) Catalog search Perform queries to search catalogs that describe document servers, document types, XML schemas, or documents. Such catalogs may be combined to support search among multiple servers. A document-retrieval system could use queries to allow the user to select server catalogs, represented in XML, by the information provided by the servers, by access cost, or by authorization. (9) Multiple syntactic environments Queries may be used in many environments. For example, a query might be embedded in a URL, an XML page, or a JSP or ASP page; represented by a string in a program written in a general-purpose programming language; provided as an argument on the command-line or standard input; or supported by a protocol, such as DASL or Z39.50."
[May 11, 2000] The W3C XML Query Working Group has published a first public working draft of the XML Query Data Model. Reference: W3C Working Draft 11-May-2000; edited by Mary Fernandez (AT&T Labs) and Jonathan Robie (Software AG). Document abstract: "This document defines the W3C XML Query Data Model, which is the foundation of the W3C XML Query Algebra; the XML Query Algebra will be specified in a future document. Together, these two documents [will] provide a precise semantics of the XML Query Language." Description: "This document defines the W3C XML Query Data Model, which is the foundation of the W3C XML Query Algebra; the XML Query Algebra will be specified in a future document. Together, these two documents [will] provide a precise semantics of the XML Query Language. . ." [cache]
[February 04, 2000] A public working draft for XML Query Requirements has been published by the W3C XML Query Working Group as part of the W3C XML Activity. References: W3C Working Draft 31-January-2000, edited by Peter Fankhauser (GMD-IPSI), Massimo Marchiori (W3C and MIT), and Jonathan Robie (Software AG). The draft document "specifies goals, usage scenarios, and requirements for the W3C XML Query data model, algebra, and query language." The goal of the XML Query Working Group "is to produce a data model for XML documents, a set of query operators on that data model, and a query language based on these query operators. The data model will be based on the W3C XML Information Set, and will include support for Namespaces. Queries operate on single documents or fixed collections of documents. They can select whole documents or subtrees of documents that match conditions defined on document content and structure, and can construct new documents based on what is selected." Comments on the working draft are invited, and may be sent to the W3C mailing list, where they will publicly archived.
[September 1999] An XML Query Working Group was announced in the Fall of 1999 in connection with W3C's publication of the XML Phase III activity. "Following the W3C Query Languages Workshop (QL'98), the mission of the XML Query working group is to provide flexible query facilities to extract data from real and virtual documents on the Web. The XML Query Working Group plans to develop requirements in 1999 and continue with design work in 2000. Paul Cotton of IBM is the chair of the XML Query Working Group. W3C contact: Massimo Marchiori."
[September 20, 1999] Public W3C mailing list 'www-ql@w3.org' announced by Massimo Marchiori: http://lists.w3.org/Archives/Public/www-ql/.
[March 05, 1999] "The Quest for an XML Query Standard." By Lisa Rein. From XML.com (March 02, 1999). "Is there a need for a query language that speaks XML? Learn what went on when XML experts got together to talk about the need for an XML query standard at QL'98. [This] W3C workshop on query languages for XML produced a number of interesting proposals for extracting information more efficiently from XML documents. In terms of next steps, the W3C must decide whether they will simply incorporate feedback into the existing XML working group activity? Or will a separate Query Language Working Group or XML Query Language Working Group be formed? One of the difficulties is the somewhat delicate placement of the work within the W3C's architecture, since so many different groups will depend heavily on its deliverables. At this point, the W3C is gathering input from its members as to how the query effort should proceed."
QL'98 - W3C Query Languages Workshop
QL'98 - The Query Languages Workshop - W3C Workshop. December 3-4, 1998. DoubleTree Guest Suites Hotel, Boston, Massachussets. The scope of the W3C Workshop on Query Languages is "to begin the discussion of query languages for the Web (with particular emphasis on querying XML and RDF), of the needed requirements for such query language(s), and of proposing solutions." Sample position papers are referenced below. See also: the local Conference Page.
"Element Sets: A Minimal Basis for an XML Query Engine." - QL '98 Position Paper. Tim Bray. Proposes "a minimum set of functionality required for an XML query facility to be useful. The argument is that a facility which offers set arithmetic plus inclusion and containment is sufficient to express a high proportion of queries, and lends itself to an efficient implementation."
"Experiences with Information Locator Services." - QL '98 Position Paper. Eliot Christian. The paper "relates experiences in developing and promoting services interoperable with the Global Information Locator Service (GILS) standard that has now been adopted and promoted in many fora worldwide. The author describes example implementations and touches on the strategic choices made in public policy, standards, and technology."
"XQuery: A unified syntax for linking and querying general XML documents." - QL '98 Position Paper. Steven J. DeRose. ". . .proposes a query language syntax for XML documents, called ZQuery. Such a query language has quite different requirements than traditional languages; much more different than is commonly appreciated. Many past proposals have taken a basically relational query language (typically SQL), and modified it by the addition of a few constructs: typically a 'contains' operator and some features for matching strings within text chunks against regexes, or against word-roots. Such features, while needed, are not enough. The hard problem arises because the most basic design principles of relational databses do not hold for XML documents."
"Queries on Links and Hierarchies." - QL '98 Position Paper. Steven J. DeRose, C. Michael Sperberg-McQueen, and Bill Smith. 'November 18, 1998.' "This document briefly sets out functional requirements for languages to characterize and retrieve structure and content in linked hierarchies (such as XML documents). This kind of data poses more complex requirements than some others, because not only content but also structure and linkage must be available to queries." [local archive copy]
"XML-QL: A Query Language for XML." - NOTE-xml-ql-19980819, Submission to the World Wide Web Consortium 19-August-1998. Authors of the NOTE are: Alin Deutsch (University of Pennsylvania), Mary Fernandez (AT&T Labs), Daniela Florescu (INRIA), Alon Levy (University of Washington), and Dan Suciu (AT&T Labs). See also the submission request and W3C Staff Comment.
"Enabling Inferencing." [Query languages for RDF] - QL '98 Position Paper. R.V. Guha, et al. The paper "presents an overview of the query services that might be built on top of XML/RDF data. It does not present a specific proposal for an RDF query language; instead, it argues for a query language that is expressed in terms of the RDF logical data model rather than one particular concrete syntax."
"XML Query Language (XQL)." The paper is authored principally by Jonathan Robie (Texcel, Inc.), Joe Lapp (webMethods, Inc.), and David Schach (Microsoft Corporation), with contributions by Michael Hyman and Jonathan Marsh (both of Microsoft Corporation). The XQL paper proposes to use the XSL pattern language as the basis for a general query mechanism.
"Querying and Transforming XML." David Schach, Joe Lapp, Jonathan Robie. "The XQL Proposal, a superset of the XSL pattern syntax, addresses the information retrieval aspects of queries. This paper describes the benefits of using the XSL transformation language together with the XQL Proposal, to provide an integrated environment for queries and transformations."
"Providing flexible access in a query language for XML." - QL '98 Position Paper. Frank Tompa. "A query language for XML imposes semantics on the encoded data. As such, it forms an integral part of the resulting data model. The properties of the language will therefore promote or inhibit future applications of XML. Among other features, such a language must: 1) support access to data stored in any conforming XML document, respecting the encoding choices made by each application's data designers, and 2) allow for data views to be defined over collections of XML documents. In this paper, we do not advocate specific syntactic constructs. . . we propose that an XML document be modelled as a rooted, directed, ordered, labelled tree."
"Querying XML with Lore." - QL '98 Position Paper. Jennifer Wisdom.
Software
Some sample QL software, much experimental. W3C maintains a list of XQuery implementations.
[June 2004] BumbleBee is "an automated test harness for evaluating XQuery engines and validating queries expressed in the XQuery language. BumbleBee takes the pain and uncertainty out of learning and using XQuery. It starts by letting you immediately put several XQuery engines to the test so you know how they stack up against the XQuery specification. Then it lets you easily write your own tests to continually make sure your XQuery expressions produce reliable results when you upgrade your XQuery engine, try different engines, or otherwise make changes to your queries. BumbleBee is all about push-button regression testing. Indeed, BumbleBee tests act as automated change detectors to eliminate costly XQuery debugging cycles..." See Jason Hunter's article.
[June 03, 2002] "Enosys Software Introduces the First XML-Based Real-Time Enterprise Information Integration Platform. Early Adopters Of Enosys' XML Query (XQuery) Solution Attest to Reduced Cost of Custom Data Integration, Rapid Deployment of New Web Applications, and Improved Customer Service." - "Enosys Software, an XML platform software provider, today introduced the Enosys XQuery-based suite of products, the industry's first XML-based platform to enable real-time information integration inside and outside company firewalls... Enosys Software addresses the need within large enterprises to provide an integrated, real-time view of the vast amounts of data that are distributed across multiple information sources, in the form of relational databases, mainframe applications, legacy systems, documents or spreadsheets. These information sources are frequently owned by multiple departments or even trading partners of the enterprise, making it extremely difficult and expensive to create a single view of relevant data in a time-sensitive manner... The Enosys platform is an advanced Enterprise Information Integration (EII) solution that is based on open Java standards and patent-pending XQuery technology. Unlike other proprietary EII-based solutions, Enosys provides a universal layer for accessing enterprise data and eliminates the need to develop specific stovepipe solutions for each backend enterprise data source and application. Developers simply build queries that retrieve the information end users and applications need--independent of the source, location, and access method of the data-- and deliver the information as reusable XML results or web services components... Enosys Software was founded in 2000 by two experts in the area of XML processing, professors Yannis Papakonstantinou (University of California, San Diego) and Vasilis Vassalos (New York University). Their work on data integration and semi-structured data (XML) started at Stanford, where they were members of the same Computer Science program as founding members of Junglee and Google, and worked alongside Computer Science and database research luminaries Hector Garcia-Molina and Jeff Ullman..."
sggrep ".. .works like the grep program in searching a file for regular string expressions. However, unlike grep , it is aware of the tree structure of XML files. . ." [The XML Library LT XML]
sgrep-2 - Has an SGML/XML/HTML scanner; from Jani Jaakkola, University of Helsinki
[June 22, 2001] XML Query Engine 1.0. "The 1.0 release of XML Query Engine is now shipping. XML Query Engine is a Java-based search engine component that allows you to search small to medium-size collections of XML documents for full-text content using XQL, a de facto XML query language standard that's nearly identical to the abbreviated form of XPath. The engine is small (roughly 160 kb) and has a straightforward API that let's you wire it in to your own own Java applications using a SAX1 or SAX2 parser of your choice. The engine also has early experimental support for XQuery. There's a free eval version downloadable at the website. The eval version is unsupported but is otherwise identical to the production version, minus the availability of a persistent index feature."
[May 09, 2001] XCache: XML-Query Caching System (MQP) and XCache Demo. "Xcache is a web-based XML query engine. XML is a new language for the web that allows data to be stored in a specified format in a regular text file. This allows it to be queried much more efficiently than a regular HTML document. The Xcache system not only allows users to query XML documents, but it stores the results from these queries in its cache and uses them to answer future queries. This query caching process allows many queries to be answered much faster than they would be normally since the query engine can retrieve many of the results from its cache rather than having to access the data across the network as in a typical search engine." See demo. See details from the Worcester Polytechnic Insitute (WPI) Database System Research Group (DSRG).
[February 15, 2001] From Charles Daringer. "XML parser that supports DOM and Saxon for SQL like searches of XML including arrays." Labat Anderson has developed some code that reads XML forms, Pureedge, XFDL, XML, DOM, SAX, into an array where an itemization of the array is notated from a left to right fashion at the depth of the node count from a left to right fashion. This enables a recursive search of these nodes down to the array level based on the content of a preexisting value of an array. The process reads the XML from stream, file, or table and then forms the array to only the required size of the XML present. Once 'persisted' to an array multiple 'conditional' passes or searches are facilitated in a SQL fashion where a result set can be utilized to formulate the next search. The code is written in Visual Basic 6 and can be called as a DLL which returns the array, or the code can run inside a SERVICE that runs on NT. If there is a requirement to parse xml on a conditional basis down to an array level... where the number of items in an array, or the number of arrays are a runtime uncertainty, than this is the 'fastest' approach. You can contact AL BUONI at Labat Anderson, Inc. for details."
[November 04, 2000] "SODA2 - An XML Semistructured Database System." - "The novel SODA2 architecture facilitates several crucial features which are seldom available in other database systems. The SODA2 query processor is mainly located at the client side. Each query processor contains an internal query translator that maps an query from one language into a SODA2 internal micro-query language. Therefore, SODA2 supports multiple query languages which include XPath expressions, XQL and XML-QL to date...The query interface provides an embedded query interface to allow user developed applications to query SODA2 in the easiest way. As SODA2 query processor contains a query translator and operates internally on a micro-query language and SOM, multiple query languages are supported. Curently SODA2 supports XPath expressions, XQL, and XML-QL. It follows the W3C's DOM recommendations so that non-database applications can be easily built and ported to SODA2 with DOM interface and ignore without its rich database functionalities. Alternatively, advanced users can call more sophisticated functions defined in the SODA2 Object Model (SOM) interface from the lower layer to develop time-critical applications or create customized plug-ins with their own indexing and optimization schemes.
[July 2000] "XMLGet." By Neil Ferguson. Notice posted November 27, 2001 to 'xql@franklin.oit.unc.edu'. "XMLGet is an efficient, open source query tool for XML documents based on the XQL query language You can download the Java source and binaries for the latest version (1.0) of XMLGet. You can also download the user documentation. A document entitled 'XMLGet -- Testing, Profiling, Benchmarking, and Proposed Future Work' can be downloaded; it describes some of the caveats found in XMLGet during testing, profiles various sections of the XMlGet parser, compares performance against other XQL engines, and makes suggestions for future enhancements to XMlGet." [cache]
NIAGARA. "Niagara can be used for retrieving XML data, querying and monitoring them for some interesting changes. These three functionalities are implemented by its three main components: Search Engine, Query Engine and Trigger Manager. XML is gaining popularity for representing semi-structured data on the net. Additionally it is being used as a medium for data exchange. Self-describing nature of XML data help in doing a better job of retrieving relevant sources, when compared to a plain text file. For example if we are looking for a ship named 'Montreal', then all the current search engine will give us data sources that will most likely contain information about the city named 'Montreal'. With our XML search engine we can return data sources that will contain the name 'Montreal' in the ship element... See the slide presentation.
XPERT - 'Developing Best XML Content Management Tools in Java'. EPERT provides an efficient indexing and allows powerful retrieval with XPath based query language. We have been developing XML content management tools in Java. One of Java software applications is XPERT (XPath based query language Evaluation and Retrieval Tool). It allows different types of XML documents to be indexed together and makes it possible to be retrieved by an XPath based query language. We compress the index and save it to the disk in a very efficient way, keeping the overhead as low as possible. Furthermore, we optimize the query evaluation and allow the fast query processing. We believe that it scales well in large heterogeneous XML collections. You can download XPERT and manage the XML content in an easy and efficient manner..."
[November 03, 2000] XML Query Engine Update. Howard Katz posted an announcement for the availability of XML Query Engine v0.89: "This is a major update that fixes a number of outstanding bugs and adds several new features and optimizations. This version is getting very close to beta territory, since all major features, with the exception of a persisted store, are now in place. Updates include: (1) logical subquery operators 'and' and 'or'; (2) set operators 'union' and 'intersect'; (3) namespace support (4) setDoFullText(boolean) api for turning off element text indexing (speeds up indexing, query retrieval, and drastically reduces index size in appropriate cases); (5) showDocTree(docID) api for quick visualization of element hierarchy (6) simple compound-word matching in element content; (7) attribute content moved into index for improved speed and precision; (8) a number of other optimizations, primarily to improve performance and reduce memory footprint during full-text queries." The XML Query Engine is "a JavaBean component that lets you search your XML documents for element, attribute, and full-text content. It can index multiple documents using a SAX parser of your choice. The index, once built, can be queried using XQL, a de facto standard for searching XML that is, very nearly, a proper subset of XPath. XML Query Engine extends XQL's syntax to provide a full-text capability, something lacking in standard XQL. This lets you say such things as Find me the first paragraph within either a division or a chapter that contains both the words 'xml' and 'xsl' or Give me a list of all elements containing an href attribute which points to a '.com' organization. XML Query Engine is an embeddable component that's callable from your application. It requires some straightforward Java programming to wire the query engine to your front-end code. The engine uses a result-listener architecture to deliver its results: You register an XQL result listener with the engine before calling your first query. Once your query's been resolved, the result-set document is delivered to your listener's results() method. Query results can be delivered in one of three formats. Two of these are XML, one of which is a standard result format, similar in structure to that returned by other XQL vendors, while the other is specialized to return 'navigational metadata' describing the nodes it contains in terms of their location within their originating documents. You can use this metadata to easily re-navigate, via either SAX or DOM, back into the original documents for further post-processing if desired. The third result-set format is CSV, Comma-Separated-Values, for particularly fast and compact result delivery of navigational metadata. XML Query Engine is a work in progress. The current version is fast approaching beta status. I've implemented all the core XQL features necessary to support full-text capability on top of the standard language..."
[October 04, 2000] Fujitsu XLink Processor. Developed by Fujitsu Laboratories Ltd., as "an implementation of XLink and XPointer. This processor supports XML Linking Language (XLink) Version 1.0 Candidate Recommendation. You may use this processor and other included programs without charge for 60 days after the installation. You must read 'LICENSE', before you begin your installation..." Multi-Platform: Developed with Java, this processor can be used on many platforms which support Java Runtime Environment. Support for XLink Ver.1.0CR: This processor supports XLink Ver.1.0CR, which is now being discussed in W3C. XLink/XPointer processing with DOM: This processor works with DOM. This processor can work with any XML processor or parser which can create DOM, on condition that an appropriate interface between this processor and it is implemented. Supported Features: XLink Features: simple-type element and its related attributes extended-type element and its related attributes locator-type element and its related attributes resource-type element and its related attributes arc-type element and its related attributes title-type element, Linkbases. XPointer Features: Bare Names, Child Sequences. Links: demo application; download; license. Contact: xlp@ml.fujitsu.co.jp or Masatomo Goto.
XML-QL: A Query Language for XML ". . . techniques and tools should exist for extracting data from large XML documents, for translating XML data between different schemas (DTD's), for integrating XML data from multiple XML sources, and for transporting large volumes of XML data to clients or for sending queries to XML sources. We have implemented a query language for XML, called XML-QL, to investigate solutions to the problems above. The language has a WHERE-CONSTRUCT clause, similar to SQL's SELECT-WHERE construct, and borrows features of query languages recently developed by the database research community for semistructured data. The implementation is publically available." By Mary Fernandez. See also the XML-QL Users' Guide.
[March 28, 2000] XMLQUERY: An extension of the LTNSL query language to allow more complex queries including operators for finding sequences of XML elements. Now including a prototype version of XMLPERL2 (a transformation system similar to XMLPERL but using the extended XMLQUERY language).
[May 24, 2000] Jonathan Robie (Software AG) recently announced design and development work on 'Quilt: An XML Query Language'. "Quilt is an XML query language designed for queries on heterogeneous data sources, and drawing from the design of XQL, XML-QL, SQL, and OQL. The authors included me, Don Chamberlin (one of the two authors of SQL), and Dana Florescu (well known in the object database community, and one of the authors of XML-QL). The two papers to look at are found here: http://www.almaden.ibm.com/cs/people/chamberlin/quilt.html. Viz., (1) "Quilt: An XML Query Language," by Jonathan Robie, Don Chamberlin, and Daniela Florescu [to be presented at XML Europe, Paris, June 2000], and (2) "Quilt: An XML Query Language for Heterogeneous Data Sources," by Chamberlin/Robie/Florescu [to be presented at WebDB 2000, Dallas, May 2000]. Quilt has no official status in the XML Query WG. The authors are all members of that WG (and one was an editor of both the Requirements WD and the Data Model WD), and we designed the language to meet the requirements of the Query WG. Since the Query WG is not yet working on concrete query languages, it is too early to say how it will respond to this proposal..." From one of the design papers: "As increasing amounts of information are stored in XML, exchanged in XML, and presented as XML through various interfaces, the ability to intelligently query XML data sources becomes increasingly important. The data model of XML is quite different from the data models of traditional databases, and traditional database languages are not well suited to querying XML sources. In addition, XML blurs the distinction between data and documents, allowing documents to be treated as data sources and traditional data sources to be treated as documents. Query languages, including XML query languages, still tend to be designed either for documents or for data. Since XML may represent a rich variety of information coming from many sources and structured in many ways, an XML query language must try to provide the same rich variety in the queries that may be formulated. Quilt is a query language for XML. It originated when the authors attempted to apply XML query languages such as XML-QL, XPath, XQL, YATL, and XSQL to a variety of use cases. We found that each of these languages had strong advantages for some of the queries we examined, but was unable to express other queries we found equally important. Therefore, we chose to take some of the best ideas from these languages, plus some ideas from SQL and OQL, integrating them with a fresh syntactic approach. Our goal was to design a small, implementable language that met the requirements specified in the W3C XML Query Working Group's XML Query Requirements. During our design work, we have adapted features from various languages, carefully assembling them to form a new design -- hence the name 'Quilt'. The resulting language supports queries that draw information from various sources and patch them together in a new form. This is another reason for the name 'Quilt'." Note also in this connection "XML Query Languages: Experiences and Exemplars." The paper "identifies essential features of an XML query language by examining four existing query languages: XML-QL, YATL, Lorel, and XQL. The first three languages come from the database community and possess striking similarities. The fourth comes from the document community and lacks some key functionality of the other three..." Local: cache, XMLEuro, cache, WebDB.
METU-SRDC. "METU-SRDC presents an implementation of XML-QL. XML-QL is originally implemented by AT&T , by mapping the queries into STRU-QL. METU-SRDC's implementation works directly on XML files and contains some advanced features like "Recursive Functions" and "Aggregate Functions" (Sum, Count ...). The extensions to the original specification are explained through examples. Implementation of the package is done in pure Java . JavaCC and IBM's xml parser for java , xml4j were used as auxiliary applications."
"Quilt; An XML Query Language for Heterogeneous Data Source." By Daniela Florescu, Jonathan Robbie, and Don Chamberlin. Proceedings of the workshop on Web and databases (WebDb) in conjunction with SIGMOD'00, Dallas, Texas. [cache]
[August 18, 2000] QuiltParser: JavaCC Grammar for Quilt XML Query Language. "Quilt Parser in java is available for download." From Dongwon Lee. JavaCC parser source code and test queries are available at http://www.cobase.cs.ucla.edu/projects/xpress/quilt/. Description: "QuiltParser is a parser for the Quilt XML query language written with JavaCC as a part of XPRESS project at UCLA / CSD. This small, implementable language has been recently proposed by Robie, Chamberlin, and Florescu; it integrates the advantages of various languages while meeting the W3C's XML Query Requirements. More information on Quilt may be found at http://www.almaden.ibm.com/cs/people/chamberlin/quilt.html. The current implementation is based on the BNF and test examples from this paper (presented at XML Europe, Paris, June 2000).
"Querying XML Data." By Daniela Florescu, Alin Deutsch, Mary Fernandez, Alon Levy, David Maier, and Dan Suciu. In Data Engineering Bulletin Volume 22, Number 3 (1999), pages 27-34. [cache]
[March 28, 2000] 'New Version of Andrei's generic SGML tools' - including XML query tool.
[March 29, 1999] On behalf of GMD-IPSI (German National Research Center for Information Technology) and the XML Competence Center at GMD, Ingo Macherius has announced Java based XQL and W3C-DOM implementations. "The engine consists of two main parts: (1) A persistent implementation of the W3C-DOM, and (2) A full implementation of the XQL language. The XQL engine implements the W3C-QL '98 workshop paper syntax of XQL. It uses a novel indexing algorithm for XML (publication pending), which indexes the document while processing the first query. Subsequent queries to the same document are considerably accelerated.
[Matthew Sergeant:] "there's an implementation of XML-QL in my directory on CPAN for perl users, which needs fixing up a little bit, but it's quite usable (if a little slow). It facilitates the use of perl's regexp syntax for queries as well as the system used by XML-QL, which makes it nice and powerful..."
XRS. Dongwook Shin (Lister Hill Center, National Library of Medicine) recently announced the public availability of XRS: An XML Retrieval Engine. XRS is an XML search engine that is able to retrieve any elements a user wants very effectively. Unlike other XML search engines that get back whole XML documents to you, you can impose conditions on any elements with weights and get back relevant elements efficiently. XRS uses a couple of new techniques that have been recently developed. One of those is the BUS (Bottom Up Scheme) technique developed for indexing and retrieving structured documents efficiently..."
SIM - The Structured Information Manager. The Structured Information Manager (SIM) provides a complete environment with which to develop end-to-end document management solutions. SIM gives superior performance in managing very large, complex stores of data. It stores and indexes SGML, XML, MARC and RTF natively, and also accommodates multimedia formats. Because of the native support for SGML/XML, SIM document management solutions are very efficient and scale up as the document repository grows from tens of gigabytes to hundreds of gigabytes in size. SIM is completely Web enabled. Particular strengths of SIM include sophisticated SGML/XML support, high performance text indexing, powerful Web based application support and native support for international standards such as SGML, XML, RTF, MARC, ODBC and Z39.50. . . . By directly supporting the import and export of documents marked up in SGML/XML, SIM automatically has access to a large range of text processing tools such as text editors, and translators that can convert text between a variety of word processing standards. Storing text in SGML/XML format also means that the SIM retrieval engine has access to the structure of the documents as well as their contents. This structural information can be used to improve the processing of user queries and to determine the best ways to display a document."
Xdex. "XML data and documents incorporate rich layers of metadata that can be leveraged for extremely accurate information categorization and context-based searching. Sequoia's Xdex is a uniquely powerful XML indexing engine that allows you to easily take advantage of this functionality. . ."
TEXTMLServer. "The LEXml Library is a web-ready Intranet library solution for legal documents where you can store and research the full text of your documents. Take the Guided Tour and see why our library is a must for your corporate portal. The LEXml Library is available in a free LITE edition that is time bomb free and fully functional up to 500 documents... IRT (Interactive Repository Technology) is the fruit of over ten years of experience in document indexing and publishing technology. IRT offers a robust COM API designed with XML in mind and therefore offers all the performance and flexibility of XML for indexing and publishing purposes.
[November 24, 1999] From Duane Nickull, XSL List, Wed, 24 Nov 1999 11:36:41 -0800: "The goXML XML Context based Search Engine has a private API for conducting queries and submitting content via XML. The Xml Query Interface (XQI) is available at http://www.goxml.com/xqi.htm. The documentation and source code are yours to examine in hopes of shedding some light on this subject. Also, you may want to use the XML search engine to find content on Unicode characters."
Lorel Query Language - Software is available for download. "The Lore project focuses on defining a declarative query language for XML, developing new technology for interactive searches over XML data, and building an efficient XML query processor. Lore is a database management system (DBMS) for XML, a simple and increasingly popular data model. Nested, tagged data is the essence of XML. . . We have developed the Lorel language specifically for querying XML or other semistructured data. Based on OQL, Lorel provides powerful path traversal operators and makes extensive use of type coercion to help yield "intuitive" results for all queries over XML data. The language is described in this paper."
[March 08, 2000] "Lore: A Database Management System for XML. [INTERNET PROGRAMMING.]" By Roy Goldman, Jason McHugh, and Jennifer Widom. In Dr. Dobb's Journal Volume 25, Issue 4 [#311] (April 2000), pages 76-80. ['Lore is a DBMS designed specifically for XML. In the same way that SQL queries relational DBMSs, Lore provides the query language Lorel for issuing expressive queries over XML data. Additional resources include 'lore.txt' listings.'] For publications and software, see the Lore Web site.
BUS - Bottom Up Scheme of indexing and retrieval for SGML/XML documents. See: "BUS: An Effective Indexing and Retrieval Scheme in Structured Documents." By Dongwook Shin, Hyuncheol Jang, and Honglan Jin [Department of Computer Science, Chungnam National University, Taejon, South Korea]. Pages 235-243 (with 16 references) in Digital Libraries '98. Proceedings of the Third ACM Conference on Digital Libraries (Held June 23-26, 1998). New York, N.Y.: Association for Computing Machinery, 1998. Abstract: "In recent digital library systems or the World Wide Web environment, many documents are beginning to be provided in the structured format, tagged in mark up languages like SGML or XML. Hence, indexing and query evaluation of structured documents have been drawing attention since they enable to access and retrieve a certain part of documents easily. However, conventional information retrieval techniques do not scale up well in structured documents. This paper suggests an efficient indexing and query evaluation scheme for structured documents (named BUS) that minimizes the indexing overhead and guarantees fast query processing at any level in the document structure. The basic idea is that indexing is performed at the lowest level of the given structure and query evaluation computes the similarity at a higher level by accumulating the term frequencies at the lowest level in the bottom up way. The accumulators summing up the similarity play the role of accumulating all the term frequencies of the related part at a certain level. This paper also addresses the implementation of BUS and proves that BUS works correctly. . .
Xtract - a command-line `grep'-like tool for searching XML documents. Just as `grep' returns lines which match your regular expression, so Xtract returns all those sub-trees from XML documents which match a query pattern."
TEXTML Server. "The free TEXTML SERVER Lite is available in our download section. This evaluation edition of our Interactive Repository Technology has all the functionality of the licensed version and is 'Time Bomb Free'. The LEXml Library is a web-ready Intranet library solution for legal documents where you can store and research the full text of your documents. The LEXml Library is available in a free LITE edition that is time bomb free and fully functional up to 500 documents. What is Interactive Repository Technology? IRT is the fruit of over ten years of experience in document indexing and publishing technology. IRT offers a robust COM API designed with XML in mind and therefore offers all the performance and flexibility of XML for indexing and publishing purposes. Read about the benefits and features of IRT in our product section, and see how the TEXTML Server is your ideal tool for creating XML document management solutions..."
Articles and Other Resources
See references also in "XML and Attribute Grammars."
[May 17, 2005] "DataDirect XQuery Enables XQuery Everywhere. Easy, Powerful Java XQuery Component Simplifies XML and Relational Data Integration — Now Available in Beta." - "DataDirect Technologies, the industry leader in standards-based components for connecting applications to data, and an operating unit of Progress Software Corporation, today announced the availability of DataDirect XQuery, an easily embeddable XQuery implementation for XML applications that need to process both XML and relational data sources. Developers can get started with XQuery and the XQuery API for Java (XQJ) today using the beta release of DataDirect XQuery, the first XQuery component to implement XQJ... Middleware applications such as report generation, Web services, and other Web applications require accessing different XML and relational data sources and then integrating, manipulating, and transforming the data from these sources into a coherent output format. In the past, solutions for working with XML and relational data have been difficult to build, typically requiring applications to spend significant effort navigating and casting data from XML structures, and often requiring too many different languages and software systems which were proprietary or not scalable. DataDirect XQuery simplifies working with XML and relational data together, allowing Java developers to programmatically invoke and process XQuery expressions against any major relational database including Oracle, Microsoft SQL Server and IBM DB2, directly from within their Java applications. Not only does this powerful new approach to writing database-independent XML data integration code require less code than yesterday's solutions — because it is based on XQuery and XQJ standards, it is easier and more intuitive to learn and use, saving developers precious time and money... DataDirect XQuery is the first embeddable component for XQuery that implements the XQuery for Java API (XQJ). The XQuery API for Java (XQJ) is an API designed to support the XQuery language (in the same way that the JDBC API supports the SQL query language). XQJ, which is based on the XQuery Data Model rather than the relational model, allows a Java application to submit XQuery queries to any XML or relational data sources and to process the results. The XQJ standard (JSR 225) is being developed under the Java Community Process and is in the early stages of being defined..."
[May 16, 2005] "Ipedo 'Dual-Core' EII Platform Processes Both SQL and XQuery." By Tom Sullivan. From InfoWorld (May 16, 2005). "Ipedo on Monday launched XIP 4.0 (Extensible Information Platform) and described it as a 'dual-core' EII offering that works with both SQL and the emerging XQuery querying languages. New to Version 4.0 are support for Business Objects XI and Crystal Reports XI, a visual rules GUI, Web services publishing, and the dual querying engines. Integration with Business Objects and Crystal Reports BI tools enables data federated by Ipedo to be available in reports. 'Ipedo looks like one big database, so Business Objects users can point to a number of data sources,' said Tim Matthews, Ipedo co-founder and vice president of marketing. The visual GUI helps users create rules based on data values, such as the ability to send alerts or invoke more detailed analysis when appropriate. XIP 4.0 enables developers to publish views of data as Web services, which the company claims allows organizations to leverage EII within a SOA..." See details in the announcement: "Ipedo Releases New Version of Enterprise Information Integration (EII) Platform. Ipedo XIP 4.0 Provides Integration across Broadest Range of Data and Content with New Dual-Core Query Architecture."
[May 06, 2005] "Debunking XQuery Myths and Misunderstandings." By Frank Cohen. From IBM developerWorks (May 06, 2005). "If you work with XML, Web services, or Service Oriented Architecture (SOA), you will likely benefit from the emerging XML Query (XQuery) standard. XQuery is not even a formally accepted standard, yet dozens of implementations help software architects and developers every day. What began as a standard for querying XML documents now includes the next-generation standards for XML selection (XPath 2), XML serialization, full-text search, and functional XML data modeling. A project of this size is bound to have much myth and misunderstanding that needs to be debunked. Here are some of the more common myths and misunderstandings surrounding XQuery. Frank Cohen details and clarifies many of the myths and misunderstandings that surround XQuery. XQuery shows great promise because it reduces the amount of code you need to write to build services that work with XML. The greater XQuery ecosystem provides a unified way to query XML documents, including XML selection, serialization, full-text search, and functional data modeling. Work continues at the XQuery specification Working Group, and this will lead to even more benefits for software developers who work with XML..."
[May 03, 2005] "Microsoft MVPs Petition for XQuery Support in Whidbey." By John K. Waters. From Application Development Trends (May 03, 2005). "When Microsoft announced in January that it was dropping XQuery support from the next release of the .NET Framework, the company's reasoning seemed sound enough: XQuery will not receive final approval from the World Wide Web Consortium (W3C), the standards body shepherding its development, until early 2006. But some XQuery users are hoping to get Microsoft to reconsider its decision. DataDirect Technologies, a provider of data connectivity components and XML development tools, issued an online petition this week, which it hopes will 'convince Microsoft of the overall importance of supporting XQuery in the .NET framework.' As of Monday, the 'XQuery for All' campaign had garnered a reported 140 petition signatures from members of Microsoft's Most Valuable Professionals (MVPS) program. DataDirect acknowledges that Microsoft hasn't abandoned XQuery altogether. The Redmond software giant is a longtime supporter of the language, currently serves as a member of the W3C XQuery standards committee and has publicly committed to helping to complete the standards work. Other large software vendors, including Oracle and IBM, are supporting XQuery 1.0 in upcoming versions of their enterprise-class products..."
[March 21, 2005] "SQL vs. XML in a Database World." By Joab Jackson and Jonathan Robie. From Government Computer News (March 21, 2005). ['Jonathan Robie is one of the chief authors of XQuery, an SQL-like query language that can be used for searching both XML documents and relational databases. W3C, the international body that creates and maintains Web standards, is now reviewing XQuery as a formal draft.'] Robie on XQuery and Structured Query Language: "SQL is a relational query language, XQuery is an Extensible Markup Language query language. If all you are doing is querying relational databases, then SQL is the language you want. XQuery works best if you're querying XML or a combination of XML and relational sources. XML's logical structure is based on hierarchy and sequence. Two things that relational databases don't do is hierarchy and sequence. Yet Web sites typically get a lot of information from databases, but they don't put two-dimensional tables up on the screen. You create hierarchies for that information. In XML you structure everything with hierarchy and sequence. Let me give you a scenario. Your Web site has a message coming in: Someone wants to ask for the price of some stocks. Now we can join XML documents and various relational tables. The XML documents will identify the person who wants the information. It will give the data range that person is interested in. We join that against a relational database to figure out how the stocks have performed, and we build an XML structure that contains the result... I think that since the standards boards of the International Standards Organization and the American National Standards Institute are in the process of adding all of XQuery into SQL, it is just hard to claim that the SQL community is not accepting XQuery. If you go to the major database conferences, you will see a lot on XML. Every relational database vendor has added XML support. People are very conservative about their database choices. But the flow of information is just as important as the control of information..."
[March 02, 2005] "Getting Started with XQuery." By Bob DuCharme. From XML.com (March 02, 2005). "Although the W3C's XQuery language for querying XML data sources is still in Working Draft status, the recent XML 2004 conference showed that there's already plenty of interest and many implementations. While the Saxon implementation may not scale up as much as the disk-based versions that use persistent indexes and other traditional database features, you can download the free version of Saxon, install it, and use XQuery so quickly that it's a great way to start playing with the language in order to learn about what this new standard can offer you. To illustrate running a query, DuCharme starts with a toy example that demonstrates how to tell Saxon which query to run against which XML, then moves on to examples that show useful queries run against real XML data. Part of the appeal of XQuery to people with more of a traditional database background and less of an XML geek background is that XQuery also offers a more SQL-like syntax. Issuing a query against multiple documents at once is an example of a task that, while not impossible in XSLT, is much easier in XQuery when we use the collection function. You can use 'collection' in XSLT 2.0 as well as in XQuery, because it's one of the XQuery 1.0 and XPath 2.0 Functions and Operators'; its use with XQuery generally allows more concise requests than it does with XSLT. In part two of the article, the author will demonstrate XQuery's ability to sort and aggregate data; it will also show how user-defined functions in queries can expand the possibilities for how you select and use the data in your XML documents with XQuery..."
[February 2005] "XQuery: A New Way to Search." By Caroline Kvitka. From Oracle Magazine (February 2005). "The challenge for XQuery is to search XML documents and pull out the document that you want or the parts of the document that you want efficiently, easily, reliably, and predictably. 'The situation is actually complicated a little bit more by the fact that XML has come to serve different purposes, sometimes at the same time,' says Jim Melton, consulting member of the technical staff at Oracle, co-chair of the W3C's XML Query Working Group, and editor of all parts of the SQL standard. 'XML is used to mark up documents, but it's also used to mark up data to say, 'Here is the employee ID,' 'Here is the employee salary,' and so on. Because XML serves a mixture of purposes, the query language has to be able to deal with the different uses of XML.' From the looks of it, XQuery is standing up to the challenge. Currently a working draft specification of the World Wide Web Consortium (W3C), XQuery was first proposed in 1998. Oracle, IBM, Microsoft, DataDirect Technologies, Bell Labs, and BEA are all active in the development of the specification. In addition, several universities have been involved in solving some of the theoretical data model problems..."
[December 29, 2004] "XQuery's Niche." By Edd Dumbill. From XML.com (December 29, 2004). "At the beginning of December, Roger Costello posed a question that has given rise to a thread that has lasted all month, mentioning (1) that the capabilities provided by XQuery represent a subset of the capabilities provided by XSLT/XPath 2.0, (2) The XQuery syntax is a hybrid -- it has some XML characteristics but is not XML. Norm Walsh notes that It's a non-queryable, non-transformable subset of XSLT 2.0'. Given this, I am wondering: 'What niche XQuery is expecting to fill? It was a timely moment to ask such a thing, as over the course of this year lines of division have been forming over the question of XSLT 2 vs. XQuery. As key implementations emerge, such as Microsoft's, XQuery has been welcomed by those who are mentally incompatible with XSLT's declarative nature. At the same time, XSLT devotees just can't see what the fuss is about, perceiving XQuery as a custom syntax for what XSLT can already do. Jonathan Robie, who given his long experience with XQuery should be well-placed to answer, gave a couple of answers..."
[December 16, 2004] "Priscilla Walmsley on XQuery and XML Schema Technologies." By Ivan Pedruzzi and Priscilla Walmsley [email]. In The Stylus Scoop Newsletter (December 16, 2004), Stylus Studio Developer Network. 'Priscilla Walmsley has been working closely with XML Schema and XQuery for years. She was a member of the W3C XML Schema Working Group from 1999 to 2004, where she served as editor of the second edition of XML Schema Part 0 (Primer). As a result of her work with XML Schema, Ms. Walmsley wrote the respected book Definitive XML Schema for Prentice Hall. She has also been an Observer of the XML Query Working Group for two years. During that time she has written another book, Definitive XQuery, which will be published in 2005. Currently, Ms. Walmsley serves as Managing Director of Datypic, where she specializes in XML- and SOA-related consulting and training. Ivan Pedruzzi, Stylus Studio's Senior Product Architect, and editor of The Stylus Scoop newsletter, caught up with Ms. Walmsley at the XML Conference & Exhibition 2004 (XML 2004) last month, where Ms. Walmsley gave a presentation entitled 'Introduction to XQuery'..." The two chatted about the XQuery buzz, XML Schema, XQJ technologies, and other hot topics in the XQuery development arena.'] Walmsley: "I was immediately attracted to XQuery because it has an intuitive syntax that I enjoy using and stretching to its limits. Having spent many years using SQL, XQuery feels familiar, yet much more powerful. I've enjoyed working with XSLT and XPath 1.0 over the years, but for some of the work I've done they felt like an awkward fit. For a transformation scenario where I'm saying 'every time you get an x element, do this' it works great. But for applications that involve selecting a subset of an XML document, joining it with other data, and performing calculations or manipulating it in some way, I've sometimes felt like XSLT was making me force a square peg into a round hole. XQuery embedded in program code is a great way to reduce (and transform) the set of data you're working with rather than tediously traversing the DOM model of an entire document. In the past I've done this with XPath, but XQuery lets me join multiple data sources easily and sort my results, actions that are not part of XPath. Being a true data-head, I also really like the typing capabilities of XQuery. Some of the advanced functionality of XQuery is more data-oriented, and there are some compelling benefits for using XQuery with XML Schemas..."
[December 16, 2004] "Introduction to XQuery." By Priscilla Walmsley (Datypic) Tutorial given Monday, November 15, 2004 at the XML 2004 Conference & Exposition. 86 slides. "XQuery is a language for querying XML data and documents. This tutorial covers the basics of XQuery from a technical perspective. It will provide attendees with a solid understanding of the syntax and structure of XQuery expressions..." Note also the author's book announcement: Definitive XQuery. (Prentice Hall PTR, Forthcoming 2005, ISBN: 0131013750). Book overview: "Definitive XQuery provides complete coverage of the W3C XQuery 1.0 standard. In addition, it provides the background knowledge in namespaces, schemas, built-in types and regular expressions that is relevant to writing XML queries. The book is designed for query writers who have some knowledge of XML basics, but not necessarily advanced knowledge of XML-related technologies. It can be used both as a tutorial, by reading cover to cover, and as a reference, by using the appendices to locate specific functions and types... [from the interview, cited above:] it is designed to be used as both a tutorial and a reference. It covers the entire XQuery language, including all the overlap with XPath 2.0. The reference part of the book has detailed descriptions and examples of all the built-in functions and types; something that will be useful to both XQuery and XPath 2.0 users. It will probably be out in the third quarter of 2005, depending on when XQuery becomes a Candidate Recommendation...as I was writing the book, I came up with a series of illustrative examples which I call 'useful functions'. What sets them apart from regular examples is that they are likely be used by readers in their own queries. They range from string functions like substring-after-last and last-index-of, to functions that modify element and attribute nodes, such as add-attribute, change-element-namespace, and so on. These functions are not built into XQuery because clearly there is a benefit to keeping the recommendation smaller. But they would be useful to a lot of query authors. I eventually realized that I had hundreds of ideas for these functions — far too many to put in the book. So, I'm working on putting these functions together into a library that will be available through my company..."
[March 10, 2004] "BumbleBee, the XQuery Test Harness." By Jason Hunter. In XML.com (March 10, 2004). "Will XQuery be the key that unlocks a new generation of data and content? Nearly every vendor, from the well-known old guard (IBM, Oracle, and Microsoft) to the plucky upstarts (Cerisent, X-Hive, and Qizx) has expressed their support for XQuery and are actively collaborating in its standardization. Under development by the W3C and in Last Call, XQuery looks poised to become the standard query language by which companies access and manipulate semi-structured data and merge together disparate data and content repositories. Using XQuery can be frustrating, as you're faced with choosing from a variety of XQuery vendors that support different versions and interpretations of the XQuery specification; once you've selected the XQuery engine that's best for you, it can be hard to know if the queries you write today will produce reliable results tomorrow after you upgrade your engine or make changes to your queries. The BumbleBee XQuery test harness addresses these frustrations and takes the pain and uncertainty out of learning and using XQuery. Named because it buzzes around FLWORs, BumbleBee provides a cross-platform, vendor-neutral automated testing environment for XQuery development. In other words, BumbleBee is to XQuery what JUnit is to Java. Write your query, define the expected result, and let the tool do the rest... BumbleBee provides a powerful, portable, vendor-neutral automated test environment for XQuery. With BumbleBee you can automate your regression testing, compare multiple XQuery engines, and learn the language through structured challenges. The latest BumbleBee release, version 1.2, includes support for seven vendors and numerous specification draft releases. The easy-to-write .bee file format allows for quick development of tests, including negative tests and compound tests. Future BumbleBee versions may include a graphical query execution environment and test authoring tool."
[March 19. 2004] "XQuery Normalizer and Static Analyzer." By Deepak M. Srinivasa and Rajeshwari Rajendra (Technology Incubation Center, IBM Software Labs, India). 10 pages (with 9 references). "XQuery Normalizer and Static Analyzer is a tool for processing XQuery expressions. Specifically, the two components of the tool — the Normalizer and the Static Analyzer perform two kinds of processing..." See the references and description also on the IBM alphaWorks web site: "XQuery Normalizer and Static Analyzer can be used to parse, normalize, and find the output type and an XQuery Expression. The XQuery grammar supported in this version is the specification in XQuery 1.0: An XML Query Language, W3C Working Draft 15 November 2002. The parser is used to check whether a given XQuery expression conforms to the specified grammar and is devoid of syntactic errors. As the first step, any application that makes use of XQuery language will need a parser, which is readily available in this package. Further, the result of parsing, that is, the parse tree, will be the input for further processing phases. This parse tree for XQuery expression is given as the XML representation of the XQuery expression by the parser. The normalizer is used to tranform an XQuery expression to a normalized form conforming to the XQuery Core Grammar specified in the XQuery 1.0 and XPath 2.0 Formal Semantics, W3C Working Draft 15 November 2002. See the below questions for more details on normalization. The static type analyzer is used to determine the output type for a given XQuery Expression without actually executing it. Static type analysis checks whether each expression is type-safe, and if so, determines its static type. If the expression is not type-safe, static type analysis yields a type error. For instance, a comparison between an integer value and a string value might be detected as a type error during the static type analysis. In other words, static type analyzer is a semantic checker. Any application required to find the type of an XQuery expression without its execution can make use of this tool..." (FAQ document)
[September 05, 2003] "XQuery from the Experts: Influences on the Design of XQuery. Book Excerpt Explores the Origins of the XML Query Language." By Don Chamberlin (IBM Fellow, Almaden Research Lab). From IBM developerWorks, XML zone. September 03, 2003. Excerpted from Chapter 2, "Influences on the Design of XQuery," in the book XQuery from the Experts: A Guide to the W3C XML Query Language (Addison-Wesley). "Early in its history, the XML Query Working Group confronted the question of whether XML is sufficiently different from other data formats to require a query language of its own. The SQL language is a very well established standard for retrieving information from relational databases and has recently been enhanced with new facilities called 'structured types' that support nested structures similar to the nesting of elements in XML. If SQL could be further extended to meet XML query requirements, developers could leverage their considerable investment in SQL implementations, and users could apply the features of these robust and mature systems to their XML databases without learning a completely new language. Given these incentives, the working group conducted a study of the differences between XML data and relational data from the point of view of a query language: (1) Relational data is 'flat,' organized in the form of a two-dimensional array of rows and columns. In contrast, XML data is 'nested', and its depth of nesting can be irregular and unpredictable... (2) Relational data is regular and homogeneous. Every row of a table has the same columns, with the same names and types. This allows metadata -- information that describes the structure of the data -- to be removed from the data itself and stored in a separate catalog. XML data, on the other hand, is irregular and heterogeneous... (3) Like a stored table, the result of a relational query is flat, regular, and homogeneous. The result of an XML query, on the other hand, has none of these properties. For example, the result of the query Find all the red things may contain a cherry, a flag, and a stop sign, each with a different internal structure... (4) Because of its regular structure, relational data is 'dense' -- that is, every row has a value in every column. This gave rise to the need for a 'null value' to represent unknown or inapplicable values in relational databases. XML data, on the other hand, may be 'sparse'...; (5) In a relational database, the rows of a table are not considered to have an ordering other than the orderings that can be derived from their values. XML documents, on the other hand, have an intrinsic order that can be important to their meaning and cannot be derived from data values. This has several implications for the design of a query language... The significant data model differences summarized above led the working group to decide that the objectives of XML queries could best be served by designing a new query language rather than by extending a re