An announcement from Sleepycat Software Inc. describes the general availability of Berkeley DB XML, a data management solution based upon Berkeley DB. Berkeley DB XML "is designed for professional software developers who need a transactional, recoverable data manager for native XML and semi-structured data that is fast, cost-effective, and flexible. The software was validated during a 12-month beta program by more than 5,000 companies, including BEA Systems, Boeing, EDS, Leadscope and TELOS. It provides a high-performance, extremely reliable embedded database engine that stores and manages XML data. Berkeley DB XML stores and retrieves native XML documents, so no conversion to relational or object-oriented models required. It combines XML and non-XML data in a single database, with flexible indexing features that give developers control over query performance and the ability to tune retrieval speed. The Berkeley DB XML software supports XPath 1.0 and other W3C standards for XML and XML Namespaces, accepting UTF-8 encoded documents and XPath expressions. It inherits advanced database features from Berkeley DB, including concurrent access, transactions, recovery, and replication." The source code is available for free download, testing, evaluation and development.

Overview of Berkeley DB XML

"Berkeley DB XML is a high-performance, extremely reliable embedded database engine that stores and manages XML data. It is built on top of the award-winning Berkeley DB, which provides fast, reliable, scalable, and mission-critical database support."

"Berkeley DB XML is supplied as a library that links directly into the user's application. This provides superior performance by eliminating communications among processes or systems and delivering a truly embedded XML data management system."

"Documents are stored in collections. A single application may operate on many collections at the same time. A single application may also combine data from different collections easily. Non-XML data may be included by creating standard Berkeley DB tables. Tables and collections may be used together, with full support for Berkeley DB transactions and recovery services, by multiple users simultaneously."

"Berkeley DB XML enables fast look up by allowing individual collections to be indexed differently. This allows Berkeley DB XML to speed up the common queries over particular collections. Each collection supports multiple indexes. A wide variety of available indexing schemes support different XPath queries efficiently."

"Berkeley DB XML's Query Processor implements XPath 1.0. A cost-based query optimizer considers the indices that exist, the data volume that a query is likely to produce and the cost of computation and disk I/O to select a query plan with the lowest run-time cost. Results may be computed eagerly or lazily. Lazy evaluation makes it easy to stream results to the user..." [adapted from the product description]

"DB XML is an embedded database that is tuned for managing and querying hundreds, thousands, or even millions of XML documents. The atom of storage for DB XML is the document, which is stored as a string in the underlying database. For this reason, DB XML works best with small documents (that is, documents under a megabyte in size), although it can be used to improve data access for large documents as well."

DB XML provides a series of features that makes it more suitable for storing XML documents than other common XML storage mechanisms. DB XML's ability to provide efficient, indexed queries means that it is a far more efficient storage mechanism than simply storing XML data in the filesystem. And because DB XML provides the same transaction protection as does Berkeley DB, it is a much safer choice than is the filesystem for applications that might have multiple simultaneous readers and writers of the XML data."

"Moreover, because DB XML stores XML data in its native format, DB XML enjoys the same extensible schema that has attracted many developers to XML. It is this flexibility that makes DB XML a better choice than relational database offerings that must translate XML data into internal tables and rows, thus locking the data into the database's inflexible schema..." [adapted from the Getting Started Guide]

From the Announcement

"Berkeley DB XML helps companies get to market quicker with faster performing, lower TCO solutions for their customers. Our customers can rely on an established and profitable partner like Sleepycat to be there in the future," said Michael Olson, CEO of Sleepycat Software and one of the original co-authors of Berkeley DB. "Unlike many newcomers to the XML market, we have a proven track record of meeting mission-critical requirements for Global 2000 corporations. And with so many companies testing Berkeley DB XML in beta, clearly our no-compromises XML solution tapped into considerable pent up demand."

Unlike legacy relational databases with XML extensions to convert native XML data into a relational structure for storage and to re-construct the XML upon retrieval, Berkeley DB XML provides true native XML storage, eliminating the overhead of conversion and re-construction. Compared to other XML databases, Berkeley DB XML is more fully featured, including support for replication and the XA interface for distributed transactions. It is based on the proven Berkeley DB engine that first shipped in 1991 and today boasts more than 200 million deployments at some of the world's largest enterprises and service providers.

"The native XML database market has not yet fully established itself as a strong long-term niche in the database market, and therefore native XML vendors need to demonstrate a compelling value proposition to not only differentiate their offering from relational enterprise database vendors but also stimulate demand and stay in the forefront of today's Web-enabled and Web-service-oriented database technologies," said Wayne Kernochan, vice president at industry research firm the Aberdeen Group. "Sleepycat Software is well positioned to seize this opportunity with its new native XML offering built around the well-established Berkeley DB engine."

Among the thousands of companies participating in Sleepycat Software's beta program for Berkeley DB XML were global systems integrator EDS and Canadian telecommunications company TELOS.

"Berkeley DB XML gives us carrier-grade scalability and reliability for our Mobile Switching Centres that need to manage up to 600,000 mobile phone accounts in real-time," said Steve Williams, product development director at TELOS, provider of telecommunications infrastructure to network operators and manufacturers. "We wanted a transactional, recoverable, native XML data store that scales and we knew we could count on Berkeley DB."

"We needed an XML solution that was flexible, extensible and easy to manage." said Dan Hatfield, Senior Consultant, EDS. "Berkeley DB XML met these requirements and integrated easily into our IT framework."

Berkeley DB XML has also garnered support from industry-leading platform software providers such as BEA Systems who are helping their customers build XML-based solutions across the enterprise. "Customers using XML and Web services for application integration will need native XML data managers like Berkeley DB XML for fast, reliable data persistence," said Adam Bosworth, vice president of engineering of BEA Systems.

Tim Bray, chief technology officer of Antarctica and one of the authors of the original XML specification while with the W3C, said "Berkeley DB is at the center of a lot of really excellent applications, particularly in the Web context, with a reputation for being solid and fast. I am delighted to see that they are addressing XML seriously with Berkeley DB XML."

Sleepycat Software makes Berkeley DB, the most widely-used application-specific data management software in the world with more than 200 million deployments. Customers such as Alcatel, Amazon, British Telecom, Cisco Systems, EMC, Ericsson, Hitachi, HP, IBM, Motorola, RSA Security, Sun Microsystems, TIBCO and Veritas rely on Berkeley DB for fast, scalable, reliable and cost-effective data management for their mission-critical applications. Profitable since it was founded in 1996, Sleepycat is a privately-held company with offices in California and Massachusetts.