Author: W. Eliot Kimber
Passage Systems Inc., kimber@passage.comMuch of the discussion of adding hypertext functionality to software systems has focused on application-level issues, looking at technologies such as OLE, OpenDoc, and Java as models for enabling application-level communication that could be used to implement and support hypertext functions.
While application-level interoperation will be needed in any open hypermedia system, it is, I think, a less urgent problem than the issue of data-level interoperation. Application-level interoperation schemes exist and can be used today. What is missing is an agreed upon data model for the data these applications to process.
In other words, hypertext is about creating and representing relationships among data objects, not about application functionality. It is about creating lasting databases of data and relationships. These databases must survive over long periods of time--human lifetimes--and it is a certainty that the tools of today will not be the tools of tomorrow. Therefore, focus on implementation issues such as application-level APIs, before coming to agreement on a general data representation scheme will fail to solve the real problems of open hypermedia.
The World Wide Web has demonstrated the power of data standards (HTML in the Web's case) to enable open hypermedia systems. The challenge as I see it is to apply this idea to a larger set of problems, a wider range of data types, and to do so with greater sophistication, flexibility, and generality. As even the inventors of the Web freely admit, the Web presents a simple and limited set of hypertext functions. We can and must do more.
Enabling the integration of hypertext functions with ostensibly non-hypertext applications (i.e., applications whose main purpose is not simply the creation of hypertexts) requires the development the following:
Given these three features, hypertext functions can be integrated with any applications that provide sufficient API to allow inter-application communication. Features and functions can be implemented independently of the data they might process. Data authors and owners can use these features and functions, secure that their data will continue to be useful no matter how the supporting software might change.
A basic premise of an open hypertext system is that it must be possible to create hyperlinks unilaterally among data objects to which write access is not available. In other words, it must always be possible to create hyperlinks that are independent of the data objects they connect. From this it follows that hyperlinks should always be conceived of and managed as first-class independent objects, at least for the purpose of defining general data models and management schemes. (Given a general independent link model, it is always possible to embed the link in the source data of one of its anchors. Likewise, given a sufficiently general model, it is always possible to recast a non-independent link as an independent link by explicitly addressing the anchor in which it was embedded.)
Thus, there is a general notion of a database of hyperlinks, each hyperlink representing a relationship between the objects it links together (the anchors of the hyperlink). Each hyperlink in the database must do two things: it must define and describe the relationship and it must locate ("address") its anchors. (This database is both conceptual and literal. Conceptual in the sense that the set of hyperlinks in an open hypermedia system is the link database. Literal in the sense that the source for these links must be stored and managed as a database with all the things that implies, such as data integrity, common access, access control, etc.)
The database of hyperlinks must have the characteristics of a traditional database. It must be in a neutral data format that can be accessed with a minimum of cost. It must be general enough to support unanticipated uses. It must provide sufficient expressive power to enable the description of relationships as richly and precisely as authors desire.
Because the hyperlinks must be able to address any type of data object, the addressing methods used to locate objects must be general and robust. They must be flexible enough to support unanticipated uses. They must be flexible enough to provide a variety of optimization and organization strategies. They must not assume any particular implementation, specific use, or communication protocol.
The SGML standard, ISO 8879, satisfies the requirements for the hyperlink database itself. It is a neutral data format that can be accessed with any text editor, as well as through more sophisticated access methods. It is very general. It provides unbounded expressive power. It has been proven in over ten years of constant and large-scale use.
The HyTime standard, ISO/IEC 10744, in concert with other standards and tools, satisfies the requirements for addressing. HyTime defines a general addressing mechanism that enables the addressing of any type of data. It is flexible and powerful and includes facilities for integrating non-standard and non-HyTime addressing methods into the larger HyTime addressing scheme. The HyTime standard also defines a general model for hyperlinks that is sufficiently general to allow the representation of almost any accepted hyperlink model. The HyTime standard is itself an application of SGML, defining a general architecture for hyperlinking applications.
By providing a robust data representation and a robust addressing model, HyTime is an attractive candidate for the data representation standard and basic data model that must underlie any open hypermedia system.
To be useful, any open hypertext system must provide a granularity of addressing lower than the file or storage object level. In other words, it is not sufficient to link to a document or a spread sheet--you must be able to link to a section, word, cell, or range of cells. In other words, linking is not about addressing simply storage objects but about addressing semantic or syntactic constructs within storage objects.
Providing this level of granularity is easy within a single application or suite of applications from a single vendor. However, it becomes much more difficult in an open system. The problem is primarily one of predictability: what are the addressable bits that a given data object can or will be parsed into? Only in the simplest of data formats is there no ambiguity.
The textbook example is the location of words in a sentence. There is no single definition of what a "word" is in English. It can be defined by looking for blanks, blanks and punctuation, grammatical or semantic boundaries (are bi-capitalized words one word or two or more?). What are the word boundaries in textual data that is not prose, such as code examples? Other natural languages may present even greater challenges. This means that addressing cannot actually be in terms of the structures in the original data. Rather it must be in terms of the objects that data is parsed into by a particular parser.
Further complicating the issue is that the same data may be parsed in different ways. For example, one application may want to address words while another addresses characters, a third addresses sentences, and a fourth addresses lines. In many cases no single parsed view of data can serve all uses of that data.
This means that there must be a general way for parsers and data processors to describe what the results of their processing will be. There must be a way to declare how you will, for example, distinguish words in an English sentence. Given such declarations, you can now predict what a particular location address will locate. In addition, you can compare two processors to determine if they will give equivalent results.
This general declaration method has to provide a general mechanism for describing how data objects will be parsed into processible objects (the "in-memory" data a processor actually works with, as opposed to the source representation stored on disk). In addition, it must define the general shape of these objects. It must define their basic data types, how they can be related, how they can be located, and how their properties and values are accessed.
In other words, there needs to be a general data abstraction method that makes it possible for disparate applications to pretend to each other that they all use the same in-memory abstractions so that they can work with each other's data reliably and predictably. Note that they do not actually have to have the same in-memory abstractions, they need only provide a mapping from their real internal abstractions to the general abstraction for communication purposes.
Thus, in such a hypertext system, all location addressing is ultimately in terms of this generic abstraction. By providing a data view using this abstraction, any data type can be integrated into the system. The functions available for that data type are limited only by the depth to with the processor that provides the view goes. For example, for a vector graphic format, a processor might only provide a simple node list or tree view of the graphic objects, sufficient for simple structural addressing, or it might provide a more sophisticated view where each object has a unique name and descriptive properties, enabling name-based and semantic addressing in addition to simple structural addressing.
This data abstraction also enables the integration of any addressing method by mapping it to one of the standard addressing schemes.
Thus, by defining a relatively thin layer of abstraction between data processors and their own internal representations of the data they process, any application or data processor that wants to participate in the larger system can do so simply by providing the necessary abstract data view.
Note that this is not the same as an API. Any inter-process communication method can be used to communicate in terms of these abstractions. In addition, the abstractions need only be used where the applications communicating are not well known to each other.
The HyTime standard defines such an abstraction method, which it calls "groves". HyTime, and its companion standard, DSSSL (Document Style and Semantic Specification Language, ISO/IEC 10179) had to have such an abstraction in order to have a complete specification of their own functions and semantics. While it may need refinement, the grove mechanism appears to provide the necessary abstraction with the minimum amount of overhead.
By focusing on data standards, specifically, standards for representing the source form of hyperlinks and location addresses, and by defining a general abstraction for processed data, it becomes possible to design and implement open hypertext systems without the need to worry about the details of inter-application communication or application-specific or proprietary data formats up front. The key is defining and standardizing sufficient layers of abstraction to enable the ad-hoc construction of systems from different parts without constraining functionality or implementation strategies (beyond the need to communicate through and correctly reflect the standardized abstractions).
The SGML and HyTime standards define all or most of the necessary abstractions. As enabling standards, they are explicitly designed to be general, flexible, and non-limiting. These standards can be supported and implemented using common computer processing techniques and existing technologies. As International Standards, they protect data owners and authors from attempts by vendors to usurp or control data standards for their own ends.
Thus, any discussion of the approach to open hypertext systems should take as its first task coming to agreement on standardized data representation schemes for hyperlinks, standardized models for hyperlinks, standardized addressing schemes and syntaxes, and standardized abstractions for processed data to facilitate inter-process communication. Once these standards have been agreed upon, the task of designing and implementing such systems becomes a straightforward engineering task, albeit a challenging one.