REENGINEERING LEGAL PUBLISHING: AN EVOLUTIONARY APPROACH

[Mirrored from: http://www.textscience.com/w6paper.html. Please see the canonical URL.]

The following paper was presented at the SGML '94 Poster Session. There is an accompaning poster which describes various strategies for mapping Word 6 styles to SGML.

Word 6 for SGML Authoring?

Many organizations are considering using Word 6 for SGML authoring now that Microsoft's SGML Author is on the immediate horizon. We'd like to offer some reflections on our decision to use Word 6 and our experience with conversion to SGML with respect to our overall philosophy of incremental, evolutionary project engineering.

If Word 6 could be used to produce SGML it would be a very attractive choice for our project for at least one negative and one positive reason. The negative reason is that SGML-savvy writers and editors are rare. It is difficult to ask in-house editors, reluctant even to move from Word Perfect to Word, to learn the very different skill of using an SGML editor. It is impossible to require SGML submissions from authors who, if nothing else, can not be expected to have the necessary software. We can, however, ask our staff and authors to use styles which can be turned into basic SGML. As they become adept at this they are also mastering some of the thinking which goes into using SGML. The positive motivation is that we believe that Word is a great document authoring tool, reflecting the effort poured into its development.

We found that Word 6 can be used for authoring SGML through the use of styles. We have described some useful techniques in our poster and we have pointed out some of the limitations and pitfalls. Bottom line: You can, in principle, do most of things you might need to do if you can be flexible in the design of both your stylesheets and DTD to ensure that they work together. Stylesheets can be simplified (lessening the burden on authors) if the conversion scheme includes regular expression recognition, variables, and control statements. We wrote an RTF (Rich Text Format, i.e., Word ASCII) conversion engine using the freeware program Perl, which also executes conversion scripts written in Perl. We have not had the opportunity to try SGML Author yet, but we believe that it will both be useful and not capable of entirely supplanting our use of Perl. The main advantage of SGML Author is its ability to create the SGML file and parse it within Word with a feedback loop for correcting errors. Correcting errors may not always be easy since some errors may make sense according to the DTD but not make sense to the author using styles. But SGML Author is a GUI and is not designed to have the expressive power of a language like Perl, as Microsoft has emphatically stated. We therefore envision using SGML Author in a two stage conversion. The first involves using SGML Author to create SGML conforming to a DTD expressly designed to simply the work of the author with the application of styles and the correcting of errors in conversion. The second stage would be a DTD-to-DTD conversion where the second DTD is derived from the first, but with a fuller breakdown of elements, clearer demarcation of structure, and rearrangement of structure where required. Any of the language-based SGML conversion tools would be very well-suited for this task.

The above scenario assumes that the target DTDs will still be based on a document architecture, that is, the implicit structure of the Word document being converted. A document architecture is less useful than an information repository architecture, based on the division of knowledge of the domain represented, as the source of multiple, non-paper based applications. Nonetheless, today, we are working with document architectures. The spirit of our legacy data haunts the first stage of our project and we are, in consequence, still using models originating out of paper-based production. There are good reasons for going only this far at first, rather than aiming for the ultimate repository architecture in one fell swoop. SGML itself, the standard, tools, and applications, are changing as we speak. Our work in the field of law is deeply interrelated to the flow of information from the courts and legislatures, as well as other legal information providers. We hypothesize that wider acceptance of SGML in our field, the law, and the resulting increased interoperability of electronic legal information resources will have a broad and unifying effect on law databases. Finally, our self-knowledge has grown in the course of targeting and reaching the first stage which better enables us to architect the next stage.

When we move away from a document architecture to a information repository architecture will we have to leave Word by the wayside? After all, Word is designed to create visually appealing documents, not write fields into text databases. There are three arguments militating against the obvious. First, because Word is one of the most widely used text editors, with a plethora of features designed to make editing easy, it is an appropriate choice for writing SGML tagged documents. This is the position taken by Nice Technologies with their SGML Tag Wizard product. The second argument is that as long as human beings write text, visually appealing documents will be the most appropriate vehicle for the expression of ideas and the information repository architecture will forever remain a holy grail. The third argument is that improvements to the structured information capture abilities of WYSIWYG editors will make them viable tools for authoring into information architectures. It is to this third point that we'd like to throw out a few ideas on some things that Microsoft could do with Word and also some things we could do with SGML to make it more WYSIWYG-friendly.

• Nesting, style wizards. Allow styles to be defined as nested and preserve context information in conversion. Optionally validate nesting. Style wizards could provide context-sensitive editing assistance. We envision a Word implementation of these features as a kind of 'lite SGML': more lenient than SGML editors but providing structure-derived assistance to authors where it is particularly useful.
• Style Annotation. Allow styles to be 'annotated' to provide the equivalent of attributes in SGML. Combined with Word fields for generating or prompting for data could be a means of capturing attribute information which is at least equal to anything found in SGML editors today.
• Hypertext and multimedia features accessible. Word has many of the features of sophisticated hypertext, hypermedia viewers like media and application object-embedding, links, bookmarks, index generation, cross-referencing, dialogs, notes, TOC generation, forms, and dynamic fields. SGML is particularly important to applications employing these kinds of features because it gives an independent way of expressing the relationships within textual data. There has not been, however, any consensus on how to author data employing such features, and we are mostly constrained to working directly with tags which translate into the appropriate concept. Word, on the other hand, has good facilities for hypertext and multimedia authoring, but it is difficult (not impossible) to extract this information for conversion to SGML. The ability to directly map between these features in Word and SGML could make Word the most accessible hypertext, hypermedia authoring tool available today.
• Authoring for document-independent repositories. In many cases it may be an unnecessary complication to force an entire document into the structure of a single DTD. Different parts of a document may belong to different parts of the information architecture. Some sections may be unstructured. Here it may be SGML which is more at fault since it demands that the document conforms to a complete description of it and deals poorly with multiple and/or concurrent data descriptions (poorly because it does make an effort through SUBDOC and CONCUR options). From the authoring end, this kind of application could be addressed by allowing the mapping of document elements to individual destinations in the information repositories, i.e., specifying the DTD and perhaps an actual location by some criteria within the repository. We may also wish the authoring tools to generate the DTD from a document constructed out of styles with DTD fragment mappings. This is a means both to prototype document structures and to transmit models of documents to other applications so that they can process them.
• Inheritance. Word styles support inheritance, that is, properties of a style are inherited from the style it is based-on unless overridden by that style. Although SGML currently lacks any definitional support for this behavior, it is being used more and more often in SGML markup schemes. Making this information accessible when mapping the Word document to SGML could be useful in two ways: inheritance information could be imported directly into SGML markup which supports this, and, parentage could be used to actually determine what should be done in the conversion.

We are very heartened by the appearance of SGML Author and we hope that it will spur creative thinking and debate on the topic of WYSIWYG editing and SGML. We would love to see our investment in Word today pay off when we move our project to the next stage and the next level DTDs. If this hope proves vain we will retool, doing so when we are in a good position to ensure a solid return on our investment in new technology.