Over the past ten years, SGML has achieved great popularity. But not without some bad publicity. Comments heard in the past include: "We're glad we did it, but it did take us longer than we thought"; "We had to change our DTD late in the project and that cost us quite a bit"; "The product didn't work the way we had thought it would, so we had to redesign part of the system"; and so on. SGML is a relatively new tool, so it is still highly vulnerable to being implemented in ways that confound rather than assist the user. Because it is so vulnerable, it is important to continuously monitor project activities against business requirements and keep focused on the project's ultimate objectives. The use of SGML can yield great rewards. As business people, we must decide how much we are willing to pay for those rewards.
In many ways, SGML systems are like every other IT system. But in some ways, they are unique. At SGML meetings and conferences for the past 10-12 years, we have heard case studies of how various organisations have implemented SGML, the stages they went through, and the degree of success or disappointment they have had. Quite often, the amount of disappointment is unexpected, and some areas where success is achieved are also been unexpected. SGML systems have their own unique types of pitfalls and exhilarations. Knowing what to expect and when to expect it, keeping management committed, and keeping a project on course have been key themes in numerous case studies. I would like to explore these areas with a slight twist on the traditional Return on Investment analysis, something I call the Pain/Gain Ratio.
Any new SGML system to be developed and installed within a company will have been planned to some extent. As part of that planning, most companies will do a Return on Investment (ROI) analysis. An ROI analysis is used to determine how much the system will cost, how soon the system will pay for itself, and how much might be gained by the company over its planned lifespan. The "Return" in an ROI is the expected "gain"; the "Investment" is the "pain". Usually, an ROI is done during the feasibility stage of a project. The numbers are carefully gathered, added, and subtracted; chins are rubbed; sums are discussed; numbers are changed; and eventually, either an acceptable ROI is arrived at or the project is abandoned.
The ROI is published and passed before the Board of Directors, Tender Board, or other decision makers. If accepted, the project goes forward and the ROI justification goes on a shelf in the accountant's office.
Time goes by. The system is procured, built, installed. Many factors in the initial feasibility study are adapted over time for changing conditions. At various phase milestones, the accountant takes the ROI off the shelf and looks to the actual experience of the system procurement, creation, or installation. Often, the scenario anticipated in the ROI is unrecognisable in real life. If the system is returning a reasonable gain to the organisation, it is difficult to detect. The pain quotient at various project phases might be so high that hopes for gain are dashed, plans are downsized, and morale slumps. What might have been a normal fluctuation in the pain level is seen as an almost unmendable rip in the project. Ironically, the steps taken to mend the rip may wind up ensuring that future gains will never meet expectations.
This paper proposes the Pain/Gain Ratio as an addition to a conventional ROI or Cost/Benefit Analysis. It is a less formal concept that can be used by all team members throughout the project to determine if a project is truly off-course or merely undergoing a hiccup. It keeps the ultimate objectives of an SGML project clearly in front of the development team, contractors, and management, which helps to keep pain in perspective. Pain/Gain Ratios can be used to plan an SGML project, monitor it throughout its lifecycle, and to measure its ultimate effectiveness.
This presentation will examine various aspects in an SGML project lifecycle and discuss areas where pain and gain may be outside the expected boundaries.
The Pain/Gain Ratio is a concept more than a strict mathematical equation. One can think of two thermometers sitting next to one another. One measures pain; the other measures gain. Their readings make up the Pain/Gain Ratio. The measurement scales are up to the organisation. It is important only that the same scales are used to measure both pain and gain and that the meaning of the scales are well-documented.
For example, the pain thermometer may have a reading of 6 and the gain thermometer may have a reading of 12. The ratio is then 6:12 or 1:2. Is 1:2 a good ratio? Is it great?
The remainder of this paper explores a variety of ways the Pain/Gain Ratio can be used and interpreted by an organisation.
Simply put, there is no "right" Pain/Gain Ratio. In an industry where margins are extremely tight, a ratio of 1:1.1 might be seen as highly successful. Conversely, in an industry where margins are greater, anything less than a 1:10 ratio might be seen as a failure. The ratio for tangibles might be taken directly from the ROI. But a company might also have an ideal ratio for intangibles.
There is no limit. As mentioned above, there will probably be overall tangible and intangible Pain/Gain Ratios for the entire project. But there may be many more. Each phase of a project may have its own Pain/Gain Ratios. For example, the DTD analysis phase may have a Pain/Gain Ratio of 5:2. The loss is planned and expected. It's only if the gain index falls below 2 that there is a need for damage limitation.
Individual tasks might have Pain/Gain Ratios. Even team members may have Pain/Gain Ratios. The point is for the entire SGML project team to understand what is being expended (time, money, resources), and what is being gained at each point in the cycle. In the example above, the DTD analysis might be seen as a "lost leader" to information reuse. In other words, more time is spent investigating various ways to create the DTD based on a range of reuse scenarios. If the benefits to be gained from a detailed analysis are not understood, there might be the temptation to skimp on that analysis. The cost of skimping in terms of lost gain could far exceed the cost of doing the analysis. In other words, the "where" and "when" of the "gain" are just as important as the "how much".
The following are sample costs when creating a new system.
Cost of doing the analysis (includes business, system, and document analysis; this is usually skimped on, resulting in downstream pain rather than gain);
cost of training employees (getting people up-to-speed on new system, on new workflow, and on SGML);
cost of new software (mainstream, publishing, groupware, and SGML software);
cost of new hardware (includes networks, servers, and client machines);
cost of consultancy and integration services (a trade-off or addition to getting in-house expertise);
cost of running two systems in parallel for some time (redundancy is a safety measure until shake-out of new system);
cost of the legacy data conversion (questions arising here include: will this data be used again, should the new system only use new data, should the same DTD be used for legacy as new data);
time lost on non-existent requirements (vetting requirements is a necessary step to ensure no time is spent building something that won't be needed);
training subcontractors (a costly but possibly worthwhile investment in the long run);
cost of the new system installation (physical plant, man-hours for the installation);
Intangible pains are not so easily measured, but have a noticeable impact, nonetheless.
missed deadlines (tends to deteriorate relationship with customer and lowers company image);
poor morale, employee disillusionment and fear (the adoption of SGML, like any change, can be threatening to employees; good change management skills are needed);
lost of goodwill (can chip away at the worth of the company);
ineffective training (training on SGML too early or providing technical training to those who won't need it not only wastes staff time, but can also make staff unnecessarily apprehensive);
starting new way of working (interchanging data) with subcontractors, prime contractors, vendors, etc. (communication skills or the lack of them can spell success or failure here)
Measurable gains include:
reduction in staff costs (this may not be seen as a gain in the department loosing the staff member(s), but can be a significant benefit to the company; alternatively, creating more information products with the same number of staff can also be advantageous);
reduction in production times (gains are usually made here if the SGML information is captured early in the production cycle, helping to streamline the workflow);
reduction in production costs (usually results as a consequence of reduced production times and streamlined workflow; examples include: reduced need for proof-reading cycles, fewer production line steps, etc.);
products in more than one media for low incremental cost (a traditional strong point for SGML);
reduced paper costs (as customers become more familiar with the electronic document, demand for paper documents tend to decrease; in some cases, the improved functionality of the electronic document makes paper documents highly unattractive except where electronic delivery is unfeasible);
reduced documentation delivery costs (lower weight of electronic media means lower postal costs; use of the Internet and other electronic communications also means lower costs);
reduced production of change materials or service bulletins (improved workflow and shorter production times means that the closing date for changes can be moved closer to the publication date, reducing the number of service bulletins necessary on day one; alternatively, the ability to publish more often due to reduced production and delivery costs also means less need for change materials);
new products with minimal investments due to data reuse (another traditional stronghold for SGML; predicated on rigorous markup and intelligent database management to facilitate finding existing data);
Less measurable gains include:
increased goodwill with customers (due to a number of factors including reduced need for updates/service bulletins, more accurate information, meeting deadlines, etc.);
in better position with customer/user (due to factors listed under increased goodwill with customers, the company may be in a better negotiating position on future contracts);
shorter time to market with new products (depending on the industry, this may also be seen as a tangible gain);
improved employee confidence (being able to handle the new system and understand SGML will provide one level of increased confidence; knowing that they can beat previous production and quality records will provide another level);
One of the most costly decisions a company can make is for the use of content-specific element types rather than content-generic ones. For example, instead of calling each major container in the body of a document a "chapter", a separate element type is made for each instance of a chapter, such as "introduction", "scope", "problem-statement", "solution", "conclusion". A company must be sure that there are sufficient reasons for information use downstream to warrant this level of investment in analysis and system development. If a document must not deviate from a government's or regulatory authority's guidelines, content-specific markup may be an excellent way to ensure that each of the guideline's topics are discussed in an appropriate order. If contextually-constrained searches are required, content-specific markup can be of great use. Other situations would also require this type of markup. Diminishing returns, i.e., more pain than gain, result when the end requirements are not truly requirements, when the downstream systems cannot exploit the content-specific markup, or when the size of the project exceeds the ability of the project team to manage it.
This example concerns a parts catalogue for an automotive manufacturer. The staff who had been putting together the catalogue for many years had really felt that their job was creating the book. They were very concerned with the format and look of it this catalogue which was a listing of parts. When the SGML system was put in, a lot of the formatting responsibility was removed from these staff members because format could now be applied more consistently by the automated system. At first it seemed as if these staff members would have less job satisfaction or, in fact, that there would be less need for their jobs at all. But what actually happened was a somewhat unexpected gain. They became more involved with the decisions about what parts were going to be offered and what assemblies parts would be used in. They also now had the time to ensure the accuracy of the information and took more responsibility in the cataloguing activities rather than in the formatting activities. The result was increased job satisfaction for the staff and a greatly improved catalogue for the company. The increased accuracy in the document led to fewer orders for incorrect parts, fewer returned parts, and more accurate stock levels. All of these contributed to lower costs and greater savings.
Having stated that we need well-thought-out and well-developed requirements statements, I would like to discuss an example that shows how requirements can significantly affect the DTD design. This example is of organisational level electronic documentation for aircraft maintenance manuals. By organisational level maintenance, I mean that which takes place on the aircraft itself. It primarily comprises the identification and exchange of Line Replaceable Units (LRUs).
A principal business requirement was to reduce loss flight time due to maintenance. This business requirement drove many areas of the business, documentation being just one of them. The question to the documentation group was: How can the documentation be changed/enhanced/etc. to reduce maintenance time.
A principal system requirement was to deliver the documentation electronically. The documentation group was also faced with the questions: How can we transition from paper to electronic documents without alienating our users? In fact, how can we exploit an electronic environment to help reduce maintenance downtime?
Studies conducted by the US Air Force and other organisations have found that between 30-40% of a maintenance technician's time is spent looking for the information. Enhancing the ability of the technician to find the information could have a significant impact on maintenance time.
The organisational level maintenance documentation was traditionally held is a number of different types of documents: theory of operation, fault isolation, procedures, wiring and schematic diagrams, and illustrated parts lists. In the paper world, each of these types of information was published separately. A maintenance technician might need to take 6-10 documents out to the flightline to perform a maintenance function.
Had simple document analysis techniques been applied to each of the document types within organisational level maintenance, the document analysis activity would have yielded six DTDs. However, the business needs were examined including the need to reduce maintenance time and to deliver electronically. Based on those two business and system requirements, it became more obvious that the organisational level maintenance data should be looked at holistically rather than as separate traditional documents. In other words, an aircraft was seen as a system. It might have theory of operation information, graphics, and procedures relating to the system as a whole and then break down into subsystems. Each of those subsystems could in turn have procedural steps, theory of operation, etc. and then break down into subsubsystems, and so on. By modelling the problem domain (documentation for an aircraft) rather than the publication domain (e.g., fault isolation procedures), the information set was now re-organised into one DTD rather than six. Further, that DTD concentrated on the concept of a system/sub-system breakdown similar to that used for the identification of different maintenance areas. This made the arrangement of the information more compatible with the way the aircraft is described.
From this example, we can see that understanding business and system requirements can significantly impact how a DTD is designed at a very high level. Understanding what you are trying to do, who is going to do it, and when it has to be done, will have an impact on your SGML application at all stages and at all levels.
Taken in isolation, the time needed to develop this new, single, highly content-specific DTD might seem like too much pain. Taken in context, it can be seen to be one of the best ways to get the anticipated gain of reduced maintenance downtime.
SGML systems, like any other new IT system, will always extract some pain from the implementors. By developing strong, qualified requirements ("gain") statements; determining the degree of "pain" each is worth; communicating those throughout the project team and upper management; translating them to measurable and tractable items; and monitoring them regularly, companies can ensure that they achieve a favourable Pain/Gain Ratio for their SGML system.
Pamela Gennusa is a Director of Database Publishing Systems Ltd where she leads the consultancy, development, and conversion service activities. She has served as consultant for SGML-related application in the oil, pharmaceutical, telecommunications, and defence industries. She has participated on both the ANSI and ISO committees responsible for SGML and, until 1992, she served as Co-Chair of the CALS Industry Standards Group Electronic Publishing Committee. In 1992 she became President of the international SGML Users' Group and sits on the Board of Directors of the Graphic Communications Association. She served on the first Board of Directors of SGML Open as Chief Marketing Officer and President, 1993-1995.