Jade/DSSSL future

Date:       Mon, 24 May 1999 14:41:09 +0700
From:       James Clark <jjc@jclark.com>
To:         dssslist@mulberrytech.com
Subject:    Jade/DSSSL future

Note: This note from James Clark is part of a wider conversation about Jade/DSSSL held on the (Mulberry) DSSSList; see the May/June 1999 threads with subjects "Jade Maintenance," "Jade/DSSSL future," and "More on the Future of DSSSL."

Adam Di Carlo wrote:

> Honestly, I haven't seen
> any sort of comprehensive analysis of what is wrong with Jade at this
> time, and what priorities should be. AFAIK, I have seen the following
> pointed out, and this is how I would prioritize it:

My priorities would be a bit different.

> * some misc DSSSL features and operators missing (there was a patch
> submitted on this list which looked pretty good to me)

A lot of these can be trivially implemented with the existing operators, and there's little performance benefit in implementing them in C++, in fact there may be a loss because of code bloat. A better solution for such operators would be an ability to automatically preload a startup file with function definitions (you have to be careful to ensure that these preloaded definitions work even when other primitives get redefined).

I agree that it would be nice to have the operators that cannot be defined in terms of the others, but I don't think this will make a big difference to the overall utility of Jade.

> * the proper DSSSL transformation language is missing (I'm not sure
> if this is worth implementing -- it would seem that DTD-to-DTD
> transformations are possibly better handled with XSLT, although
> for SGML users it would need to be accompanied by some SGML->XML
> conversions as well)

I think this would be almost completely useless. The jade -t sgml approach is better, and XSLT is better yet.

> * some major DSSSL advanced features are not implemented, the
> biggest one brought up to this list being the
> complex-page-sequence stuff...

There's not much point in implementing full page-sequence flow object in the front-end unless it can be supported in the back-ends. None of the existing backends could support the complex-page-sequence. What's really unique about Jade is the ability to get multiple formats from a single stylesheet.

> * TeX backend may need a re-write (see below)

Yes. It would be better to do more work in the backend and less work in macros.

> * jade is slow (is this even fixable?)

I don't think it's slow considering what it's doing.

It has a full programming language and there's no way you can stop people writing slow programs in any language. You need to look at the combination of jade plus the stylesheet you're interested in (I guess it's DocBook) and figure out how the combination can be faster. Part of the problem is that there are no profiling tools in Jade: there's no way for a stylesheet author to figure out why their stylesheet was slow. So some way of profiling Jade would be great. It may just be a matter of optimizing the stylesheet, or the solution may be to add some additional primitives to Jade to handle the things that are slow. Another thing that could help is some documentation explaining what things are slow and what things are fast.

SGML also brings a lot of overhead to Jade. A DSSSL stylesheet has to be parsed as SGML, the DTD has to be parsed, all the content checking has to be done, then the meta-DTD also has to be read and that has to be parsed, and the architectural document has to be created; only then does the real DSSSL parsing start. A version of Jade that used an XML parser rather than SP and in which DSSSL files didn't have an SGML architectural wrapper would be a lot faster.

> * NROFF backend would be nice (note that James himself wrote GROFF --
> I wonder what James thinks about this. Personally, I think its
> kinda low priority, since HTML -> ASCII with lynx works decently)

I think a decent HTML+CSS backend would be more useful. The current one was really just an attempt at a proof of concept; I did only enough to discover that it wasn't going to be practical with CSS implementations as they then were. But CSS implementations are at last starting to get to the point where they're useable. In particular Mozilla's is excellent. It should be possible to do an HTML+CSS backend which would get you browseable output from the same stylesheet that you use for the RTF or TeX backend. You wouldn't have to write a separate HTML-specific stylesheet.

Apart from the things I've mentioned, my priorities would be:

- Making the backends useable independently of the rest of Jade. Create a small program (using an XML parser like expat rather than SP) that reads the FOT file and calls an FOT backend. This would require some tweaks to the FOT format and to the FOT interface. The point of this is that it would then be possible to use XSLT with DSSSL flow objects.

- Make it easier to use Jade for XML. At the moment the fact that the DSSSL stylesheet is an SGML document makes it awkward to use Jade with XML. There are several possible solutions here and I'm not sure what the best is.

- Support XSLT in Jade. XSLT has an xsl:functions element that allows XSLT stylesheets to include extensions written in arbitrary extension languages. With Jade, that extension language would be the DSSSL expression/query language. This is probably too much work.

My general feeling is that just as the future is XML not SGML, so the future is also XSL not DSSSL. When XSLT and XSL are done, there will (I hope) be nothing you can do in DSSSL that you can't do with XSL(T). DSSSL has not achieved widespread acceptance, and of course that's disappointing to all of us involved in DSSSL. But I think we have to face up to the fact that the main reason it has failed to achieve acceptance is not because of lack of promotion or explanation or marketing (although that has probably been a factor), but because of real useability problems in the language itself. XSL started off as a collaboration between DSSSLers and Microsoft to create a new syntax for the DSSSL style language that would be easier to use and could achieve wide acceptance. It hasn't quite turned out like that. On the XSL flow objects side, politics and market realities have necessitated building DSSSL functionality on top of CSS formatting objects/properties rather than starting with the DSSSL flow objects. On the XSLT side, as we've continued to work on the language we've found many ways to improve it, and the language has evolved substantially from DSSSL; however it's still very much the same approach to transformation as jade -tsgml. There are big advantages to being in the mainstream, and XSLT looks set to become a mainstream technology. It's attracted the support of some of the biggest players out there -- Microsoft, Oracle, IBM, Sun, Lotus, Adobe (look at the list of WG members in Acknowledgements section of the XSLT WD) -- and they have collectively invested millions of dollars helping to develop it and the language has benefited enormously from this. I have sometimes wondered whether I wasted five years of my life doing DSSSL and Jade, but I comfort myself with the astonishing fact that a little bit of their spirit now seems likely to live on in, of all places, Microsoft Windows.

So what does this mean for my involvement in Jade? I think the reality is that although there are improvements that I would dearly like to make, it's unlikely, at least in the near future, that they are going to get to the top of priority list, and I don't want to stand in the way of others who wish to make improvements.

I do have one request: if you release something independently of me, you call it something other than simply Jade, but preferably a name that acknowledges Jade; maybe OpenJade or something like that.

I also have one strong recommendation on how to proceed with Jade development. Before you start hacking the code, create some documentation on Jade internals. Jade was not designed for a bazaar development model and there is no internals/design documentation. But there is an internal architecture that was designed with a lot of care, and unless those hacking on Jade really understand it, reliability and quality is likely to suffer. For example, when writing extension functions there are rules you have to follow to make the garbage collector work; if you don't follow them everything will compile and appear to work at first but you will get seemingly random, unreproducible crashes on large stylesheets and documents; fixing this kind of bug has taken me literally days in the past. To hack Jade, you really need to know what you're doing. I don't have the time to write the internals docs myself, however I am willing to take the time to explain it to somebody else if there is somebody who (a) has read the code and has a good level of understanding already and (b) promises to write up an internals document.

I am also willing to make all my RCS files available, which I would suggest be used to start a public CVS repository.

James
(speaking for myself alone not the DSSSL WG and not the XSL WG)

DSSSList info and archive:  http://www.mulberrytech.com/dsssl/dssslist

Prepared by Robin Cover for the The SGML/XML Web Page archive.

SEARCH Advanced Search ABOUT Site Map CP RSS Channel Contact Us Sponsoring CP About Our Sponsors NEWS Cover Stories Articles & Papers Press Releases CORE STANDARDS XML SGML Schemas XSL/XSLT/XPath XLink XML Query CSS SVG TECHNOLOGY REPORTS XML Applications General Apps Government Apps Academic Apps EVENTS LIBRARY Introductions FAQs Bibliography Technology and Society Semantics Tech Topics Software Related Standards Historic	Jade/DSSSL future Date: Mon, 24 May 1999 14:41:09 +0700 From: James Clark <jjc@jclark.com> To: dssslist@mulberrytech.com Subject: Jade/DSSSL future Note: This note from James Clark is part of a wider conversation about Jade/DSSSL held on the (Mulberry) DSSSList; see the May/June 1999 threads with subjects "Jade Maintenance," "Jade/DSSSL future," and "More on the Future of DSSSL." Adam Di Carlo wrote: > Honestly, I haven't seen > any sort of comprehensive analysis of what is wrong with Jade at this > time, and what priorities should be. AFAIK, I have seen the following > pointed out, and this is how I would prioritize it: My priorities would be a bit different. > * some misc DSSSL features and operators missing (there was a patch > submitted on this list which looked pretty good to me) A lot of these can be trivially implemented with the existing operators, and there's little performance benefit in implementing them in C++, in fact there may be a loss because of code bloat. A better solution for such operators would be an ability to automatically preload a startup file with function definitions (you have to be careful to ensure that these preloaded definitions work even when other primitives get redefined). I agree that it would be nice to have the operators that cannot be defined in terms of the others, but I don't think this will make a big difference to the overall utility of Jade. > * the proper DSSSL transformation language is missing (I'm not sure > if this is worth implementing -- it would seem that DTD-to-DTD > transformations are possibly better handled with XSLT, although > for SGML users it would need to be accompanied by some SGML->XML > conversions as well) I think this would be almost completely useless. The jade -t sgml approach is better, and XSLT is better yet. > * some major DSSSL advanced features are not implemented, the > biggest one brought up to this list being the > complex-page-sequence stuff... There's not much point in implementing full page-sequence flow object in the front-end unless it can be supported in the back-ends. None of the existing backends could support the complex-page-sequence. What's really unique about Jade is the ability to get multiple formats from a single stylesheet. > * TeX backend may need a re-write (see below) Yes. It would be better to do more work in the backend and less work in macros. > * jade is slow (is this even fixable?) I don't think it's slow considering what it's doing. It has a full programming language and there's no way you can stop people writing slow programs in any language. You need to look at the combination of jade plus the stylesheet you're interested in (I guess it's DocBook) and figure out how the combination can be faster. Part of the problem is that there are no profiling tools in Jade: there's no way for a stylesheet author to figure out why their stylesheet was slow. So some way of profiling Jade would be great. It may just be a matter of optimizing the stylesheet, or the solution may be to add some additional primitives to Jade to handle the things that are slow. Another thing that could help is some documentation explaining what things are slow and what things are fast. SGML also brings a lot of overhead to Jade. A DSSSL stylesheet has to be parsed as SGML, the DTD has to be parsed, all the content checking has to be done, then the meta-DTD also has to be read and that has to be parsed, and the architectural document has to be created; only then does the real DSSSL parsing start. A version of Jade that used an XML parser rather than SP and in which DSSSL files didn't have an SGML architectural wrapper would be a lot faster. > * NROFF backend would be nice (note that James himself wrote GROFF -- > I wonder what James thinks about this. Personally, I think its > kinda low priority, since HTML -> ASCII with lynx works decently) I think a decent HTML+CSS backend would be more useful. The current one was really just an attempt at a proof of concept; I did only enough to discover that it wasn't going to be practical with CSS implementations as they then were. But CSS implementations are at last starting to get to the point where they're useable. In particular Mozilla's is excellent. It should be possible to do an HTML+CSS backend which would get you browseable output from the same stylesheet that you use for the RTF or TeX backend. You wouldn't have to write a separate HTML-specific stylesheet. Apart from the things I've mentioned, my priorities would be: - Making the backends useable independently of the rest of Jade. Create a small program (using an XML parser like expat rather than SP) that reads the FOT file and calls an FOT backend. This would require some tweaks to the FOT format and to the FOT interface. The point of this is that it would then be possible to use XSLT with DSSSL flow objects. - Make it easier to use Jade for XML. At the moment the fact that the DSSSL stylesheet is an SGML document makes it awkward to use Jade with XML. There are several possible solutions here and I'm not sure what the best is. - Support XSLT in Jade. XSLT has an xsl:functions element that allows XSLT stylesheets to include extensions written in arbitrary extension languages. With Jade, that extension language would be the DSSSL expression/query language. This is probably too much work. My general feeling is that just as the future is XML not SGML, so the future is also XSL not DSSSL. When XSLT and XSL are done, there will (I hope) be nothing you can do in DSSSL that you can't do with XSL(T). DSSSL has not achieved widespread acceptance, and of course that's disappointing to all of us involved in DSSSL. But I think we have to face up to the fact that the main reason it has failed to achieve acceptance is not because of lack of promotion or explanation or marketing (although that has probably been a factor), but because of real useability problems in the language itself. XSL started off as a collaboration between DSSSLers and Microsoft to create a new syntax for the DSSSL style language that would be easier to use and could achieve wide acceptance. It hasn't quite turned out like that. On the XSL flow objects side, politics and market realities have necessitated building DSSSL functionality on top of CSS formatting objects/properties rather than starting with the DSSSL flow objects. On the XSLT side, as we've continued to work on the language we've found many ways to improve it, and the language has evolved substantially from DSSSL; however it's still very much the same approach to transformation as jade -tsgml. There are big advantages to being in the mainstream, and XSLT looks set to become a mainstream technology. It's attracted the support of some of the biggest players out there -- Microsoft, Oracle, IBM, Sun, Lotus, Adobe (look at the list of WG members in Acknowledgements section of the XSLT WD) -- and they have collectively invested millions of dollars helping to develop it and the language has benefited enormously from this. I have sometimes wondered whether I wasted five years of my life doing DSSSL and Jade, but I comfort myself with the astonishing fact that a little bit of their spirit now seems likely to live on in, of all places, Microsoft Windows. So what does this mean for my involvement in Jade? I think the reality is that although there are improvements that I would dearly like to make, it's unlikely, at least in the near future, that they are going to get to the top of priority list, and I don't want to stand in the way of others who wish to make improvements. I do have one request: if you release something independently of me, you call it something other than simply Jade, but preferably a name that acknowledges Jade; maybe OpenJade or something like that. I also have one strong recommendation on how to proceed with Jade development. Before you start hacking the code, create some documentation on Jade internals. Jade was not designed for a bazaar development model and there is no internals/design documentation. But there is an internal architecture that was designed with a lot of care, and unless those hacking on Jade really understand it, reliability and quality is likely to suffer. For example, when writing extension functions there are rules you have to follow to make the garbage collector work; if you don't follow them everything will compile and appear to work at first but you will get seemingly random, unreproducible crashes on large stylesheets and documents; fixing this kind of bug has taken me literally days in the past. To hack Jade, you really need to know what you're doing. I don't have the time to write the internals docs myself, however I am willing to take the time to explain it to somebody else if there is somebody who (a) has read the code and has a good level of understanding already and (b) promises to write up an internals document. I am also willing to make all my RCS files available, which I would suggest be used to start a public CVS repository. James (speaking for myself alone not the DSSSL WG and not the XSL WG) DSSSList info and archive: http://www.mulberrytech.com/dsssl/dssslist Prepared by Robin Cover for the The SGML/XML Web Page archive.