The generic API to SP

SP provides a generic API in addition to its native API. The generic interface is much simpler than the native interface. It is generic in the sense that it could be easily implemented using parsers other than SP. It provides all ESIS information as well as some other information about the instance that is commonly needed by applications. However, it doesn't provide access to all information available from SP; in particular, it doesn't provide information about the DTD. It is also slightly less efficient than the native interface.

The interface uses two related abstract classes. An SGMLApplication is an object that can handle a number of different kinds of event which correspond to information in an SGML document. An EventGenerator is an object that can generate a sequence of events of the kinds handled by an SGMLApplication. The ParserEventGeneratorKit class makes an EventGenerator that generates events using SP.

Types

SGMLApplication has a number of local types that are used in several contexts:

Char

This typedef is an unsigned integral type that represents a single bit combination (character). It is unsigned short if MULTI_BYTE is defined and unsigned char otherwise.

CharString

This struct represents a string of Char. It has the following members:

const Char *ptr: A pointer to the Chars of the string.
size_t len: The number of Chars in the string.

Location

This struct holds information about a location in the entity structure of a document. It is constucted using an OpenEntityPtr and a Position. The CharStrings in it will remain valid as long as the OpenEntity that is pointed to by the OpenEntityPtr that was used to construct it remains valid.

It has the following members:

unsigned long lineNumber: The line number. (unsigned long)-1 if invalid.
unsigned long columnNumber: The column number. Note that tabs are not treated specially. (unsigned long)-1 if invalid.
unsigned long byteOffset: The number of bytes in the storage object preceding the location. (unsigned long)-1 if invalid.
unsigned long entityOffset: The number of bit combinations in the entity preceding the location. (unsigned long)-1 if invalid.
CharString entityName: The name of the external entity containing the location. An empty string if invalid.
CharString filename: The name of the file containing the location. An empty string if invalid.
const void *other: Other implementation-dependent information about the location. In the SP implementation it will be a pointer to a StorageObjectSpec. 0 if invalid.

When a location is in an internal entity, the location of the reference to the entity will be used instead.

OpenEntity

This class represents a currently open entity. The only use for an OpenEntity is, in conjunction with a Position, to create a Location. An OpenEntity is accessed using an OpenEntityPtr.

OpenEntityPtr

This class is a reference-counted pointer to an OpenEntity.

Position

This is an integral type that represents a position in an open entity. The meaning of a Position is completely determined by the OpenEntity object with which it is associated. The only use for an Position is, in conjunction with an OpenEntity, to create a Location.

ExternalId

This struct represents an external identifier. It has the following members:

bool haveSystemId: True iff the external identifier included an explicit system identifier.
CharString systemId: The system identifier included in the external identifier. Valid only if havePublicId is true.
bool havePublicId: True iff the external identifier included an explicit public identifier.
CharString publicId: The public identifier included in the external identifier. Valid only if havePublicId is true.
bool haveGeneratedSystemId: True iff a system identifier was generated for the external identifier.

CharString generatedSystemId: The system identifier generated for the external identifier. Valid only if haveGeneratedSystemId is true.

Notation

This struct represents a notation. It has the following members:

CharString name: The name of the notation.
ExternalId externalId: The external identifier of the notation.

Entity

This struct represents an entity. It has the following members.

CharString name

The name of the entity.

Entity::DataType dataType

The type of the data of the entity.

Entity::DataType is a local enum with the following possible values:

Entity::sgml
Entity::cdata
Entity::sdata
Entity::ndata
Entity::subdoc
Entity::pi

Entity::DeclType declType

The type of the declaration of the entity.

Entity::DeclType is a local enum with the following possible values:

Entity::general: The entity is a general entity.
Entity::parameter: The entity is a parameter entity.
Entity::doctype: The entity was declared in a doctype declaration.
Entity::linktype: The entity was declared in a linktype declaration.

bool isInternal

True iff the entity is internal rather than external.

CharString text

The replacement text of the entity. Valid only if isInternal is true.

ExternalId externalId

The external identifier of the entity. Valid only if isInternal is false.

const Attribute *attributes

Pointer to the data attributes of the entity. Valid only if isInternal is false.

size_t nAttributes

The number of data attributes of the entity. Valid only if isInternal is false.

Notation notation

The entity's notation. An empty string if the entity has no notation. Valid only if isInternal is false.

Attribute

This struct represents an attribute. More precisely it represents the assignment of an attribute value to an attribute name. It has the following members:

CharString name

The attribute name.

Attribute::Type type

An enumeration describing the type of the attribute.

Attribute::Type is a local type with the following possible values:

Attribute::invalid: The attribute is invalid.
Attribute::implied: The attribute is an impliable attribute for which no value was specified.
Attribute::cdata: The attribute is a CDATA attribute.
Attribute::tokenized: The attribute is a tokenized attribute.

Attribute::Defaulted defaulted

An enumeration specifying whether the entity was defaulted, and, if so, how. This is non-ESIS information.

Attribute::Defaulted is a local enum with the following possible values:

Attribute::specified: The value was explicitly specified.
Attribute::definition: The value was defaulted from the attribute definition.
Attribute::current: The value was defaulted using the CURRENT value of the attribute.

size_t nCdataChunks

The number of Attribute::CdataChunks comprising the value of the attribute. Valid only if type is cdata.

const Attribute::CdataChunk *cdataChunks

The Attribute::CdataChunks comprising the value of this attribute. Valid only if type is cdata.

Attribute::CdataChunk is a local struct with the following members:

bool isSdata: True iff this chunk is the replacement text of an internal SDATA entity.
CharString data: The data of this chunk.
CharString entityName: The name of the internal SDATA entity that this chunk is the replacement text of. Valid only if isSdata is true. This is non-ESIS information.

CharString tokens

Valid only if type is Attribute::tokenized.

bool isId

True iff the declared value is ID. This is non-ESIS information.

size_t nEntities

The number of entities associated with this attribute. This will be zero unless the declared value is ENTITY or ENTITIES.

const Entity *entities

The entities associated with this attribute.

Notation notation

The notation associated with this attribute. If the declared value of the attribute is not NOTATION, then the name member will be an empty string.

Events

For each event xyzEvent handled by SGMLApplication, there is a virtual function of SGMLApplication named xyz to handle the event, and a local struct of SGMLApplication named XyzEvent.

Pointers within an event xyzEvent are valid only during the call to xyz. None of the structs in events have copy constructors or assignment operators defined. It is up to the event handling function to make a copy of any data that it needs to preserve after the function returns.

Except as otherwise stated, the information in events is ESIS information. All position information is non-ESIS information.

There are the following types of event:

AppinfoEvent

Generated for the APPINFO section of the SGML declaration. The event has the following members:

Position pos: The position of APPINFO parameter of the SGML declaration.
bool none: True iff APPINFO NONE was specified.
CharString string: The interpreted value of the minimum literal specified in the appinfo parameter of the SGML declaration. Valid only if none is false.

PiEvent

Generated for a processing instruction. The event has the following members:

Position pos: The position of the processing instruction.
CharString data: The system data of the processing instuction.
CharString entityName: If the processing instruction was the result of the reference to a PI entity, the name of the entity. If not, an empty string. This is non-ESIS information.

StartElementEvent

Generated for the start of an element. The event has the following members:

Position pos

The position of the start of the element.

CharString gi

The generic identifier of the element.

Element::ContentType contentType

The type of the element's content. This is non-ESIS information.

Element::ContentType is an enum with the following possible values:

Element::empty: The element has empty content, either because it was declared as EMPTY or because there was a #CONREF attribute.
Element::cdata: The element has CDATA content.
Element::rcdata: The element has RCDATA content.
Element::mixed: The element has mixed content.
Element::element: The element has element content.

bool included

True iff the element was an included subelement (rather than a proper subelement). This is non-ESIS information.

size_t nAttributes

The number of attributes of this element.

const Attribute *attributes

A pointer to the attributes for this element.

EndElementEvent

Generated for the end of an elemenet. The event has the following members:

Position pos: The position of the end of the element.
CharString gi: The generic identifier of the element.

DataEvent

Generated for character data. The event has the following members:

Position pos: The position of the first character of the data.
CharString data: The data.

SdataEvent

Generated for a reference to an internal sdata entity in content. The event has the following members:

Position pos: The position of the entity reference.
CharString text: The replacement text of the entity.
CharString entityName: The entity name. This is non-ESIS information.

ExternalDataEntityRefEvent

Generated for a reference to an external data entity. The event has the following members:

Position pos: The position of the entity reference.
Entity entity: The referenced entity.

SubdocEntityRefEvent

Generated for a reference to a subdoc entity. The event has the following members:

Position pos: The position of the entity reference.
Entity entity: The referenced entity.

StartDtdEvent

Generated at the start of a document type declaration. This is non-ESIS information. The event has the following members:

Position pos: The position of the start of the document type declaration.
CharString name: The document type name.
bool haveExternalId: The external identifier for the entity declared in the document type declaration.
ExternalId externalId: Valid iff haveExternalId is true.

EndDtdEvent

Generated at the end of a document type declaration. This is non-ESIS information. The event has the following members:

Position pos: The position of the end of the DTD.
CharString name

EndPrologEvent

Generated at the end of the prolog. The event has the following members:

Position pos: The position of the start of the instance.

GeneralEntityEvent

Generated for each general entity in the name space of the governing doctype, but only if the ParserEventGeneratorKit::outputGeneralEntities option is enabled. This is non-ESIS information. The event has the following members:

Entity entity: The entity.

No event will be generated for the declaration of the #default entity; instead an event will be generated when an entity reference uses the #default entity if that is the first time on which an entity with that name is used. This means that GeneralEntityEvent can occur after the end of the prolog.

CommentDeclEvent

Generated for each comment declaration in the instance, but only if ParserEventGeneratorKit::outputCommentDecls option is enabled. This is non-ESIS information. The event has the following members:

Position pos: The position of the start of the comment declaration.
size_t nComments: The number of comments in the comment declaration.
const CharString *comments: The content of each comment in the declaration. This excludes the com delimiters.
const CharString *seps: The separator following each comment in the declaration.

MarkedSectionStartEvent

Generated for the start of a marked section in the instance, but only if the ParserEventGeneratorKit::outputMarkedSections option is enabled. This is non-ESIS information. The event has the following members:

Position pos

The position of the start of the marked section declaration.

MarkedSectionStartEvent::Status status

The effective status of the marked section.

MarkedSectionStartEvent::Status is a local enum with the following possible values:

MarkedSectionStartEvent::include
MarkedSectionStartEvent::rcdata
MarkedSectionStartEvent::cdata
MarkedSectionStartEvent::ignore

size_t nParams

The number of top-level parameters in the status keyword specification.

const MarkedSectionStartEvent::Param *params

The top-level parameters in the status keyword specification.

Param is a local struct with the following members:

MarkedSectionStartEvent::Param::Type type

The type of top-level parameter:

MarkedSectionStartEvent::Param::Type is a local enum with the following possible values:

MarkedSectionStartEvent::Param::temp
MarkedSectionStartEvent::Param::include
MarkedSectionStartEvent::Param::rcdata
MarkedSectionStartEvent::Param::cdata
MarkedSectionStartEvent::Param::ignore: The parameter is the corresponding keyword.
MarkedSectionStartEvent::Param::entityRef: The parameter is an entity reference.

CharString entityName

Valid when type is MarkedSectionStartEvent::Param::entityRef.

MarkedSectionEndEvent

Generated for the end of a marked section in the instance, but only if the ParserEventGeneratorKit::outputMarkedSections option is enabled. This is non-ESIS information. The event has the following members:

Position pos

The position of the end of the marked section declaration.

MarkedSectionEndEvent::Status status

The effective status of the marked section.

MarkedSectionEndEvent::Status is a local enum with the following possible values:

MarkedSectionEndEvent::include
MarkedSectionEndEvent::rcdata
MarkedSectionEndEvent::cdata
MarkedSectionEndEvent::ignore

IgnoredCharsEvent

Generated for a sequence of characters in an ignored marked section in the instance, but only if the ParserEventGeneratorKit::outputMarkedSections option is enabled. This is non-ESIS information. The event has the following members:

Position pos: The position of the first of the ignored characters.
CharString data: The ignored characters.

ErrorEvent

Generated for each error detected by the parser, and also for any other cases where the parser produces a message. This is non-ESIS information. It has the following members:

Position pos

The position at which the error was detected.

ErrorEvent::Type type

The type of error.

ErrorEvent::Type is a local enum with the following possible values:

ErrorEvent::quantity: Exceeding a quantity limit.
ErrorEvent::idref: An IDREF to a non-existent ID.
ErrorEvent::capacity: Exceeding a capacity limit.
ErrorEvent::otherError: Any other kind of error.
ErrorEvent::warning: A warning. Not actually an error.
ErrorEvent::info: An informational message. Not actually an error.

CharString message

The message produced by the parser. If messages are not disabled, this will be the same as the message printed to standard error.

SGMLApplication also has a virtual function

void openEntityChange(const OpenEntityPtr &);

which is similar to an event. An application that wishes to makes use of position information must maintain a variable of type OpenEntityPtr representing the current open entity, and must provide an implementation of the openEntityChange function that updates this variable. It can then use the value of this variable in conjunction with a Position to obtain a Location; this can be relatively slow. Unlike events, an OpenEntityPtr has copy constructors and assignment operators defined.

EventGenerator

The EventGenerator interface provides the following functions:

unsigned run(SGMLApplication &app): Generate the sequence of events, calling the corresponding member of app for each event. Returns the number of errors. This must not be called more than once for any EventGeneratorobject.
EventGenerator *makeSubdocEventGenerator(const SGMLApplication::Char *s, size_t n): Makes a new EventGenerator for a subdocument of the current document. s and n together specify the system identifier of the subdocument entity. These should usually be obtained from the generatedSystemId member of the externalId member of the Entity object for the subdocument entity. This function can only be called after run has been called; the call to run need not have returned, but the SGMLApplication must have been passed events from the prolog or instance (ie the SGML declaration must have been parsed).
void inhibitMessages(bool b): If b is true, disables error and warning messages, otherwise enables them. Initially errors and warnings are enabled. This function may be called at any time, including while run() is executing.
void halt(): Halt the generation of events by run(). This can be at any point during the execution of run(). It is safe to call this function from a different thread from that which called run().

ParserEventGeneratorKit

The ParserEventGeneratorKit class is used to create an EventGenerator that generate events using SP. It provides the following members:

EventGenerator *makeEventGenerator(int nFiles, char *const *files)

This returns a new EventGenerator that will generate events for the SGML document whose document entity is contained in the files. The returned EventGenerator should be deleted when it is no longer needed. makeEventGenerator may be called more than once.

void setOption(ParserEventGeneratorKit::Option opt)

This can be called any time before makeEventGenerator() is called.

ParserEventGeneratorKit::Option is a local enum with the following possible values:

ParserEventGeneratorKit::showOpenEntities: This corresponds to the -e option of nsgmls.
ParserEventGeneratorKit::showOpenElements: This corresponds to the -g option of nsgmls.
ParserEventGeneratorKit::outputCommentDecls: This will cause CommentDeclEvents to be generated.
ParserEventGeneratorKit::outputMarkedSections: This will cause MarkedSectionStartEvents, MarkedSectionStartEvents and IgnoredCharsEvents to be generated.
ParserEventGeneratorKit::outputGeneralEntities: This will cause GeneralEntityEvents to be generated.

void setOption(ParserEventGeneratorKit::OptionWithArg opt, const char *arg)

This can be called any time before makeEventGenerator() is called.

ParserEventGeneratorKit::OptionWithArg is a local enum with the following possible values:

ParserEventGeneratorKit::addCatalog: This corresponds to the -m option of nsgmls.
ParserEventGeneratorKit::includeParam: This corresponds to the -i option of nsgmls.
ParserEventGeneratorKit::enableWarning: This corresponds to the -w option of nsgmls.
ParserEventGeneratorKit::addSearchDir: This corresponds to the -D option of nsgmls.

Using the interface

Creating an application with this interface involves the following steps:

Derive a class from SGMLApplication, called, say, MyApplication.
For each kind of event FooEvent that the application needs information from, define a member of MyApplication void MyApplication::foo(const FooEvent &).
Declare an object of type ParserEventGeneratorKit.
Optionally set options for the ParserEventGeneratorKit using ParserEventGeneratorKit::setOption.
Create an EventGenerator using ParserEventGeneratorKit::makeEventGenerator.
Create an instance of MyApplication (usually on the stack).
Call EventGenerator::run passing it a reference to the instance of MyApplication.
Delete the EventGenerator.

The application must include the ParserEventGeneratorKit.h file (which in turn includes EventGenerator.h and SGMLApplication.h), which is in the generic directory. If your compiler does not support the standard C++ bool type, you must ensure that bool is defined as a typedef for int, before including this. One way to do this is to include config.h and then Boolean.h from the lib subdirectory of the SP distribution.

On Unix, the application must be linked with the lib/libsp.a library.

Example

Here's a simple example of an application that uses this interface. The application prints an outline of the element structure of a document, using indentation to represent nesting.

// The next two lines are only to ensure bool gets defined appropriately.
#include "config.h"
#include "Boolean.h"

#include "ParserEventGeneratorKit.h"
#include <iostream.h>

ostream &operator<<(ostream &os, SGMLApplication::CharString s)
{
  for (size_t i = 0; i < s.len; i++)
    os << char(s.ptr[i]);
  return os;
}

class OutlineApplication : public SGMLApplication {
public:
  OutlineApplication() : depth_(0) { }
  void startElement(const StartElementEvent &event) {
    for (unsigned i = 0; i < depth_; i++)
      cout << "    ";
    cout << event.gi << '\n';
    depth_++;
  }
  void endElement(const EndElementEvent &) { depth_--; }
private:
  unsigned depth_;
};

int main(int argc, char **argv)
{
  ParserEventGeneratorKit parserKit;
  // Use all the arguments after argv[0] as filenames.
  EventGenerator *egp = parserKit.makeEventGenerator(argc - 1, argv + 1);
  OutlineApplication app;
  unsigned nErrors = egp->run(app);
  delete egp;
  return nErrors > 0;
}

There's a bigger example in the sgmlnorm directory in the SP distribution. This uses the SGMLApplication interface, but it doesn't use the ParserEventGeneratorKit interface.

James Clark
jjc@jclark.com