All Packages  Class Hierarchy  This Package  Previous  Next  Index

Class xe.xe_Tokenizer

java.lang.Object
   |
   +----xe.xe_Tokenizer

public class xe_Tokenizer
extends Object
Perform low-level parsing of an XML source and keep the state of the current parse.

The source is supplied via a Reader, which must be set by calling SetSource() prior to calling the range of methods which perform detailed parsing of various sorts. Most of the parsing methods pass back their results in the form of xe_Tokens.


Constructor Index

 o xe_Tokenizer()
Default constructor.

Method Index

 o atEnd()
Determine if we have reached the end of the input source.
 o getLineCount()
Return the number of lines parsed so far.
 o getStopIfVerifyErrorFlag()
Find out whether we check the semantics of the source.
 o getStrictXmlSyntaxFlag()
Find out whether strict XML syntax is to be rigidly applied.
 o getTotalCharCount()
Return the number of characters parsed so far.
 o getValidateFlag()
Find out whether we validate the source.
 o parseContentToken()

Parse the next piece of the content of an element, and return it as a token.

 o parseLiteralToken(boolean)
Parse the next token as a literal.
 o parseLiteralToken(char)
Parse the next token as a literal, assuming that its opening quote (either single or double) has already been parsed.
 o parseName()
Parse a name.
 o parseName(StringBuffer)
Parse a name, which must start with a character acceptable at the start of a name and whose remaining characters must be acceptable within a name.
 o parseNameToken()
Parse the next token as a name.
 o parseNameToken(boolean)
Parse the next token as a name.
 o parseNameTokenExpected(String, boolean)
Parse the next token as a name, and check it has the expected value.
 o parseNextToken(boolean)
Parse the next token.
 o parseNextTokenExpected(int, boolean)
Parse the next token, and make sure it is of the expected type; an exception is thrown if it is not.
 o parseNumber()
Parse a decimal integer number, which can only include 0-9.
 o parseUntil(char)
Parse until we encounter the InputEndChar (or the end of the source).
 o parseUntil(String)
Parse until we encounter the InputEndString (or the end of the source).
 o parseWhitespace(boolean)
Parse all whitespace characters, until we find a non-whitespace character.
 o readNextChar()
Read the next character from the source.
 o reportWarning(String)
Report a warning message.
 o reportWarning(String, int)
Report a warning message.
 o resetStatistics()
Reset the counts of lines, etc., ready for another parse.
 o setSource(Reader)
Set the reader from which the XML source is to be read.
 o setStopIfVerifyErrorFlag(boolean)
Specify whether we check the semantics of the source.
 o setStrictXmlSyntaxFlag(boolean)
Specify whether strict XML syntax is to be rigidly applied.
 o setValidateFlag(boolean)
Specify whether we validate the source.
 o throwParseException(String)
Throw a parse exception, using the InputErrorMessage.
 o throwParseException(String, int, int)
Throw a parse exception, using the InputErrorMessage.

Constructors

 o xe_Tokenizer
 public xe_Tokenizer()
Default constructor.

Methods

 o parseContentToken
 public xe_Token parseContentToken() throws xm_ParseException, IOException

Parse the next piece of the content of an element, and return it as a token. Content can comprise any combination of the following types of entity:

Reference: &Name; or &#nn; or &#xnn;
CDATA section:
Comment:
Processing instruction:
Element: ...
PCData: Text

Returns:
Token representing the piece of content parsed
Throws: xm_ParseException
XML wellformedness error
Throws: IOException
Error reading from source reader
 o parseNextTokenExpected
 public xe_Token parseNextTokenExpected(int InputExpectedTokenType,
                                        boolean InputConsumeWhitespaceFlag) throws xm_ParseException, IOException
Parse the next token, and make sure it is of the expected type; an exception is thrown if it is not.

Parameters:
InputExpectedTokenType - The type of token we expect
InputConsumeWhitespaceFlag - If true, all preceding whitespace is separated; otherwise, it is treated as part of the token itself
Returns:
Token representing the value parsed
Throws: xm_ParseException
XML wellformedness error
Throws: IOException
Error reading from source reader
 o parseNextToken
 public xe_Token parseNextToken(boolean InputConsumeWhitespaceFlag) throws xm_ParseException, IOException
Parse the next token.

Parameters:
InputConsumeWhitespaceFlag - If true, all preceding whitespace is separated; otherwise, it is treated as part of the token itself
Returns:
Token representing the value parsed
Throws: xm_ParseException
XML wellformedness error
Throws: IOException
Error reading from source reader
 o parseNameTokenExpected
 public xe_Token parseNameTokenExpected(String InputExpectedValue,
                                        boolean InputConsumeWhitespaceFlag) throws xm_ParseException, IOException
Parse the next token as a name, and check it has the expected value. It should only be called if the next token must be a name - and throws an exception if it is not. It also throws an exception if the name does not have the expected value.

Parameters:
InputExpectedValue - The value the token is expected to have
InputConsumeWhitespaceFlag - If true, all preceding whitespace is separated; otherwise, it is not allowed (ie. is an error)
Returns:
NAME token
Throws: xm_ParseException
XML wellformedness error
Throws: IOException
Error reading from source reader
 o parseNameToken
 protected xe_Token parseNameToken(boolean InputConsumeWhitespaceFlag) throws xm_ParseException, IOException
Parse the next token as a name. It should only be called if the next token must be a name - and throws an exception if it is not.

Parameters:
InputConsumeWhitespaceFlag - If true, all preceding whitespace is separated; otherwise, it is not allowed (ie. is an error)
Returns:
NAME token
Throws: xm_ParseException
XML wellformedness error
Throws: IOException
Error reading from source reader
 o parseNameToken
 protected xe_Token parseNameToken() throws xm_ParseException, IOException
Parse the next token as a name. It should only be called if a valid name start character has already been found, or the next token must be a name.

Returns:
NAME token
Throws: xm_ParseException
XML wellformedness error
Throws: IOException
Error reading from source reader
 o parseName
 public String parseName() throws xm_ParseException, IOException
Parse a name.

Returns:
A name as a string
Throws: xm_ParseException
XML wellformedness error
Throws: IOException
Error reading from source reader
 o parseName
 protected void parseName(StringBuffer InputNameStringBuffer) throws xm_ParseException, IOException
Parse a name, which must start with a character acceptable at the start of a name and whose remaining characters must be acceptable within a name. (It's rather obvious, really!) The name which is parsed will be added to the InputNameStringBuffer (which doesn't have to be empty beforehand). Currently no namespace support is provided.

Parameters:
InputNameStringBuffer - Buffer to which the name parsed is added
Returns:
A name as a string
Throws: xm_ParseException
XML wellformedness error
Throws: IOException
Error reading from source reader
 o parseNumber
 protected String parseNumber() throws xm_ParseException, IOException
Parse a decimal integer number, which can only include 0-9.

Returns:
The number as a string
Throws: xm_ParseException
XML wellformedness error
Throws: IOException
Error reading from source reader
 o parseWhitespace
 public String parseWhitespace(boolean InputWantWhitespaceFlag) throws xm_ParseException, IOException
Parse all whitespace characters, until we find a non-whitespace character.

Parameters:
InputWantWhitespaceFlag - If true, we parse whitespace; if false we do nothing. While the latter seems pointless, it obviates checks elsewhere.
Returns:
String of the whitespace which was read, or a null String if there was no whitespace
Throws: xm_ParseException
XML wellformedness error
Throws: IOException
Error reading from source reader
 o parseLiteralToken
 public xe_Token parseLiteralToken(boolean InputConsumeWhitespaceFlag) throws xm_ParseException, IOException
Parse the next token as a literal. We expect an opening quote (either single or double), some text and then a closing quote (the same as the opening one). The quotes themselves are not stored as part of the literal itself.

Parameters:
InputConsumeWhitespaceFlag - If true, all preceding whitespace is separated; otherwise, whitespace is treated as part of the main string
Returns:
Token representing the value parsed
Throws: xm_ParseException
XML wellformedness error
Throws: IOException
Error reading from source reader
 o parseLiteralToken
 public xe_Token parseLiteralToken(char InputStartQuoteChar) throws xm_ParseException, IOException
Parse the next token as a literal, assuming that its opening quote (either single or double) has already been parsed. We expect some text and then a closing quote (the same as the opening one). The quotes themselves are not stored as part of the literal itself.

Parameters:
InputStartQuoteChar - The opening quote character
Returns:
Token representing the literal value parsed
Throws: xm_ParseException
XML wellformedness error
Throws: IOException
Error reading from source reader
 o parseUntil
 public String parseUntil(String InputEndString) throws xm_ParseException, IOException
Parse until we encounter the InputEndString (or the end of the source). The InputEndString is omitted from the end of the resulting value.

Parameters:
InputEndString - The string to look for
Returns:
String representing the value parsed
Throws: xm_ParseException
XML wellformedness error
Throws: IOException
Error reading from source reader
 o parseUntil
 public String parseUntil(char InputEndChar) throws xm_ParseException, IOException
Parse until we encounter the InputEndChar (or the end of the source). The InputEndChar is omitted from the end of the resulting value.

Parameters:
InputEndChar - The character to look for
Returns:
String representing the value parsed
Throws: xm_ParseException
XML wellformedness error
Throws: IOException
Error reading from source reader
 o throwParseException
 public void throwParseException(String InputErrorMessage,
                                 int InputStartPosition,
                                 int InputEndPosition) throws xm_ParseException
Throw a parse exception, using the InputErrorMessage.

Parameters:
InputErrorMessage - Message text
InputStartPosition - Start of area in source to highlight
InputEndPosition - End of area in source to highlight
Throws: xm_ParseException
An exception whose throwing has been explicitly requested
 o throwParseException
 public void throwParseException(String InputErrorMessage) throws xm_ParseException
Throw a parse exception, using the InputErrorMessage.

Throws: xm_ParseException
An exception whose throwing has been explicitly requested
 o reportWarning
 public void reportWarning(String InputMessage)
Report a warning message.

Parameters:
InputMessage - The text of the main message to report
 o reportWarning
 public void reportWarning(String InputMessage,
                           int InputColumnOffset)
Report a warning message.

Parameters:
InputMessage - The text of the main message to report
InputColumnOffset - The number of characters back from where we are now at which the event occurred
 o setSource
 public void setSource(Reader InputSourceReader) throws IOException
Set the reader from which the XML source is to be read.

Parameters:
InputSourceStream - The reader from which to read source to parse
Throws: IOException
Error reading first character from source reader
 o setValidateFlag
 public void setValidateFlag(boolean InputValidateFlag)
Specify whether we validate the source.

Parameters:
InputValidateFlag - True means we validate
 o setStopIfVerifyErrorFlag
 public void setStopIfVerifyErrorFlag(boolean InputStopIfVerifyErrorFlag)
Specify whether we check the semantics of the source.

Parameters:
InputStopIfVerifyErrorFlag - True means we check semantics
 o setStrictXmlSyntaxFlag
 public void setStrictXmlSyntaxFlag(boolean InputStrictXmlSyntaxFlag)
Specify whether strict XML syntax is to be rigidly applied.

Parameters:
InputStrictXmlSyntaxFlag - True means apply strict XML syntax
 o resetStatistics
 public void resetStatistics()
Reset the counts of lines, etc., ready for another parse.

 o getValidateFlag
 public boolean getValidateFlag()
Find out whether we validate the source.

Returns:
s True means we validate
 o getStopIfVerifyErrorFlag
 public boolean getStopIfVerifyErrorFlag()
Find out whether we check the semantics of the source.

Returns:
s True means we check semantics
 o getStrictXmlSyntaxFlag
 public boolean getStrictXmlSyntaxFlag()
Find out whether strict XML syntax is to be rigidly applied.

Returns:
s True means we apply strict XML syntax
 o getLineCount
 public int getLineCount()
Return the number of lines parsed so far.

Returns:
The number of lines parsed up to this point
 o getTotalCharCount
 public int getTotalCharCount()
Return the number of characters parsed so far.

Returns:
The number of characters parsed up to this point
 o atEnd
 protected boolean atEnd()
Determine if we have reached the end of the input source.

Returns:
True if at end of source
 o readNextChar
 protected void readNextChar() throws IOException
Read the next character from the source. Input is not buffered.

Throws: IOException
Error reading from source reader

All Packages  Class Hierarchy  This Package  Previous  Next  Index