NewsML Toolkit (1.1beta): Conformance Testing

Document Revision: $Revision: 1.2 $
Date: $Date: 2001/11/26 16:33:37 $

Starting with version 1.1beta, the NewsML Toolkit includes support for testing a NewsML document's conformance to the functional specification. Low-level DTD-based structural validation is available through the underlying XML parser; higher-level, conformance testing is now handled by the org.newsml.toolkit.conformance package. The conformance framework is fully open and configurable, so users can easily add conformance tests for local business logic.

NewsML Conformance for Programmers

Writing Tests

The NewsML conformance suite consists of a collection of tests that extend the abstract org.newsml.toolkit.conformance.TestBase class. This class has one abstract method that all subclasses must implement:

public void run (BaseNode contextNode, boolean useExternal)
  throws NewsMLException;

The test manager will invoke this method whenever it needs to validation a NewsML node: it will pass the node being evaluated as the contextNode argument. The second argument, useExternal, indicates whether the test is allowed to reference resources outside the NewsML document (such as external Web sites) -- the test should ensure that it honours this flag and adjusts its results accordingly.

If the node passes the test, the run method must simply return. If there is an error, the method must throw a NewsMLException. If there is a warning (a non-fatal error condition), the method should throw the special ConformanceWarning exception, which extends NewsMLException. The TestBase class provides convenience methods for reporting warnings and errors:

public static void warning (String message)
  throws ConformanceWarning;

public static void error (String message)
  throws NewsMLException;

public static void internal (String message)
  throws RuntimeException;

The last method throws a runtime exception if something is wrong internally in the test. This should not usually occur in production-grade code, but it's useful for debugging.

Registering Tests

Once the application has created any local tests, it must create a new NewsMLTestManager class and register all tests with it using the addTest method:

public void addTest (String xpath, TestBase test);

The first argument of this method is an XPath (relative to a context node that the application provides later) matching the nodes to which the test will be applied. For example, the following code creates a new test manager and registers MyNewsItemTest to be applied to all NewsItems:

NewsMLTestManager testManager = new NewsMLTestManager();
testManager.addTest("//NewsItem", new MyNewsItemTest());

There are many default tests available, based on the NewsML functional specification. It would be tedious to add each of these individually, so there is a single convenience method to add them in a batch:

testManager.addDefaultTests();

Normally, applications should begin by enabling all of the default tests, and then add additional tests. In special cases, where some of the default tests are not appropriate or desired, applications can add only the desired default tests using addTest.

Running Tests

Once an application has finished registering tests, it can configure the behaviour of the NewsMLTestManager before running the tests on a NewsML document. There are two types of configuration available:

  1. Verbosity: the application can configure the test manager to display progress messages to standard output using the setVerbose method.

  2. Error reporting: the application can register an implementation of org.newsml.toolkit.ErrorVisitor to handle error messages using the setErrorVisitor method. By default, warnings are printed to standard error and errors stop the program; a GUI application (for example) can provide a custom ErrorVisitor to collect all error messages and display them in a scrollable list instead.

Once the application has configured the test manager, it runs the tests by invoking the following method:

public void runTests (BaseNode contextNode, boolean useExternal)
  throws NewsMLException;

This method runs all of the tests on the nodes matching the XPath expressions each was registered with; the useExternal flag controls whether the tests will be allowed to access resources outside the NewsML document.

The test manager traps any errors or warnings, and reports them to the ErrorVisitor (if provided) including the NewsML node that the XPath expression matched, so that an application can easily find where the error occurred. If the method terminates without an exception, then all tests passed (or else the errors were intercepted by a user-supplied ErrorVisitor).

Examples

This simple example creates a test manager, enables verbose progress reporting, adds the default tests, and runs them on a top-level NewsML node newsml using the built-in ErrorVisitor and prohibiting access to external resources:

NewsMLTestManager testManager = new NewsMLTestManager();
testManager.setVerbose(true);
testManager.addDefaultTests();
testManager.runTests(newsml, false);

This example creates a new test to match a local business rule that all headlines must be in French:

import org.newsml.toolkit.BaseNode;
import org.newsml.toolkit.LanguageNode;
import org.newsml.toolkit.NewsMLException;
import org.newsml.toolkit.Text;
import org.newsml.toolkit.conformance.TestBase;
public class FrenchHeadlineTest 
  extends TestBase
{
  public void run (BaseNode node, boolean useExternal)
    throws NewsMLException
  {
    Text lang = ((LanguageNode)node).getLang();
    if (lang == null)
      error("No xml:lang specified");
    else if (!"fr".equals(lang.toString()))
      error("Language " + lang + " not allowed");
  }
}

A simple test like this could also be defined anonymously at the time of registration:

testManager.addTest("//Headline", new TestBase () {
  public void run (BaseNode node, boolean useExternal)
    throws NewsMLException
  {
    Text lang = ((LanguageNode)node).getLang();
    if (lang == null)
      error("No xml:lang specified");
    else if (!"fr".equals(lang.toString()))
      error("Language " + lang + " not allowed");
  }
});

NewsML Conformance for End-Users

The NewsML conformance suite is designed primarily to be incorporated into other applications, including GUI applications, and it is most likely that end-users will encounter it there. However, the distribution does include a simple, standalone Java application for running conformance tests from the command line, with the following usage:

java ValidateNewsML [-v|--verbose] [-h|--help] [-x|--external] url+

The command-line options are as follow:

-v, --verbose
Display verbose progress reports (terse by default).
-h, --help
Display usage help and exit.
-x, --external
Allow the tests to access external resources (forbidden by default).

The special pseudo-option "--" terminates option processing, so that any relative URLs starting with "-" will not be misinterpreted as options. This application will print an error report to standard output.

To run the application, the user must have newsml-toolkit.jar, gnu-regexp.jar, jaxen-full.jar, saxpath.jar, and xerces.jar on the Java CLASSPATH (copies of all of these are bundled in the NewsML Toolkit distribution).

Default Tests

The NewsML Toolkit comes bundled with the following default tests:

  1. CatalogTest
  2. ContentItemTest
  3. DateTimeTest
  4. DefaultVocabularyForTest
  5. EuidTest
  6. FormalNameTest
  7. NewsComponentTest
  8. NewsIdentifierTest
  9. NewsItemTest
  10. NewsLineTest
  11. PatternTest
  12. PropertyTest
  13. RefTest
  14. ResourceTest
  15. RevisionIdTest
  16. RevisionStatusTest
  17. TopicSetTest
  18. TopicTest
  19. TopicUseTest
  20. XMLLangTest

Follow the links to the JavaDoc documentation for a detailed description of each test.