Basic Tpv Tutorial

[Cache from http://ciips.ee.uwa.edu.au/Research/SCL/Docs/TestSpecGuide.html; please use this canonical URL/source if possible.]

A Guide to Preparing Test Specifications

Main Tags

<Component>
<Implementation>
<Interface>
<TestGroup>
<TestSet>
<Operation>

Left (test selection) panel of the TPV

Credits

The original version of this guide was prepared by Simon Newton.

<Component Name="specification_name">

Top level element of the XML document. Use the Name attribute to name this test specification

<Implementation>

Describes the implementation of the component: at this stage, only Java implementations can be tested.

<Interface TestSetName="test_set_name">

Describes the interface - it assumes that multiple interfaces may be provided by the same component. test_set_name must match the Name attribute of a test set.

<TestSet Name="test_set_name">

A test set should contain a logically related set of tests, eg tests for one component or class. Test sets may have any number of <TestGroup> elements within them.

There should only be only <TestSet> per file. (This is anticipating a planned change to the specification syntax which will see the <TestSet> become the root element for the document.)

<TestGroup>

A group of tests within a test set. Related tests should be organized into a hierarchy of test groups with appropriate names: the TPV will display the value of the Name attribute in its selection panel. The test group hierarchy may be used in any convenient way to group tests: one strategy might place all Operation's testing one method within a test group.

<Operation>

A set of constructors and method calls constituting a single test. An operation would normally represent a test of one equivalence class, ie you could have one operation to test "standard" cases, one to test boundary cases etc.

Method (and Constructor) Calls

Inside an operation you want to call methods and usually at least one Constructor.

Constructor

To call a Constructor, the syntax is:

        <Constructor Name="class_name">
        </Constructor>

This will invoke a constructor for class_name for the class your wanting to test.

Method Calls

A method call is almost identical, except the you need a target object to call it on.

        <MethodCall Name="Method_Name" Target="Target_Object">
        </MethodCall>

Here Method_Name is the name of the method to call and Target_Object is the object to call the method on.
For example box1.getVolume() is written as:

        <MethodCall Name="getVolume" Target="box1">
        </MethodCall>

Passing Arguments

Most methods require arguments. Add <Arg> elements to specify them. Say the method being called is getHeight(int h)

        <MethodCall Name="setHeight" Target="box1">
                <Arg Name="h" DataType="int">2</Arg>
        </MethodCall>

Naturally you can have more than one argument. Just put them after each other.

Using the Result

Calling a method is no use unless you can use the result somehow. This is where the <Result> tag comes in. Add one to specify the type of the expected value and whether it is to be saved or not.

        <MethodCall Name="getVolume" Target="box1">
                <Result Name="volumeOfBox" DataType="int" Save="Y">
                </Result>
        </MethodCall>

Now the volume of the box is stored in the variable volumeOfBox which is an int. You can now use this variable in subsequent method calls. If you want to save the result for later use (in addition to checking its type now) you specify a Name for the result and add Save="Y".

Arguments II

Now that results can be saved, they can be used in other methods. Say the volume of the box is now stored in volumeOfBox and we want to find what it will cost to fill the box with something.

        <MethodCall Name="getCost" Target="Some_Object">
                <Arg Name="volume" Source="volumeOfBox" DataType="int"></Arg>
        </MethodCall>

This is calling the getCost(int volume) method and passing it the volumeOfBox variable.

Expecting

The whole point of the TPV is to test programs. So there needs to be a way to check if a method returns what is expected. Hence the <Exp> tag. Using this tag you can tell the TPV what the method should return. Say we know that the Volume of the box is 10,

        <MethodCall Name="getVolume" Target="box1">
                <Result Name="volumeOfBox" DataType="int">
                        <Exp>10<Exp>
                </Result>
        </MethodCall>

Just a note here: if a String is expected, do not put "'s around it. So for a getName() method which returns a String, the test would be:

        <MethodCall Name="getName" Target="box1">
                <Result Name="box_name" DataType="java.lang.String">
                        <Exp>box one<Exp>
                </Result>
        </MethodCall>

One more thing to notice. java.lang.String The TPV requires that you provide the fully qualified class name: these can be found in the standard API for common classes. This also goes for exceptions which are next....

Exceptions

Instead of returning a result, an exception may be thrown. The TPV needs to be told to expect an exception and what type to expect. Unexpected exceptions are treated as errors and cause a test to be marked "Fail".

        <MethodCall Name="setHeight()" Target="box1">
                <Arg Name="h" DataType="int">2</Arg>
                <Exception Name="e" DataType="java.someException">
                        <Exp>Height must be Positive<Exp>
                </Exception>
        </MethodCall>

The bit between the <Exp> tags is the message String that is set when the exception is thrown.
ie throw new someException("message string")
Notice again that you have to put in the full path (or whatever its called) in the API ( java.lang.someException ). Unless of course you created your own exceptions.

With both results and exceptions,

If a DataType attribute is present, the returned value will be checked to be of the specified type,
If there is no DataType attribute, any object will be accepted: there can be no <Exp> element.
If there is no <Exp> element, the returned object's value will not be checked.
null returns can be checked with:
```
              <Exp>null</Exp>
```

Arrays

Java uses a strange notation for the class of an array:

Array Declaration	Class "Name"
`int[]`	`[I`
`int[][]`	`[[I`
`double[]`	`[D`
`double[][]`	`[[D`
`String[]`	`[Ljava.lang.String;`
`Box[]`	`[LBox;`

All other objects follow the String pattern: observe the ";" at the end, it's needed!

So when testing a method:

            static int[] makeZeroArray( int n );

use:

        <MethodCall Name="makeZeroArray" Target="TestArrays" Static="Y">
           <Result Name="zero" DataType="[I" Save="Y"></Result>
        </MethodCall>

Helper Classes

The tpv can use helper classes to create DataTypes and check complex structures. Say for instance you want to test a method such as

        addToClass(String[] people)

You can use a helper class to create the array and then pass the array over to the method to be tested. This example uses a StringHelper to create an array of Strings an return the array. If you look and the StringHelper class it has a static method which takes a String in which the separate elements of the array are separated by ":" . So the test script is as follows:

        <MethodCall Name="makeStringArray">
                <Arg Name="Strings" DataType="java.lang.String">Adam:Bevan:Gav:Kyle:Simon</Arg>
                <Result Name="people" DataType="[Ljava.lang.String;" Save="Y">
                </Result>
        </MethodCall>

        <MethodCall Name="addToClass" Target="some_target">
                <Arg Name="people" Source="people" DataType="[Ljava.lang.String;"></Arg>
        </MethodCall>

Final Points

If you want to send a null reference then simply type in null. This doesn't work in old versions of the TPV ( < TPV 1.0.10) but works fine in 1.0.9 and above.

        <MethodCall Name="setName" Target="person1">
                <Arg Name="name" DataType="java.lang.String">null</Arg>
        </MethodCall>

What your supposed to do if you want to pass it the String "null" I'm not sure, but why you'd want to do that I'm also not sure! You can always add to a Helper class:

         public static String makeNull() { return "null"; }

Credits

Original version of these notes: Simon Newton.