You can implement many tests using the test classes supplied with QMTest. However, expert test implementors may wish to create new test classes to customize existing tests or implement new testing behavior. In addition, implementors will wish to create resource classes to add customized setup and cleanup behavior to tests and groups of tests.
In QMTest, a test class is represented by a Python class. The class must inherit from qm.test.test.Test. The class must include two things: an arguments attribute, whose value is a sequence of field objects, and a Run function.
The test class must include an arguments attribute, indicating the types of the test class's parameters. The arguments attributes value should be a sequence consisting of field objects. A field object is an instance of a subclass of qm.fields.Field. The names of the arguments (specified by the "name" attribute of the field object) are the names of the parameters of the test class.
For instance, this definition of the arguments attribute declares two parameters for the test class. One parameter, called "input_text", takes a text value. The other parameter, called "value_list", takes a set of integers.
arguments = [ qm.fields.IssueFieldText( name="input_text"), qm.fields.IssueFieldSet(qm.fields.IssueFieldInteger( name="value_list")), ]
Test classes should also initialize the title and description attributes of each field. The values of these attributes help users of the test class identify the purpose and semantics of each of the test class's parameters.
The heart of the test class is the Run function. This function runs the test and produces a test result.
The Run function takes two arguments: the context and the result. The context object satisfies the interface of the qm.test.context.Context Python class (though it may in actuality be an instance of a different Python class).
The result object is an instance of qm.test.result.Result. By default, the result will indicate that the test passes. If the test fails, the test class should call the Fail method on the result to indicate failure.
If the Run raises an unhandled exception, QMTest creates a result for the test with the outcome ERROR. Test classes should be designed so that they do not raise unhandled exceptions in the course of normal use (including test failures). An unhandled exception should reflect an internal error in the implementation of the test class.
A context object is simply a dictionary of properties. Use Python's map syntax to access a context's properties. A property key is always a string composed of letters, digits, hyphens, underscores, and periods. Property values are strings.
A test's Run function only sees context properties added by QMTest itself and properties added by the setup functions of required resources.
In case of a FAIL result, it is conventional to assign the Result.CAUSE property a string value providing a description of why the test failed. These two equivalent examples demonstrate how to indicates test failure (both assume that the qm.test.result.Result class has been imported into the module's global namespace).
result.Fail() result[Result.CAUSE] = "Unexpected end of input."
or
result.Fail("Unexpected end of input.")
Sometimes, a Run function implementation detects a failure by catching an exception. The method Result.NoteException provides a convenient mechanism for creating a result that includes information about the exception. For example:
try: # ... run test code here ... except EndOfFileError: result.NoteException()The NoteException method will automatically add annotations describing the cause of the exception.
Writing resource classes is similar to writing test classes. The requirements are the same except that, instead of a Run function, you must provide two functions named SetUp and CleanUp to perform resource setup and cleanup. The SetUp function must have the same signature as a test classs Run. The CleanUp function is similar, but does not take a context parameter.
The setup function may add additional properties to the context. These properties will be visible only to tests that require this resource. To insert a context property, use Python's map assignment syntax.
Below is an example of setup and cleanup functions for a resource which calls create_my_resource and destroy_my_resource to do the work of creating and destroying the resource. The resource is identified by a string handle, which is inserted into the context under the name Resource.handle, where it may be accessed by tests. Context property names should always have the form Class.name so that there is no risk of collision between properties created by different resource classes.
def SetUp(self, context, result): try: handle = create_my_resource() self.__handle = handle except: result.NoteException() else: context["resource_handle"] = str(handle) def CleanUp(self, result): try: destroy_my_resource(self.__handle) except: result.NoteException()