Updates to ABLUnit

Methods with lifecycle annotations

The ABLUnit test framework has the concept of lifecycle methods that add behavior to test cases and individual tests, for instance, code that must run before each test method. Previous releases allowed multiple methods with the same annotation to be run, but at run time, only a single lifecycle method was run.

Now, all of the methods that have lifecycle annotations are run in the order in which they are defined. Methods must all have the same signature (return void, no parameters), but the OOABL class requires the methods to have different names.

class Tests.MyTest:
    @Test.
    method public void Test1():
    // test method 
    end method.

    @Setup.
    method public void Setup_A():
        // setup logic
    end method.

    @Setup.
    method public void Setup_B():
        // setup logic
	end method. 
end class.

In order for multiple methods with lifecycle annotations to be run, additional behavior changes were made.

Lifecycle method failure and error reporting

A test that returns OpenEdge.Core.AssertionFailedError is a failure because a particular assertion was tested. All other conditions raised are considered errors.

If a test method has one or more lifecycle methods, and one of those methods (either the test method itself or one or more of the lifecycle methods) raises any error, then the most severe condition raised is reported. For example, if the test method raises a failure due to an assertion failing, and the lifecycle method raises an error, then the error is reported because an error is more severe than a failure.

errNum key for expected error numbers

If no errNum key is provided, then only the expected type is used to determine a test's success.

For example, the following test passes because it does not expect the class to exist.

@Test(expected="Progress.Lang.SysError", errNum=15287).
method public void Test_ClassNotFound():
    define variable obj as Progress.Lang.Object no-undo.
    obj = dynamic-new 'This.Class.Does.Not.Exist' ().
end method.

Failures and errors written to results.xml

A single test may have zero, one, or more lifecycle methods associated with it. The @Setup and @TearDown methods run before and after each test method, respectively. Failures or errors resulting from the test method or its lifecycle methods each have a failure or error element created in the results.xml.

For example, there are two failures in the test named test3:

<testcase classname="TestRunFailure.AfterEachFailure" name="test3"
          status="Error" time="0">
    <failure message="Expected: TRUE but was: no" 
             type="OpenEdge.Core.AssertionFailedError">
        IsTrue OpenEdge.Core.Assert at line 177
    </failure>
    <error message="** Invalid character in numeric input T. (76)"
           type="Progress.Lang.SysError">
        TestRunFailure.AfterEachFailure at line 9
    </error>
</testcase>

<skipped> element for ignored tests

If a test is ignored, then a <skipped> element is added with a message attribute.

<testcase classname="TestRunFailure.BeforeAllFailure" name="test"
          status="Success" time="0">
    <skipped message="Ignored" />
</testcase>

Stop conditions caught

ABLUnit handles stop conditions raised in test methods (explicitly or implicitly). The message written to the results.xml file varies depending on the value of the -catchStop parameter.

If the -catchStop parameter has a value of 0, then the Progress.Lang.AppError error is raised and caught, and the message "stop condition raised for test-name" is added as an error to the test result.

If the -catchStop parameter has a value of 1, then the Progress.Lang.Stop exception is caught, and an error message is added to the test results. The message is typically "Stop condition raised" except for lock conflicts. If there is a lock conflict, then the message is "Lock conflict raised".

<properties> element for test suites

A <properties> element is added to <testcase> (test classes and procedures) and <testsuite> elements. This element contains a number of properties that describe the session.

The following is a list of properties and the ABL functions or attributes used to determine their values:

<properties>
    <property name="proversion"           value=proversion(1) />
    <property name="process-architecture" value=process-architecture />
    <property name="opsys"                value=opsys />
    <property name="window-system"        value=session:window-system />
    <property name="cpinternal"           value=session:cpinternal />
    <property name="startup-parameters"   value=session:startup-parameters />
</properties>

@TestSuite or @Test annotation required

Only programs (classes or procedures) that have an @TestSuite annotation or at least one @Test annotation are considered tests, and are added to the list of tests to run. All other programs are ignored by the ABLUnit test runner.

For more information about the ABLUnit framework, see Lifecycle of ABLUnit framework in Progress Developer Studio for OpenEdge Online Help.