The ABLUnit testing framework uses annotations to specify that a method in a test class or an internal procedure in a test procedure is a test case.

In ABLUnit, every test should be annotated with @Test annotation.

A test case procedure or test case class can have one BeforeAll and AfterAll method or procedure which are annotated with @BeforeAll and @AfterAll methods. These are run either before all tests or after all tests, respectively.

Note: These annotations were previously referred to as @Before and @After which were ambiguous. The new annotation names are more explicit in their function, though the older annotation names are still supported.

A test case procedure or a test case class can have any number of tests and multiple BeforeEach and AfterEach methods or procedures which are annotated with @BeforeEach and @AfterEach. These are run before each, or after each, test, respectively.

Note: These annotations were previously referred to as @Setup and @Teardown which were ambiguous. The new annotation names are more explicit in their function, though the older annotation names are still supported.

The ABLUnit testing framework can be invoked, for example, by providing a test class or a test procedure. The framework identifies the tests by looking for test annotations associated with methods or procedures. The execution of a test case proceeds in the following sequence:

  1. If a method or procedure is marked with the @BeforeAll annotation, it is executed before any tests.
  2. For each test method or procedure:
    • @BeforeEach annotated method or procedure is executed, if there is any.
    • The test method or procedure is executed.
    • @AfterEach annotated method or procedure is executed, if there is any.
  3. If a method or procedure marked with the @AfterAll annotation is present, it is executed after executing all the tests.
Figure 1. ABLUnit Execution Cycle

Updates to ABLUnit

Application code that extends or calls any classes in the OpenEdge.ABLUnit namespaces may need to be recompiled or updated to use the new public API (classes and methods). The following changes require the modification and removal of a number of public members:
  • Methods with lifecycle annotations
  • Lifecycle method failure and error reporting
  • errNum key for expected error numbers
  • Failure and errors written to results.xml
  • <skipped> element for ignored tests
  • Stop conditions caught
  • <properties> element for test suites
  • @Testsuite or @Test annotations required
Note: All output produced in the results.xml file with the JUnit results standard.

Methods with lifecycle annotations

The ABLUnit test framework has the concept of lifecycle methods that add behavior to test cases and individual tests, for instance, code that must run before each test method. Previous releases allowed multiple methods with the same annotation to be run, but at run time, only a single lifecycle method was run.

Now, all of the methods that have lifecycle annotations are run in the order in which they are defined. Methods must all have the same signature (return void, no parameters), but the OOABL class requires the methods to have different names.

In the following example, BeforeEach_A and BeforeEach_B both run before Test1 runs.
class Tests.MyTest:
    @Test.
    method public void Test1():
    // test method 
    end method.

    @BeforeEach.
    method public void BeforeEach_A():
        // BeforeEach logic
    end method.

    @BeforeEach.
    method public void BeforeEach_B():
        // BeforeEach logic
	end method. 
end class.

In order for multiple methods with lifecycle annotations to be run, additional behavior changes were made.

Lifecycle method failure and error reporting

A test that returns OpenEdge.Core.AssertionFailedError is a failure because a particular assertion was tested. All other conditions raised are considered errors.

If a test method has one or more lifecycle methods, and one of those methods (either the test method itself or one or more of the lifecycle methods) raises any error, then the most severe condition raised is reported. For example, if the test method raises a failure due to an assertion failing, and the lifecycle method raises an error, then the error is reported because an error is more severe than a failure.

errNum key for expected error numbers

The @Test annotation enables developers to specify the type of exception raised using an expected key with an OOABL type name value. To specify an error number, use the errNum key. The value for errNum must be an integer. The test passes if the following are true:
  • The test raises an exception, and the exception matches the expected type.
  • An errNum value is provided.
  • The exception's first error number equals the errNum value.

If no errNum key is provided, then only the expected type is used to determine a test's success.

For example, the following test passes because it does not expect the class to exist.

@Test(expected="Progress.Lang.SysError", errNum=15287).
method public void Test_ClassNotFound():
    define variable obj as Progress.Lang.Object no-undo.
    obj = dynamic-new 'This.Class.Does.Not.Exist' ().
end method.

Failures and errors written to result.xml

A single test may have zero, one, or more lifecycle methods associated with it. The @BeforeEach and @AfterEach methods run before and after each test method, respectively. Failures or errors resulting from the test method or its lifecycle methods each have a failure or error element created in the results.xml file.

For example, there are two failures in the test named test3:

<testcase classname="TestRunFailure.AfterEachFailure" name="test3"
          status="Error" time="0">
    <failure message="Expected: TRUE but was: no" 
             type="OpenEdge.Core.AssertionFailedError">
        IsTrue OpenEdge.Core.Assert at line 177
    </failure>
    <error message="** Invalid character in numeric input T. (76)"
           type="Progress.Lang.SysError">
        TestRunFailure.AfterEachFailure at line 9
    </error>
</testcase>

<skipped> element for ignored tests

If a test is ignored, then a <skipped> element is added with a message attribute.

<testcase classname="TestRunFailure.BeforeAllFailure" name="test"
          status="Success" time="0">
    <skipped message="Ignored" />
</testcase>

Stop conditions caught

ABLUnit handles stop conditions raised in test methods (explicitly or implicitly). The message written to the results.xml varies depending on the value of the -catchStop parameter.

If the -catchStop parameter has a value of 0, then the Progress.Lang.AppError error is raised and caught, and the message "stop condition raised for <test-name>" is added as an error to the test result.

If the -catchStop parameter has a value of 1, then the Progress.Lang.Stop exception is caught, and an error message is added to the test results. The message is typically "Stop condition raised" except for lock conflicts. If there is a lock conflict, then the message is "Lock conflict raised".

<properties> element for test suites

A <properties> element is added to <testcase> (test classes and procedures) <testsuite> elements. This element contains a number of properties that describe the session.

The following is a list of properties and the ABL functions or attributes used to determine their values:

<properties>
    <property name="proversion"           value=proversion(1) />
    <property name="process-architecture" value=process-architecture />
    <property name="opsys"                value=opsys />
    <property name="window-system"        value=session:window-system />
    <property name="cpinternal"           value=session:cpinternal />
    <property name="startup-parameters"   value=session:startup-parameters />
</properties>

@TestSuite or @Test annotation required

Only programs (classes or procedures) that have an @TestSuite annotation or at least one @Test annotation are considered tests, and are added to the list of tests to run. All other programs are ignored by the ABLUnit test runner.