Thoughts

Some thoughts on how to handle assertions

Most Ruby test frameworks raise a special error when an assertion fails. For instance, that exception class is Test::Unit::AssertionFailedError for Test::Unit, MiniTest::Assertion for MiniTest and AE::Assertion for the Assertive Expressive assertions framework. For these systems to support Ruby Test they only need to add an instance method, #assertion? to their respective classes.

def assertion?
  true
end

By adding this method, Ruby Test will be able to recognize the exception as an assertion error, as opposed to an another type of error.

This approach works well, making it easy for a wide variety of test frameworks to be supported by Ruby Test. However it does have two minor short-comings. Raising an assertion cuts off any following assertions within the same unit test from being executed and it prevents any universal tally of assertions failed or passed.

When Ruby Test invokes a unit test, it rescues all exceptions and adds them to a set of lists. In the case of failed assertions it could be global $TEST_ASSERTIONS variable.

  begin
    unit.call
  rescue Exception => error
    if error.assertion?
      $TEST_ASSERTIONS << error

This indicates the alternative approach a test framework can take to handle assertions. Rather than raise an assertion error, it simply needs to add a compatible assertion object onto the #TEST_ASSERTIONS list. A compatible object is simply an Exception instance that responds to #assertion?, but also #failed?. Ruby Test provides a class for this purpose. Here is a simple example of how a test framework could make use of it.

  def assert(message=nil, &block)
    if block.call
      $TEST_ASSERTIONS << Assertion.new(message, true)
    else
      $TEST_ASSERTIONS << Assertion.new(message)
    end
  end

Notice the second argument to new which marks the assertion as passed, instead of the usual failed.

There is one problem to this approach however. Ruby Test will think the test has passed even if it has not. To get around that would require that it monitor $TEST_ASSERTIONS for changes. Is it worth the overhead?