Skip to content

Add ability to specify expected failures

Tarn Burton requested to merge yitzchak-expected-failures into master

Changes

  • Add EXIT keyword argument to do-tests. This will exit with a non-zero status code on test failures and a zero status code on success.
  • Add EXPECTED-FAILURES keyword argument to DO-TESTS. If this argument is provided then it will override the contents of *EXPECTED-FAILURES*. If it is a pathname then the test names will be read from the file with the CL-TEST package as *PACKAGE*.
  • Add support for :EXPECTED-FAILURES option to DEFTEST. This option should be a list of keywords. If any of these keywords are in *FEATURES* then the test will be added to *EXPECTED-FAILURES*. This will only be done if the keyword argument EXPECTED-FAILURES is not provided to DO-TESTS.
  • Add init.lisp which loads, but does not run the tests. An implementation that wants to specify its own expected failures could then do
    clasp --norc --load init.lsp --eval "(rt:do-tests :exit t :expected-failures \"clasp.sexp\")"

I'm not sure about changes to gcload.lisp. It would be nice to have single init file and the ability to specify some of the disable-note stuff, but I'll leave that for now.

Updates from initial comments

  • Add EXIT keyword argument to do-tests. This will exit with a non-zero status code on test failures and a zero status code on success.
  • Add EXPECTED-FAILURES keyword argument to DO-TESTS. If this argument is provided then it will override the contents of *EXPECTED-FAILURES*. If it is a pathname then the test names will be read from the file with the CL-TEST package as *PACKAGE*. Keywords in this list will be used to call DISABLE-NOTE.
  • Add init.lisp which loads, but does not run the tests. An implementation that wants to specify its own expected failures could then do
    clasp --norc --load init.lsp --eval "(rt:do-tests :exit t :expected-failures \"clasp.sexp\")"
Edited by Tarn Burton

Merge request reports