ansi-test issueshttps://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues2024-03-10T22:33:56Zhttps://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/47No tests for reader errors for very large and very small floating point numbers?2024-03-10T22:33:56ZRaymond ToyNo tests for reader errors for very large and very small floating point numbers?Perhaps, I missed it, but I didn't find any tests for reader errors
for very small or very large floating-point numbers. See
https://www.lispworks.com/documentation/lw60/CLHS/Body/02_caa.htm
which says too small or too large exponents f...Perhaps, I missed it, but I didn't find any tests for reader errors
for very small or very large floating-point numbers. See
https://www.lispworks.com/documentation/lw60/CLHS/Body/02_caa.htm
which says too small or too large exponents for a float should cause a
reader-error to be signaled. But too many digits wont cause a
reader-error even though those extra digits can't be represented.https://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/46No tests for loop variable type declaration?2023-09-15T22:39:58ZRaymond ToyNo tests for loop variable type declaration?From http://www.lispworks.com/documentation/HyperSpec/Body/m_loop.htm,
the for-as-arithmetic clause can be `var [type-spec]
for-as-arithmetic-subclause` where `type-spec` can be a
`simple-type-spec` which is just one of `fixnum`, `float`...From http://www.lispworks.com/documentation/HyperSpec/Body/m_loop.htm,
the for-as-arithmetic clause can be `var [type-spec]
for-as-arithmetic-subclause` where `type-spec` can be a
`simple-type-spec` which is just one of `fixnum`, `float`, `t`, or
`nil`.
AFAICT, there are no tests for this. I did some experiments with
`(for var t from 1 to 2 collect var)`. Clisp produces `(1 2)`, but
ccl, cmucl, ecl, and sbcl all complain that `T` is not a subtype of
`NUMBER`.
Likewise for `(for var nil from 1 to 2 collect var)`, clisp
works, but the others produce an error. I'm not really sure what a
type of `NIL` is supposed to do here since nothing can be of type
`NIL`. I couldn't find anything in the CLHS about this.
I found this when running some tests from [sacla](http://minejima.jp/lisp/sacla/index-en.html). There are other failures that I see, but I have not investigated if they are bugs in the test or in the implementation.https://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/36add new loop tests (based on sicl loop defects)2023-08-27T14:54:10ZDaniel Kochmańskiadd new loop tests (based on sicl loop defects)This is a short session I've run on IRC when poking SICL loop that revealed a few defects not included in tests. The first result is incorrect, the second result is correct:
- [ ] check whether suitable default value of typed variable i...This is a short session I've run on IRC when poking SICL loop that revealed a few defects not included in tests. The first result is incorrect, the second result is correct:
- [ ] check whether suitable default value of typed variable is supplied
```
CL-USER> (sicl-loop:loop with x of-type (unsigned-byte 8) do (print x)
until t
finally (print `(ha ,x)))
NIL
(HA NIL)
NIL
CL-USER> (loop with x of-type (unsigned-byte 8) do (print x)
until t
finally (print `(ha ,x)))
0
(HA 0)
NIL
```
- [ ] check whether the step is executed at the end of each iteration of ON clause
```
CL-USER> (sicl-loop:loop for x on '(1 2 3)
initially (print `(x ,x))
do (print x)
finally (print `(ha ,x)))
(X (1 2 3))
(1 2 3)
(2 3)
(3)
(HA (3))
NIL
CL-USER> (loop for x on '(1 2 3)
initially (print `(x ,x))
do (print x)
finally (print `(ha ,x)))
(X (1 2 3))
(1 2 3)
(2 3)
(3)
(HA NIL)
NIL
```
- [ ] check whether the variable is set at the beginning of each iteration (i.e after the prologue code) [in operator]
```
CL-USER> (sicl-loop:loop for x in '(1 2 3)
initially (print `(x ,x))
do (print x)
finally (print `(ha ,x)))
(X 1)
1
2
3
(HA 3)
NIL
CL-USER> (loop for x in '(1 2 3)
initially (print `(x ,x))
do (print x)
finally (print `(ha ,x)))
(X NIL)
1
2
3
(HA 3)
NIL
```https://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/39mapcan/mapcon (new test)2023-07-31T14:51:34ZAlexander Fedorovmapcan/mapcon (new test)Functions `mapcan` and `mapcon` have two edge cases where most implementations make mistakes.
(Including ccl, abcl, ecl, cmucl and clisp;
sbcl just fixed that on the mainstream;
clasp handles it right.)
Per http://www.lispworks.com/docu...Functions `mapcan` and `mapcon` have two edge cases where most implementations make mistakes.
(Including ccl, abcl, ecl, cmucl and clisp;
sbcl just fixed that on the mainstream;
clasp handles it right.)
Per http://www.lispworks.com/documentation/HyperSpec/Body/f_mapc_.htm
`(mapcon f x1 ... xn)` should be equivalent to `(apply #'nconc (maplist f x1 ... xn))`
and similarly `(mapcan f x1 ... xn)` to `(apply #'nconc (mapcar f x1 ... xn))`.
1) All but last arguments to nconc must be proper lists, so `(nconc 1 2 3)` should error.
`(mapcan #'identity '(1 2 3))` and `(mapcon #'car '(1 2 3))` are equivalent to `(nconc 1 2 3)`,
so they should error too, but the mentioned implementations return `3` instead.
2) nconcing lists with shared structure is valid
(there is an example in specification: http://www.lispworks.com/documentation/HyperSpec/Body/f_nconc.htm),
so it should be valid to do in `mapcan` and `mapcon` as well.
Here is an example where the mentioned implementations go into an infinite loop:
```lisp
(setf *print-circle* t)
;; This returns #1=(1 2 . #1#)
(let ((lst (list 1 2)))
(flet ((f (x)
(declare (ignore x))
lst))
(apply #'nconc (mapcar #'f '(1 2)))))
;; Should be equivalent to the previous example, but hangs instead.
(let ((lst (list 1 2)))
(flet ((f (x)
(declare (ignore x))
lst))
(mapcan #'f '(1 2))))
```https://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/44Add tests for format directivies ~g and ~$2023-07-19T09:47:52ZTarn Burtontwburton@gmail.comAdd tests for format directivies ~g and ~$There doesn't appear to be any tests for [22.3.3.3](https://novaspec.org/cl/22_3_Formatted_Output#sec_22_3_3_3) or [22.3.3.4](https://novaspec.org/cl/22_3_Formatted_Output#sec_22_3_3_4).There doesn't appear to be any tests for [22.3.3.3](https://novaspec.org/cl/22_3_Formatted_Output#sec_22_3_3_3) or [22.3.3.4](https://novaspec.org/cl/22_3_Formatted_Output#sec_22_3_3_4).https://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/31ASDF support?2023-06-22T10:46:02ZRaymond ToyASDF support?Any interest in modifying ansi-test so you can do something like `(asdf:test-system :ansi-test)` instead of running the make file?Any interest in modifying ansi-test so you can do something like `(asdf:test-system :ansi-test)` instead of running the make file?https://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/42stream-external-format only accepts file-streams2023-05-22T14:36:13ZRaymond Toystream-external-format only accepts file-streamsIn [CLHS stream-element-type](http://www.lispworks.com/documentation/HyperSpec/Body/f_stm_ex.htm), the single argument `stream` is specified to be a `file-stream`. I think this means `two-way-stream`, `concatenated-stream`, and `string-s...In [CLHS stream-element-type](http://www.lispworks.com/documentation/HyperSpec/Body/f_stm_ex.htm), the single argument `stream` is specified to be a `file-stream`. I think this means `two-way-stream`, `concatenated-stream`, and `string-stream` are not allowed for `stream-external-format`.
See also #37. This is complicated; the spec is a bit messy on this because [broadcast-stream](http://www.lispworks.com/documentation/HyperSpec/Body/t_broadc.htm) explicitly says `stream-external-format` works, even though it is not a `file-stream`.https://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/41defpackage.28 should record more info on what failed2023-05-20T23:12:32ZRaymond Toydefpackage.28 should record more info on what failedThe test `defpackage.28` just counts how many incorrect values occurred. That's not particularly helpful in figuring out what went wrong. It would be really beneficial if the test recorded more information and returned that as a test r...The test `defpackage.28` just counts how many incorrect values occurred. That's not particularly helpful in figuring out what went wrong. It would be really beneficial if the test recorded more information and returned that as a test result so that it's easier to see what failed.https://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/40Backport changes from the author's repository2023-05-11T12:12:05ZDaniel KochmańskiBackport changes from the author's repositoryPaul worked on ansi-test in his own repository here:
https://github.com/pfdietz/ansi-test
and there is a lot of activity. We should:
- backport changes here
- propose our changes there
so both repositories may benefit from all the good...Paul worked on ansi-test in his own repository here:
https://github.com/pfdietz/ansi-test
and there is a lot of activity. We should:
- backport changes here
- propose our changes there
so both repositories may benefit from all the good stuff.https://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/38pathname-match-p.5 and 6 assume NIL in wild pathname means :wild?2023-03-05T15:41:16ZRaymond Toypathname-match-p.5 and 6 assume NIL in wild pathname means :wild?These two tests are:
```
(deftest pathname-match-p.5
(let ((pn1 (make-pathname :directory '(:relative :wild)))
(pn2 (make-pathname :directory nil)))
(and (wild-pathname-p pn1)
(not (pathname-directory pn2))
...These two tests are:
```
(deftest pathname-match-p.5
(let ((pn1 (make-pathname :directory '(:relative :wild)))
(pn2 (make-pathname :directory nil)))
(and (wild-pathname-p pn1)
(not (pathname-directory pn2))
(not (pathname-match-p pn1 pn2))))
nil)
(deftest pathname-match-p.6
(let ((pn1 (make-pathname :version :wild))
(pn2 (make-pathname)))
(and (wild-pathname-p pn1)
(not (pathname-version pn2))
(not (pathname-match-p pn1 pn2))))
nil)
```
For these tests to return `NIL`, `(pathname-match-p pn1 pn2)` must
return true.
IIUC, that would mean for test 5 that a directory of `'(:relative
:wild)` is expected to match a wild pathname with a directory of
`NIL`. Similarly for test 6, a version of `:WILD` is expected to
match a wild pathname with a version of `NIL`.
I don't see how that can work unless you expected `NIL` in a wild
pathname to be filled with `:WILD`. That's not really clear from the
spec, as mentioned in
https://gitlab.common-lisp.net/ansi-test/ansi-test/-/merge_requests/53#note_10690.
Or is the test incorrect and it's really supposed to say
`(pathname-match-p pn2 pn1)` instead of `(pathname-match-p pn1 pn2)`.
(Reversing the argument order.) The former is kind of suggested because
there's a test that `pn1` is a wild pathname, not for `pn2`, and the
second arg of `pathname-match-p` is the wild pathname.
I noticed this while modifying cmucl's `directory` to use
`pathname-match-p` to make sure the two are consistent. Part of the
change was not causing `NIL` components to be filled by `:WILD`
components.Daniel KochmańskiDaniel Kochmańskihttps://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/25Factor and annotate PRINT-UNREADABLE-OBJECT-{2,3}2020-08-29T19:16:01ZMichał Herdaphoe@disroot.orgFactor and annotate PRINT-UNREADABLE-OBJECT-{2,3}These two tests check the following from the [spec](http://clhs.lisp.se/Body/m_pr_unr.htm):
> If `type` is true, the output from forms is preceded by a brief description of the object's type **and a space character**. If `identity` is t...These two tests check the following from the [spec](http://clhs.lisp.se/Body/m_pr_unr.htm):
> If `type` is true, the output from forms is preceded by a brief description of the object's type **and a space character**. If `identity` is true, the output from forms is followed by **a space character** and a representation of the object's identity, typically a storage address.
Strict reading of the spec implies that that `#<NULL >` and `#< #x41300>` are correct printouts in cases where, respectively, `:TYPE` is true but `:IDENTITY` is false, and where `:IDENTITY` is true but `:TYPE` is false.
I know at least one implementation (CCL) whose maintainers argue that the strictly-standard-compliant variants are ugly, due to the extra space that does not delimit meaningful data inside the angle brackets, and therefore `#<NULL>` and `#<#x41300>` are preferable. A further claim is that the spec does not take into account situations where `:TYPE` and `BODY` are null but `:IDENTITY` is true and where `:IDENTITY` and `BODY` are null but `:TYPE` is true, and therefore this is an omission in the spec. Furthermore, it is very unlikely that any existing code depends on this behaviour, as the output of `PRINT-UNREADABLE-OBJECT` is meant to be eye candy for human readers (which it is somewhat less of, due to the extra space) and to signal errors in the Lisp reader (which behaviour is kept intact, since the `#<` in the beginning is not modified).
I propose to factor out checks for spaces in these two situations into a separate pair of testcases and to annotate these tests with `:PRINT-UNREADABLE-OBJECT-EXTRA-SPACE` (and debatably `:ANSI-SPEC-PROBLEM`), so an implementation may choose to ignore these tests and willingly not conform to that part of the specification.
I can perform these changes and make a MR if you think that this request is agreeable.https://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/17Random tests are random2020-08-29T19:06:46ZRaymond ToyRandom tests are randomMany tests use `RANDOM` in some form to run tests. While this is useful to catch things, it makes it hard to have consistent results since the test may sporadically fail. In addition, it makes debugging the failed test hard since it ra...Many tests use `RANDOM` in some form to run tests. While this is useful to catch things, it makes it hard to have consistent results since the test may sporadically fail. In addition, it makes debugging the failed test hard since it randomly passes and fails depending on the seed used.
It would be useful to use at least a fixed random state at the beginning of the test so that the test run is consistent.https://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/12Enable continuous integration?2019-11-19T13:32:43ZRaymond ToyEnable continuous integration?It might be nice to enable continuous integration so that changes to tests don't cause regressions.
This might be a little tricky because I assume no lisp passes all tests. We'd have to keep track of failed tests so that the list of fa...It might be nice to enable continuous integration so that changes to tests don't cause regressions.
This might be a little tricky because I assume no lisp passes all tests. We'd have to keep track of failed tests so that the list of failed tests don't change. Perhaps #10 would be important for this.
This does add an extra burden on ansi-tests to keep things up-to-date.https://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/22Add COERCE test case with OR types2019-11-17T13:21:05ZMichał Herdaphoe@disroot.orgAdd COERCE test case with OR types```
(coerce '(1 2 3 4 5 6) '(or (vector t 5) (vector t 10)))
```
This should signal an error in safe code.```
(coerce '(1 2 3 4 5 6) '(or (vector t 5) (vector t 10)))
```
This should signal an error in safe code.https://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/5cond accepts invalid clauses2019-10-29T11:47:14ZDaniel Kochmańskicond accepts invalid clauses```common-lisp
(cond (t 'bah) x)
```
This is invalid, since `x` isn't a valid clause.
http://www.lispworks.com/documentation/HyperSpec/Body/m_cond.htm
all implementation checked (sbcl, ccl, ecl, abcl, clisp) accept such code```common-lisp
(cond (t 'bah) x)
```
This is invalid, since `x` isn't a valid clause.
http://www.lispworks.com/documentation/HyperSpec/Body/m_cond.htm
all implementation checked (sbcl, ccl, ecl, abcl, clisp) accept such codehttps://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/13numerical comparison operators: add tests for complex2019-04-09T14:47:52ZDaniel Kochmańskinumerical comparison operators: add tests for complexsome implementations (i.e ecl / clasp) fail to signal a type error when complex is passed as the only argument to them. Add tests:
```common-lisp
(> #c(1 1)) ;-> type error
(>= #c(1 1)) ;-> type error
(<= #c(1 1)) ;-> type error
(< #c(1 ...some implementations (i.e ecl / clasp) fail to signal a type error when complex is passed as the only argument to them. Add tests:
```common-lisp
(> #c(1 1)) ;-> type error
(>= #c(1 1)) ;-> type error
(<= #c(1 1)) ;-> type error
(< #c(1 1)) ;-> type error
(min #c(1 1)) ;-> type error
(max #c(1 1)) ;-> type error
```https://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/1randomness tests not executed2017-08-24T12:26:22ZDaniel Kochmańskirandomness tests not executedduring cleaning up directory structure I've noticed that randomness tests aren't load at all!
during cleaning up directory structure I've noticed that randomness tests aren't load at all!
https://gitlab.common-lisp.net/ansi-test/ansi-test/-/issues/3macro `signals-error` doesn't always catch errors2017-08-24T12:26:22ZDaniel Kochmańskimacro `signals-error` doesn't always catch errors```text
Form: (SIGNALS-ERROR (COMPLEMENT #'NOT T) PROGRAM-ERROR)
Expected value:
T
Actual value:
#<a UNDEFINED-FUNCTION> [The function NIL is undefined.]
``````text
Form: (SIGNALS-ERROR (COMPLEMENT #'NOT T) PROGRAM-ERROR)
Expected value:
T
Actual value:
#<a UNDEFINED-FUNCTION> [The function NIL is undefined.]
```