The R6RS test suite for PLT Scheme is written as a collection of R6RS
libraries, and we hope that it can be useful to other R6RS
implementors. We'd very much like to have R6RS implementors and users
contribute to the test suite.
You can find the test suite in the PLT Scheme SVN repository:
http://svn.plt-scheme.org/plt/trunk/collects/tests/r6rs/
So far, we've tried to get basic coverage of the standard: using each
function and syntactic form at least once, and trying the interesting
input cases for each.
The content of the "README.txt" file follows. As you can see, we
haven't had much success running the test suite on some other
implementations. For Ikarus and Larceny, the problems are not merely
how to load the libraries that contain the tests; the test suite's
organization prevents it from loading when an implementation is
missing small features. I'm not sure how to improve that without
relying on `eval' (and the content below explains why I'd prefer to
avoid `eval') or breaking up the tests into really small sets.
Any suggestions?
======================================================================
Files and libraries
======================================================================
Files that end ".sps" are R6RS programs. The main one is "main.sps",
which runs all the tests.
Files that end ".sls" are R6RS libraries. For example, "base.sls" is a
library that implements `(tests r6rs base)', which is a set of tests
for `(rnrs base)'. Many R6RS implementations will auto-load ".sls"
files if you put the directory of tests in the right place.
In general, for each `(rnrs <id> ... <id>)' in the standard:
* There's a library of tests "<id>/.../<id>.sls". It defines and
ex****ts a function `run-<id>-...<id>-tests'.
* There's a program "run/<id>/.../<id>.sps" that im****ts
"<id>/.../<id>.sls", runs the tests, and re****ts the results.
And then there's "main.sps", which runs all the tests (as noted
above). Also, "test.sls" implements `(tests r6rs test)', which
implements the testing utilities that are used by all the other
libraries.
======================================================================
Limitations and feedback
======================================================================
One goal of this test suite is to avoid using `eval' (except when
specifcally testing `eval'). Avoiding `eval' makes the test suite as
useful as possible to ahead-of-time compilers that implement `eval'
with a separate interpreter. A drawback of the current approach,
however, is that if an R6RS implementation doesn't supply one binding
or does not sup****t a bit of syntax used in a set of tests, then the
whole set of tests fails to load.
A related problem is that each set of tests is placed into one
function that runs all the tests. This format creates a block of code
that is much larger than in a typical program, which might give some
compilers trouble.
In any case, re****ts of bugs (in the tests) and new tests would be
very much appreciated. File either as a PLT Scheme bug re****t at
http://bugs.plt-scheme.org
======================================================================
Hints on running the tests
======================================================================
Ikarus
------
Put this directory at "<somewhere>/tests/r6rs" and run with "run.sps"
cd <somewhere>
ikarus --r6rs-script tests/r6rs/run.sps
or run an individual library's test, such as "run/program.sps" as
cd <somewhere>
ikarus --r6rs-script tests/r6rs/run/program.sps
As of Ikarus 0.3.0+ (revision 1548), many libraries fail to load,
mostly because condition names like &error cannot be used as
expressions.
Larceny
-------
Put this directory at "<somewhere>/tests/r6rs" and run with "run.sps"
larceny -path <somewhere> -r6rs -program run.sps
or run an individual library's test, such as "run/program.sps" as
larceny -path <somewhere> -r6rs -program run/program.sps
As of Larceny 0.962, many test suites (such as "base.sls") take too
long and use too much memory to load on our machine; probably the test
functions are too big.
PLT Scheme
----------
If you get an SVN-based or the "Full" nightly build, then these tests
are
in a `tests/r6rs' collection already. You can run all of the tests
using
mzscheme -l tests/r6rs/run.sps
and so on.
Otherwise, install this directory as a `tests/r6rs' collection,
perhaps in the location re****ted by
(build-path (find-system-path 'addon-dir)
(version) "collects"
"tests" "r6rs")
As of PLT Scheme 4.0.2.5, two tests fail. They correspond to
do***ented non-conformance with R6RS.
Ypsilon
-------
[If there's a library-autoload mechanism, we didn't figure it
out. Better ideas are welcome...]
Load the library declarations that you're interested in. For `(rnrs
<id> ... <id>)':
* Load "test.sls"
* Load "<id>/...<id>.sls"
* Eval `(im****t tests r6rs <id> ... <id>)'
* Eval `(run-<id>-...<id>-tests)'
* Eval `(im****t tests r6rs test)'
* Eval `(show-test-results)'
|
