=== The Super Brief Introduction ===
-FiveAM is a testing framework. See the xref:API_REFERENCE[api] for
-details.
+|================================
+| (xref:OP_DEF-TEST[`def-test`] `NAME` () &body `BODY`) | define tests
+| (xref:OP_IS[`is`] (`PREDICATE` `EXPECTED` `ACTUAL`)) | check that, according to `PREDICATE` our `ACTUAL` is the same as our `EXPECTED`
+| (xref:OP_IS[`is-true`] VALUE) | check that a value is non-NIL
+| (xref:OP_RUN![`run!`] TEST-NAME) | run one (or more) tests and print the results
+|================================
+
+See the xref:API_REFERENCE[api] for details.
=== An Ever So Slightly Longer Introduction ===
You use define some xref:TESTS[tests] (using
xref:OP_DEF-TEST[`def-test`]), each of which consists of some
xref:CHECKS[checks] (with xref:OP_IS[`is`] and friends) which can pass
-or fail; you xref:RUNNING_TESTS[run] some tests (using
-xref:OP_RUN-EPOINT-[run!] and friends) and you look at the results
-(probably using xref:OP_RUN-EPOINT-[run!] again). Lather, rinse,
-repeat.
+or fail:
+
+--------------------------------
+(def-test a-test ()
+ (is (= 4 (+ 2 2)))
+ (is-false (= 5 (+ 2 2))))
+--------------------------------
+
+you xref:RUNNING_TESTS[run] some tests (using xref:OP_RUN[run] and
+friends) and you look at the results (using using
+xref:OP_EXPLAIN[explain]); or you do both at once (using
+xref:OP_RUN-EPOINT-[run!]):
+
+--------------------------------
+CL-USER> (run! 'a-test)
+..
+Did 2 checks.
+ Pass: 2 (100%)
+ Skip: 0 ( 0%)
+ Fail: 0 ( 0%)
+--------------------------------
+
+Lather, rinse, repeat:
+
+--------------------------------
+CL-USER> (run!)
+..
+Did 2 checks.
+ Pass: 2 (100%)
+ Skip: 0 ( 0%)
+ Fail: 0 ( 0%)
+--------------------------------
=== The Real Introduction ===
at first, but they'll get applied, and then they'll get used, and then
they'll be an essential part of fiveam itself...)
-=== Words ===
+==== Words ====
Since there are far many more testing frameworks than there are words
for talking about testing frameworks, the same words end up meaning
=== Creating Suites ===
-Suites are created in one of two ways: Either explicitly via the
-xref:OP_DEF-SUITE[`def-suite`] macro, or implicity via the
-xref:OP_DEF-SUITE-STAR-[`def-suite*`] and/or
-xref:OP_IN-SUITE-STAR-[`in-suite*`] macros:
-
Suites, very much like tests, have a name (which is globally unique)
which can be used to retrieve the suite (so that you can run it), and,
most of the time, suites are part of a suite (the exception being the
--------------------------------
(def-suite :my-project)
-(in-suite* :my-db-layer :in :my-project)
+(def-suite :my-db-layer :in :my-project)
+
+(in-suite :my-db-layer)
--------------------------------
[[THE_CURRENT_SUITE]]
FiveAM also has the concept of a current suite and everytime a test is
created it adds itself to the current suite's set of tests. The
-`IN-SUITE` and `IN-SUITE*` macros, in a similar fashion to
-`IN-PACKAGE`, change the current suite.
-
-Unless changed via `IN-SUITE` and `IN-SUITE*` the current suite is the
+`IN-SUITE` macro, in a similar fashion to `IN-PACKAGE`, changes the
+current suite. Unless changed via `IN-SUITE` the current suite is the
`T` suite.
Having a default current suite allows developers to ignore suites
Where `TEST-NAME` is either a test object (as returned by `get-test`)
or a symbol naming a single test or a test suite.
-=== Re-running Tests ===
-
-The `run!` function stores its arguments in a set of variables and,
-via the functions `!`, `!!` and `!!!` will rerun those named
-tests. Note that we're deliberatly talking about names, and not test
-objects, `!` will take the last argument passed to `run!` and call
-`run!` with that again, looking up the test again if the argument was
-a symbol.
-
-This ensures that `!` will always run the current definition of a
-test, even if the test has been redefined since the last time `run!`
-was called.
-
=== Running Tests at Test Definition Time ===
Often enough, especially when fixing regression bugs, we'll always
reasons you have to set this variable manually after having loaded
your test suite.
-[NOTE]
-Setting `*run-test-when-defined*` will cause `run!` to get called far
-more often than normal. `!` and `!!` and `!!!` don't know that they're
-getting called semi-automatically and will therefore tend to all
-reduce to the same test (which still isn't totally useless behaviour).
-
=== Debugging failures and errors ===
`*debug-on-error*`::
== Random Testing (QuickCheck) ==
-TODO.
+Sometimes it's hard to come up with edge cases for tests, or sometimes
+there are so many that it's hard to list them all one by one. Random
+testing is a way to tell the test suite how to generate input and how
+to test that certain conditions always hold. One issue when writing
+random tests is that you can't, usually, test for specific results,
+you have to test that certain relationships hold.
+
+For example, if we had a function which reverses a list, we could
+define a relationship like this:
+
+--------------------------------
+(equalp the-list (reverse (reverse the-list)))
+--------------------------------
+
+or
+
+--------------------------------
+(equalp (length the-list) (length (reverse the-list)))
+--------------------------------
+
+Random tests are defined via `def-test`, but the random part is then
+wrapped in a xref:OP_FOR-ALL[`for-all`] macro which runs its body
+`*num-trials*` times with different inputs:
-Every FiveAM test can be a random test, just use the for-all macro.
+--------------------------------
+(for-all ((the-list (gen-list :length (gen-integer :min 0 :max 37)
+ :elements (gen-integer :min -10 :max 10))))
+ (is (equalp a (reverse (reverse the-list))))
+ (is (= (length the-list) (length (reverse the-list)))))
+--------------------------------
== Fixtures ==
================================
--------------------------------
-(def-test NAME
- (&key DEPENDS-ON SUITE FIXTURE COMPILE-AT PROFILE)
- &body BODY)
+(def-test NAME (&key DEPENDS-ON SUITE FIXTURE COMPILE-AT PROFILE) &body BODY)
--------------------------------
include::docstrings/OP_DEF-TEST.txt[]
[[OP_IN-SUITE]]
[[OP_IN-SUITE-STAR-]]
-=== IN-SUITE / IN-SUITE* ===
+=== IN-SUITE ===
================================
----
include::docstrings/OP_IN-SUITE.txt[]
================================
-================================
-----
-(in-suite* NAME &key IN)
-----
-
-include::docstrings/OP_IN-SUITE-STAR-.txt[]
-================================
-
[[OP_IS]]
=== IS ===
include::docstrings/OP_RUN.txt[]
================================
-=== ! / !! / !!! ===
-
-================================
-----
-(!)
-----
-
-include::docstrings/OP_-EPOINT-.txt[]
-================================
-
-================================
-----
-(!!)
-----
-
-include::docstrings/OP_-EPOINT--EPOINT-.txt[]
-================================
-
-================================
-----
-(!!!)
-----
-
-include::docstrings/OP_-EPOINT--EPOINT--EPOINT-.txt[]
-================================
-
[[OP_DEF-FIXTURE]]
=== DEF-FIXTURE ===
projects / fiveam.git / blobdiff