1 <?xml version="1.0" encoding="iso-8859-1"?>
2 <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1//EN"
3 "http://www.oasis-open.org/docbook/xml/4.1/docbookx.dtd" [
4 <!ENTITY % myents SYSTEM "entities.inc">
8 <chapter id="beyond-ansi"><title>Beyond The &ANSI; Standard</title>
10 <para>&SBCL; is mostly an implementation of the &ANSI; standard for
11 Common Lisp. However, there's some important behavior which extends
12 or clarifies the standard, and various behavior which outright
13 violates the standard.
16 <sect1 id="non-conformance"><title>Non-Conformance With The &ANSI; Standard</title>
19 Essentially every type of non-conformance is considered a bug.
20 (The exceptions involve internal inconsistencies in the standard.)
21 In &SBCL; 0.7.6, the master record of known bugs is in
22 the <filename>BUGS</filename> file in the distribution.
23 Some highlight information about bugs may also be found in the
24 manual page. The recommended way to report bugs is through the sbcl-help or
25 sbcl-devel mailings lists.
30 <sect1 id="idiosyncrasies"><title>Idiosyncrasies</title>
32 <para>The information in this section describes some of the ways
33 that &SBCL; deals with choices that the &ANSI; standard
34 leaves to the implementation.</para>
36 <para>Declarations are generally treated as assertions. This general
37 principle, and its implications, and the bugs which still keep the
38 compiler from quite satisfying this principle, are discussed in the
39 <link linkend="compiler">chapter on the compiler</link>.</para>
41 <para>&SBCL; is essentially a compiler-only implementation of
42 &CommonLisp;. That is, for all but a few special cases,
43 <function>eval</function> creates a
44 lambda expression, calls <function>compile</function> on the lambda
45 expression to create a compiled function, and then calls
46 <function>funcall</function> on the resulting function object. This
47 is explicitly allowed by the &ANSI; standard, but leads to some
48 oddities, e.g. collapsing <function>functionp</function> and
49 <function>compiled-function-p</function> into the same predicate.</para>
51 <para>&SBCL; is quite strict about ANSI's definition of
52 <function>defconstant</function>. ANSI says that doing <function>defconstant</function>
53 of the same symbol more than once is undefined unless the new value
54 is <function>eql</function> to the old value. Conforming to this specification
55 is a nuisance when the "constant" value is only constant under some
56 weaker test like <function>string=</function> or <function>equal</function>. It's
57 especially annoying because, in &SBCL;, <function>defconstant</function> takes effect
58 not only at load time but also at compile time, so that just
59 compiling and loading reasonable code like
60 <programlisting>(defconstant +foobyte+ '(1 4))</programlisting>
61 runs into this undefined behavior. Many
62 implementations of Common Lisp try to help the programmer around
63 this annoyance by silently accepting the undefined code and
64 trying to do what the programmer probably meant. &SBCL; instead
65 treats the undefined behavior as an error. Often
66 such code can be rewritten
67 in portable &ANSI; Common Lisp which has the desired behavior.
68 E.g., the code above can be given an exactly defined meaning by replacing
69 <function>defconstant</function> either with <function>defparameter</function> or
70 with a customized macro which does the right thing, possibly along the
71 lines of the <function>defconstant-eqx</function> macro used internally in the
72 implementation of &SBCL; itself. In circumstances where this is not
73 appropriate, the programmer can handle the condition type
74 <errortype>sb-ext:defconstant-uneql</errortype>, and choose either the
75 <action>continue</action> or <action>abort</action> restart as
78 <para>&SBCL; gives style warnings about various kinds of perfectly
81 <listitem><para><function>defmethod</function> without
82 <function>defgeneric</function></para></listitem>
83 <listitem><para>multiple <function>defun</function>s of the same
84 symbol</para></listitem>
85 <listitem><para>special variables not named in the conventional
86 <varname>*foo*</varname> style, and lexical variables unconventionally named
87 in the <varname>*foo*</varname> style</para></listitem>
89 This causes friction with people
90 who point out that other ways of organizing code (especially
91 avoiding the use of <function>defgeneric</function>)
92 are just as aesthetically stylish.
93 However, these warnings should be read not
94 as "warning, bad aesthetics detected, you have no style" but
95 "warning, this style keeps the compiler from understanding
96 the code as well as you might like." That is,
97 unless the compiler warns about such conditions, there's no
98 way for the compiler to warn
99 about some programming errors which would otherwise be
100 easy to overlook. (related bug: The warning about
101 multiple <function>defun</function>s is pointlessly annoying when you compile
102 and then load a function containing <function>defun</function> wrapped
103 in <function>eval-when</function>, and ideally should be suppressed in
104 that case, but still isn't as of &SBCL; 0.7.6.)</para>
108 <sect1 id="extensions"><title>Extensions</title>
110 <para>&SBCL; is derived from &CMUCL;, which implements many extensions
111 to the &ANSI; standard. &SBCL; doesn't support as many extensions as
112 &CMUCL;, but it still has quite a few.</para>
114 <sect2><title>Things Which Might Be In The Next &ANSI; Standard</title>
116 <para>&SBCL; provides extensive support for
117 calling external C code, described
118 <link linkend="ffi">in its own chapter</link>.</para>
120 <para>&SBCL; provides additional garbage collection functionality not
121 specified by &ANSI;. Weak pointers allow references to objects to be
122 maintained without keeping them from being GCed. And "finalization"
123 hooks are available to cause code to be executed when an object has been
124 GCed.</para> <!-- FIXME: Actually documenting these would be good.:-| -->
126 <para>&SBCL; supports Gray streams, user-overloadable CLOS classes
127 whose instances can be used as Lisp streams (e.g. passed as the
128 first argument to <function>format</function>). Additionally, the
129 bundled contrib module <interface>sb-simple-streams</interface>
130 implements a subset of the Franz Allegro simple-streams proposal.</para>
132 <para>&SBCL; supports a MetaObject Protocol which is intended to be
133 compatible with &AMOP;; present exceptions to this (as distinct from
134 current bugs) are:</para>
136 <listitem><para>the abstract <classname>metaobject</classname> class is not
137 present in the class hierarchy;</para></listitem>
138 <listitem><para>the <classname>standard-object</classname> and
139 <classname>funcallable-standard-object</classname> classes are
140 disjoint;</para></listitem>
141 <listitem><para><function>compute-effective-method</function> only returns
142 one value, not two;</para></listitem>
143 <listitem><para>the system-supplied <property>:around</property> method for
144 <function>compute-slots</function> specialized on
145 <classname>funcallable-standard-class</classname> does not respect the
146 requested order from a user-supplied primary method.</para>
152 <sect2><title>Threading (a.k.a Multiprocessing)</title>
154 <para>&SBCL; (as of version 0.8.3, on Linux x86 only) supports a
155 fairly low-level threading interface that maps onto the host operating
156 system's concept of threads or lightweight processes. </para>
158 <sect3><title>Lisp-level view</title>
160 <para>A rudimentary interface to creating and managing multiple threads
161 can be found in the <literal>sb-thread</literal> package. This is
162 intended for public consumption, so look at the exported symbols and
163 their documentation strings.
166 <para>Dynamic bindings to symbols are per-thread. Signal handlers
170 <para>Mutexes and condition variables are available for
171 managing access to shared data: see
174 <programlisting>(apropos "mutex" :sb-thread)</programlisting>
177 <programlisting>(apropos "condition" :sb-thread)</programlisting>
179 <listitem> <para>and the <structname>waitqueue</structname> structure
183 and poke around in their documentation strings.</para>
185 <sect4><title>Sessions</title>
187 <para>If the user has multiple views onto the same Lisp image (for
188 example, using multiple terminals, or a windowing system, or network
189 access) they are typically set up as multiple `sessions' such that
190 each view has its own collection of foreground/background/stopped
191 threads. A thread which wishes to create a new session can use the
192 <function>sb-thread:with-new-session</function> to remove itself from
193 the current session (which it shares with its parent and siblings) and
194 create a fresh one. See also
195 <function>sb-thread:make-listener-thread</function>.</para>
197 <para>Within a single session, threads arbitrate between themselves
198 for the user's attention. A thread may be in one of three notional
199 states: foreground, background, or stopped. When a background process
200 attempts to print a repl prompt or to enter the debugger, it will stop
201 and print a message saying that it has stopped. The user at his
202 leisure may switch to that thread to find out what it needs. If a
203 background thread enters the debugger, selecting any restart will put
204 it back into the background before it resumes. Arbitration for the
205 input stream is managed by calls to
206 <function>sb-thread:get-foreground</function> (which may block) and
207 <function>sb-thread:release-foreground</function>.</para>
209 <para><function>sb-ext:quit</function> terminates all threads in the
210 current session, but leaves other sessions running</para>
215 <sect3><title>Implementation (Linux x86)</title>
217 <para>On Linux x86, this is implemented using
218 <function>clone()</function> and does not involve pthreads. This is
219 not because there is anything wrong with pthreads <emphasis>per
220 se</emphasis>, but there is plenty wrong (from our perspective) with
221 LinuxThreads. &SBCL; threads are mapped 1:1 onto Linux tasks which
222 share a VM but nothing else - each has its own process id and can be
223 seen in e.g. <command>ps</command> output.
226 <para>Per-thread local bindings for special variables is achieved
227 using the %fs segment register to point to a per-thread storage area.
228 This may cause interesting results if you link to foreign code that
229 expects threading or creates new threads, and the thread library in
230 question uses %fs in an incompatible way.
233 <para>There are two implementation mechanisms for queueing. If SBCL
234 was built on an NPTL-capable Linux system (2.6 or some vendor 2.4
235 ports) with the :SB-FUTEX feature, queuing will be doneusing the
236 <function>sys_futex()</function> call. Otherwise it will fall back to
237 using <function>sigtimedwait()</function> to sleep and a signal
238 (SIG_DEQUEUE, one of the POSIX RT signals) to wake.</para>
240 <para>&SBCL; at present will alway have at least two tasks running as
241 seen from Linux: when the first process has done startup
242 initialization (mapping files in place, installing signal handlers
243 etc) it creates a new thread to run the Lisp startup and initial
244 listener. The original thread stays around to reap it when it's dead
245 and deallocate its resources (e.g. stacks) when it exits.
248 <para>It should be noted that the initial thread does less and less in
249 each new release of SBCL, and one day soon will probably go away
252 <para>Garbage collection is done with the existing Conservative
253 Generational GC. Allocation is done in small (typically 8k) regions :
254 each thread has its own region so this involves no stopping. However,
255 when a region fills, a lock must be obtained while another is
256 allocated, and when a collection is required, all processes are
257 stopped. This is achieved by sending them signals, which may make for
258 interesting behaviour if they are interrupted in system calls. The
259 streams interface is believed to handle the required system call
260 restarting correctly, but this may be a consideration when making
261 other blocking calls e.g. from foreign library code.
264 <para>Large amounts of the &SBCL; library have not been inspected for
265 thread-safety. Some of the obviously unsafe areas have large locks
266 around them, so compilation and fasl loading, for example, cannot be
267 parallelized. Work is ongoing in this area.
270 <para>A new thread by default is created in the same POSIX process
271 group and session as the thread it was created by. This has an impact
272 on keyboard interrupt handling: pressing your terminal's intr key
273 (typically Control-C) will interrupt all processes in the foreground
274 process group, including Lisp threads that &SBCL; considers to be
275 notionally `background'. This is undesirable, so background threads
276 are set to ignore the SIGINT signal.</para>
278 <para><function>sb-thread:make-listener-thread</function> in addition
279 to creating a new Lisp session makes a new POSIX session, so that
280 pressing Control-C in one window will not interrupt another listener -
281 this has been found to be embarrassing.
288 <sect2><title>Support For Unix</title>
290 <para>The UNIX command line can be read from the variable
291 <varname>sb-ext:*posix-argv*</varname>. The UNIX environment can be queried with the
292 <function>sb-ext:posix-getenv</function> function.</para>
294 <para>The &SBCL; system can be terminated with
295 <function>sb-ext:quit</function>, (but see notes above about threads
296 and sessions) optionally returning a specified numeric value to the
297 calling Unix process. The normal Unix idiom of terminating on end of
298 file on input is also supported.</para>
302 <sect2><title>Customization Hooks for Users</title>
304 <para>The behaviour of <function>require</function> when called with only
305 one argument is implementation-defined. In &SBCL; it calls functions
306 on the user-settable list <varname>sb-ext:*module-provider-functions*</varname>
307 - see the <function>require</function> documentation string for details.
310 <para>The toplevel repl prompt may be customized, and the function
311 that reads user input may be replaced completely. <!-- FIXME but I
312 don't currently remember how -->
317 <sect2><title>Tools To Help Developers</title>
319 <para>&SBCL; provides a profiler and other extensions to the &ANSI;
320 <function>trace</function> facility. See the online function documentation for
321 <function>trace</function> for more information.</para>
323 <para>The debugger supports a number of options. Its documentation is
324 accessed by typing <userinput>help</userinput> at the debugger prompt.</para>
326 A true debugger section in the manual would be good. Start
327 with CMU CL's debugger section, but remember:
328 * no QUIT command (TOPLEVEL restart instead)
329 * no GO command (CONTINUE restart instead)
330 * Limitations of the x86 port of the debugger should be
331 documented or fixed where possible.
332 * Discuss TRACE and its unification with PROFILE. -->
334 <para>Documentation for <function>inspect</function> is accessed by typing
335 <userinput>help</userinput> at the <function>inspect</function> prompt.</para>
339 <sect2><title>Interface To Low-Level &SBCL; Implementation</title>
341 <para>&SBCL; has the ability to save its state as a file for later
342 execution. This functionality is important for its bootstrapping
343 process, and is also provided as an extension to the user See the
344 documentation for <function>sb-ext:save-lisp-and-die</function> for more
347 <note><para>&SBCL; has inherited from &CMUCL; various hooks to allow
348 the user to tweak and monitor the garbage collection process. These
349 are somewhat stale code, and their interface might need to be cleaned
350 up. If you have urgent need of them, look at the code in
351 <filename>src/code/gc.lisp</filename> and bring it up on the
352 developers' mailing list.</para></note>
354 <note><para>&SBCL; has various hooks inherited from &CMUCL;, like
355 <function>sb-ext:float-denormalized-p</function>, to allow a program to take
356 advantage of &IEEE; floating point arithmetic properties which aren't
357 conveniently or efficiently expressible using the &ANSI; standard. These
358 look good, and their interface looks good, but &IEEE; support is
359 slightly broken due to a stupid decision to remove some support for
360 infinities (because it wasn't in the &ANSI; spec and it didn't occur to
361 me that it was in the &IEEE; spec). If you need this stuff, take a look
362 at the code and bring it up on the developers' mailing
367 <sect2><title>Efficiency Hacks</title>
369 <para>The <function>sb-ext:purify</function> function causes &SBCL;
370 first to collect all garbage, then to mark all uncollected objects as
371 permanent, never again attempting to collect them as garbage. This can
372 cause a large increase in efficiency when using a primitive garbage
373 collector, or a more moderate increase in efficiency when using a more
374 sophisticated garbage collector which is well suited to the program's
375 memory usage pattern. It also allows permanent code to be frozen at
376 fixed addresses, a precondition for using copy-on-write to share code
377 between multiple Lisp processes. is less important with modern
378 generational garbage collectors. </para>
380 <para>The <function>sb-ext:truly-the</function> declares the type of the
381 result of the operations, producing its argument; the declaration is
382 not checked. In short: don't use it.</para>
384 <para>The <function>sb-ext:freeze-type</function> declaration declares that a
385 type will never change, which can make type testing
386 (<function>typep</function>, etc.) more efficient for structure types.</para>
388 <para>The <function>sb-ext:constant-function</function> declaration specifies
389 that a function will always return the same value for the same
390 arguments, which may allow the compiler to optimize calls
391 to it. This is appropriate for functions like <function>sqrt</function>, but
392 is <emphasis>not</emphasis> appropriate for functions like <function>aref</function>,
393 which can change their return values when the underlying data are
395 <!-- FIXME: This declaration does not seem to be supported in the
396 current compiler. -->