From: Rudi Schlatte Date: Wed, 31 Mar 2004 20:02:12 +0000 (+0000) Subject: 0.8.9.13 X-Git-Url: http://repo.macrolet.net/gitweb/?a=commitdiff_plain;h=78867137cd2b9e689fd07640d60e4cf3942bf719;p=sbcl.git 0.8.9.13 * Import initial version of manual in Texinfo markup * Add Debugger chapter of cmucl manual - Comment out list-locations, step, breakpoint commands, since they do not work at the moment - add *debug-print-variable-alist* documentation, adapt debug optimization policy description --- diff --git a/doc/manual/backmatter.texinfo b/doc/manual/backmatter.texinfo new file mode 100644 index 0000000..061b214 --- /dev/null +++ b/doc/manual/backmatter.texinfo @@ -0,0 +1,25 @@ +@node Function Index, Concept Index, The Foreign Function Interface, Top +@comment node-name, next, previous, up +@appendix Function Index + +@printindex fn + +@node Concept Index, Colophon, Function Index, Top +@comment node-name, next, previous, up +@appendix Concept Index + +@printindex cp + +@node Colophon, , Concept Index, Top +@comment node-name, next, previous, up +@unnumbered Colophon + +This manual is maintained in Texinfo, and automatically translated +into other forms (e.g. HTML or pdf). If you're @emph{reading} this +manual in one of these non-Texinfo translated forms, that's fine, but +if you want to @emph{modify} this manual, you are strongly advised to +seek out a Texinfo version and modify that instead of modifying a +translated version. Even better might be to seek out @emph{the} +Texinfo version (maintained at the time of this writing as part of the +SBCL project at @uref{http://sbcl.sourceforge.net/}) and submit a +patch. diff --git a/doc/manual/beyond-ansi.texinfo b/doc/manual/beyond-ansi.texinfo new file mode 100644 index 0000000..084fe32 --- /dev/null +++ b/doc/manual/beyond-ansi.texinfo @@ -0,0 +1,403 @@ +@node Beyond The ANSI Standard, The Foreign Function Interface, Efficiency, Top +@comment node-name, next, previous, up +@chapter Beyond The ANSI Standard + +SBCL is mostly an implementation of the ANSI standard for +Common Lisp. However, there's some important behavior which extends +or clarifies the standard, and various behavior which outright +violates the standard. + + +@menu +* Non-Conformance With The ANSI Standard:: +* Idiosyncrasies:: +* Extensions:: +@end menu + +@node Non-Conformance With The ANSI Standard, Idiosyncrasies, Beyond The ANSI Standard, Beyond The ANSI Standard +@comment node-name, next, previous, up +@section Non-Conformance With The ANSI Standard + +Essentially every type of non-conformance is considered a bug. (The +exceptions involve internal inconsistencies in the standard.) In SBCL +0.7.6, the master record of known bugs is in the @file{BUGS} file in +the distribution. Some highlight information about bugs may also be +found in the manual page. The recommended way to report bugs is +through the sbcl-help or sbcl-devel mailings lists. For mailing list +addresses, @ref{More SBCL Information}. + + +@node Idiosyncrasies, Extensions, Non-Conformance With The ANSI Standard, Beyond The ANSI Standard +@comment node-name, next, previous, up +@section Idiosyncrasies + +The information in this section describes some of the ways that SBCL +deals with choices that the ANSI standard leaves to the +implementation. + +Declarations are generally treated as assertions. This general +principle, and its implications, and the bugs which still keep the +compiler from quite satisfying this principle, are discussed in the +chapter on the compiler, @ref{The Compiler}. + +SBCL is essentially a compiler-only implementation of Common +Lisp. That is, for all but a few special cases, @code{eval} creates a +lambda expression, calls @code{compile} on the lambda expression to +create a compiled function, and then calls @code{funcall} on the +resulting function object. This is explicitly allowed by the ANSI +standard, but leads to some oddities, e.g. collapsing @code{functionp} +and @code{compiled-function-p} into the same predicate. + +SBCL is quite strict about ANSI's definition of +@code{defconstant}. ANSI says that doing @code{defconstant} of the +same symbol more than once is undefined unless the new value is +@code{eql} to the old value. Conforming to this specification is a +nuisance when the ``constant'' value is only constant under some +weaker test like @code{string=} or @code{equal}. It's especially +annoying because, in SBCL, @code{defconstant} takes effect not only at +load time but also at compile time, so that just compiling and loading +reasonable code like + +@lisp +(defconstant +foobyte+ '(1 4)) +@end lisp + +runs into this undefined behavior. Many implementations of Common Lisp +try to help the programmer around this annoyance by silently accepting +the undefined code and trying to do what the programmer probably +meant. SBCL instead treats the undefined behavior as an error. Often +such code can be rewritten in portable ANSI Common Lisp which has the +desired behavior. E.g., the code above can be given an exactly +defined meaning by replacing @code{defconstant} either with +@code{defparameter} or with a customized macro which does the right +thing, possibly along the lines of the @code{defconstant-eqx} macro +used internally in the implementation of SBCL itself. In +circumstances where this is not appropriate, the programmer can handle +the condition type @code{sb-ext:defconstant-uneql}, and choose either +the @command{continue} or @command{abort} restart as appropriate. + +SBCL gives style warnings about various kinds of perfectly legal code, +e.g. + +@itemize + +@item +@code{defmethod} without @code{defgeneric} + +@item +multiple @code{defun}s of the same symbol + +@item +special variables not named in the conventional @code{*foo*} style, +and lexical variables unconventionally named in the @code{*foo*} style + +@end itemize + +This causes friction with people who point out that other ways of +organizing code (especially avoiding the use of @code{defgeneric}) are +just as aesthetically stylish. However, these warnings should be read +not as ``warning, bad aesthetics detected, you have no style'' but +``warning, this style keeps the compiler from understanding the code +as well as you might like.'' That is, unless the compiler warns about +such conditions, there's no way for the compiler to warn about some +programming errors which would otherwise be easy to overlook. (Related +bug: The warning about multiple @code{defun}s is pointlessly annoying +when you compile and then load a function containing @code{defun} +wrapped in @code{eval-when}, and ideally should be suppressed in that +case, but still isn't as of SBCL 0.7.6.) + + +@node Extensions, , Idiosyncrasies, Beyond The ANSI Standard +@comment node-name, next, previous, up +@section Extensions + +SBCL is derived from CMUCL, which implements many extensions to the +ANSI standard. SBCL doesn't support as many extensions as CMUCL, but +it still has quite a few. + + +@menu +* Things Which Might Be In The Next ANSI Standard:: +* Threading:: +* Support For Unix:: +* Customization Hooks for Users:: +* Tools To Help Developers:: +* Interface To Low-Level SBCL Implementation:: +* Efficiency Hacks:: +@end menu + +@node Things Which Might Be In The Next ANSI Standard, Threading, Extensions, Extensions +@comment node-name, next, previous, up +@subsection Things Which Might Be In The Next ANSI Standard + +SBCL provides extensive support for calling external C code, @ref{The +Foreign Function Interface}. + +SBCL provides additional garbage collection functionality not +specified by ANSI. Weak pointers allow references to objects to be +maintained without keeping them from being GCed (garbage +collected). And ``finalization'' hooks are available to cause code to +be executed when an object has been GCed. +@c + +SBCL supports @dfn{Gray streams}, user-overloadable CLOS classes whose +instances can be used as Lisp streams (e.g. passed as the first +argument to @code{format}). Additionally, the bundled contrib module +@dfn{sb-simple-streams} implements a subset of the Franz Allegro +simple-streams proposal. + +SBCL supports a MetaObject Protocol which is intended to be compatible +with AMOP; present exceptions to this (as distinct from current bugs) +are: + +@itemize + +@item +the abstract @code{metaobject} class is not present in the class +hierarchy; + +@item +the @code{standard-object} and @code{funcallable-standard-object} +classes are disjoint; + +@item +@code{compute-effective-method} only returns one value, not two; + +@item +the system-supplied @code{:around} method for @code{compute-slots} +specialized on @code{funcallable-standard-class} does not respect the +requested order from a user-supplied primary method. + +@end itemize + + +@node Threading, Support For Unix, Things Which Might Be In The Next ANSI Standard, Extensions +@comment node-name, next, previous, up +@subsection Threading (a.k.a Multiprocessing) + +SBCL (as of version 0.8.3, on Linux x86 only) supports a fairly +low-level threading interface that maps onto the host operating +system's concept of threads or lightweight processes. + +@subsubsection Lisp-level view + +A rudimentary interface to creating and managing multiple threads can +be found in the @dfn{sb-thread} package. This is intended for public +consumption, so look at the exported symbols and their documentation +strings. + +Dynamic bindings to symbols are per-thread. Signal handlers are +per-thread. + +Mutexes and condition variables are available for managing access to +shared data: see + +@itemize + +@item +@code{(apropos "mutex" :sb-thread)} + +@item +@code{(apropos "condition" :sb-thread)} + +@item +and the @code{waitqueue} structure + +@end itemize + +and poke around in their documentation strings. + +@subsubsection Sessions + +If the user has multiple views onto the same Lisp image (for example, +using multiple terminals, or a windowing system, or network access) +they are typically set up as multiple @dfn{sessions} such that each +view has its own collection of foreground/background/stopped threads. +A thread which wishes to create a new session can use +@code{sb-thread:with-new-session} to remove itself from the current +session (which it shares with its parent and siblings) and create a +fresh one. See also @code{sb-thread:make-listener-thread}. + +Within a single session, threads arbitrate between themselves for the +user's attention. A thread may be in one of three notional states: +foreground, background, or stopped. When a background process +attempts to print a repl prompt or to enter the debugger, it will stop +and print a message saying that it has stopped. The user at his +leisure may switch to that thread to find out what it needs. If a +background thread enters the debugger, selecting any restart will put +it back into the background before it resumes. Arbitration for the +input stream is managed by calls to @code{sb-thread:get-foreground} +(which may block) and @code{sb-thread:release-foreground}. + +@code{sb-ext:quit} terminates all threads in the current session, but +leaves other sessions running. + + +@subsubsection Implementation (Linux x86) + +On Linux x86, threading is implemented using @code{clone()} and does +not involve pthreads. This is not because there is anything wrong +with pthreads @emph{per se}, but there is plenty wrong (from our +perspective) with LinuxThreads. SBCL threads are mapped 1:1 onto +Linux tasks which share a VM but nothing else - each has its own +process id and can be seen in e.g. @command{ps} output. + +Per-thread local bindings for special variables is achieved using the +%fs segment register to point to a per-thread storage area. This may +cause interesting results if you link to foreign code that expects +threading or creates new threads, and the thread library in question +uses %fs in an incompatible way. + +There are two implementation mechanisms for queueing. If SBCL was +built on an NPTL-capable Linux system (2.6 or some vendor 2.4 ports) +with the @code{:SB-FUTEX} feature, queuing will be done using the +@code{sys_futex()} system call if it's available at runtime. +Otherwise it will fall back to using @code{sigtimedwait()} to sleep +and a signal (@code{SIG_DEQUEUE}, one of the POSIX RT signals) to +wake. + +Garbage collection is done with the existing Conservative Generational +GC. Allocation is done in small (typically 8k) regions: each thread +has its own region so this involves no stopping. However, when a +region fills, a lock must be obtained while another is allocated, and +when a collection is required, all processes are stopped. This is +achieved by sending them signals, which may make for interesting +behaviour if they are interrupted in system calls. The streams +interface is believed to handle the required system call restarting +correctly, but this may be a consideration when making other blocking +calls e.g. from foreign library code. + +Large amounts of the SBCL library have not been inspected for +thread-safety. Some of the obviously unsafe areas have large locks +around them, so compilation and fasl loading, for example, cannot be +parallelized. Work is ongoing in this area. + +A new thread by default is created in the same POSIX process group and +session as the thread it was created by. This has an impact on +keyboard interrupt handling: pressing your terminal's intr key +(typically @kbd{Control-C}) will interrupt all processes in the +foreground process group, including Lisp threads that SBCL considers +to be notionally `background'. This is undesirable, so background +threads are set to ignore the SIGINT signal. + +@code{sb-thread:make-listener-thread} in addition to creating a new +Lisp session makes a new POSIX session, so that pressing +@kbd{Control-C} in one window will not interrupt another listener - +this has been found to be embarrassing. + + +@node Support For Unix, Customization Hooks for Users, Threading, Extensions +@comment node-name, next, previous, up +@subsection Support For Unix + +The UNIX command line can be read from the variable +@code{sb-ext:*posix-argv*}. The UNIX environment can be queried with +the @code{sb-ext:posix-getenv} function. + +The SBCL system can be terminated with @code{sb-ext:quit}, (but see +notes above about threads and sessions) optionally returning a +specified numeric value to the calling Unix process. The normal Unix +idiom of terminating on end of file on input is also supported. + + +@node Customization Hooks for Users, Tools To Help Developers, Support For Unix, Extensions +@comment node-name, next, previous, up +@subsection Customization Hooks for Users + +The behaviour of @code{require} when called with only one argument is +implementation-defined. In SBCL it calls functions on the +user-settable list @code{sb-ext:*module-provider-functions*} - see the +@code{require} documentation string for details. + +The toplevel repl prompt may be customized, and the function +that reads user input may be replaced completely. +@c + + +@node Tools To Help Developers, Interface To Low-Level SBCL Implementation, Customization Hooks for Users, Extensions +@comment node-name, next, previous, up +@subsection Tools To Help Developers + +SBCL provides a profiler and other extensions to the ANSI +@code{trace} facility. See the online function documentation for +@code{trace} for more information. + +The debugger supports a number of options. Its documentation is +accessed by typing @kbd{help} at the debugger prompt. +@c + +Documentation for @code{inspect} is accessed by typing @kbd{help} at +the @code{inspect} prompt. + + +@node Interface To Low-Level SBCL Implementation, Efficiency Hacks, Tools To Help Developers, Extensions +@comment node-name, next, previous, up +@subsection Interface To Low-Level SBCL Implementation + +SBCL has the ability to save its state as a file for later +execution. This functionality is important for its bootstrapping +process, and is also provided as an extension to the user. See the +documentation string for @code{sb-ext:save-lisp-and-die} for more +information. + +@quotation +Note: SBCL has inherited from CMUCL various hooks to allow the user to +tweak and monitor the garbage collection process. These are somewhat +stale code, and their interface might need to be cleaned up. If you +have urgent need of them, look at the code in @file{src/code/gc.lisp} +and bring it up on the developers' mailing list. +@end quotation + +@quotation +Note: SBCL has various hooks inherited from CMUCL, like +@code{sb-ext:float-denormalized-p}, to allow a program to take +advantage of IEEE floating point arithmetic properties which aren't +conveniently or efficiently expressible using the ANSI standard. These +look good, and their interface looks good, but IEEE support is +slightly broken due to a stupid decision to remove some support for +infinities (because it wasn't in the ANSI spec and it didn't occur to +me that it was in the IEEE spec). If you need this stuff, take a look +at the code and bring it up on the developers' mailing +list. +@end quotation + + +@node Efficiency Hacks, , Interface To Low-Level SBCL Implementation, Extensions +@comment node-name, next, previous, up +@subsection Efficiency Hacks + +The @code{sb-ext:purify} function causes SBCL first to collect all +garbage, then to mark all uncollected objects as permanent, never +again attempting to collect them as garbage. This can cause a large +increase in efficiency when using a primitive garbage collector, or a +more moderate increase in efficiency when using a more sophisticated +garbage collector which is well suited to the program's memory usage +pattern. It also allows permanent code to be frozen at fixed +addresses, a precondition for using copy-on-write to share code +between multiple Lisp processes. it is less important with modern +generational garbage collectors. + +@code{sb-ext:truly-the} declaration declares the type of the result of +the operations, producing its argument; the declaration is not +checked. In short: don't use it. + +The @code{sb-ext:freeze-type} declaration declares that a +type will never change, which can make type testing +(@code{typep}, etc.) more efficient for structure types. + +The @code{sb-ext:constant-function} declaration specifies +that a function will always return the same value for the same +arguments, which may allow the compiler to optimize calls +to it. This is appropriate for functions like @code{sqrt}, but +is @emph{not} appropriate for functions like @code{aref}, +which can change their return values when the underlying data are +changed. +@c diff --git a/doc/manual/compiler.texinfo b/doc/manual/compiler.texinfo new file mode 100644 index 0000000..cd0c1cf --- /dev/null +++ b/doc/manual/compiler.texinfo @@ -0,0 +1,1019 @@ +@node The Compiler, The Debugger, Introduction, Top +@comment node-name, next, previous, up +@chapter The Compiler + +This chapter will discuss most compiler issues other than +efficiency, including compiler error messages, the SBCL compiler's +unusual approach to type safety in the presence of type declarations, +the effects of various compiler optimization policies, and the way +that inlining and open coding may cause optimized code to differ from +a naive translation. Efficiency issues are sufficiently varied and +separate that they have their own chapter, @ref{Efficiency}. + +@menu +* Error Messages:: +* Handling of Types:: +* Compiler Policy:: +* Open Coding and Inline Expansion:: +@end menu + +@node Error Messages, Handling of Types, The Compiler, The Compiler +@comment node-name, next, previous, up +@section Error Messages +@cindex Error messages, Compiler +@cindex Compiler error messages + +The compiler supplies a large amount of source location information in +error messages. The error messages contain a lot of detail in a terse +format, so they may be confusing at first. Error messages will be +illustrated using this example program: + +@lisp +(defmacro zoq (x) + `(roq (ploq (+ ,x 3)))) + +(defun foo (y) + (declare (symbol y)) + (zoq y)) +@end lisp + +The main problem with this program is that it is trying to add +@code{3} to a symbol. Note also that the functions @code{roq} and +@code{ploq} aren't defined anywhere. + + +@menu +* The Parts of the Error Message:: +* The Original and Actual Source:: +* Error Severity:: +* Errors During Macroexpansion:: +* Read Errors:: +@end menu + +@node The Parts of the Error Message, The Original and Actual Source, Error Messages, Error Messages +@comment node-name, next, previous, up +@subsection The Parts of the Error Message + +When processing this program, the compiler will produce this warning: + +@example +file: /tmp/foo.lisp + +in: DEFUN FOO +(ZOQ Y) +--> ROQ PLOQ + +==> +Y +caught WARNING: +Result is a SYMBOL, not a NUMBER. +@end example + +In this example we see each of the six possible parts of a compiler +error message: + +@enumerate + +@item +@samp{File: /tmp/foo.lisp} This is the name of the file that the +compiler read the relevant code from. The file name is displayed +because it may not be immediately obvious when there is an error +during compilation of a large system, especially when +@code{with-compilation-unit} is used to delay undefined warnings. + +@item +@samp{in: DEFUN FOO} This is the definition top level form responsible +for the error. It is obtained by taking the first two elements of the +enclosing form whose first element is a symbol beginning with +``@samp{def}''. If there is no such enclosing ``@samp{def}'' form, +then the outermost form is used. If there are multiple @samp{def} +forms, then they are all printed from the outside in, separated by +@samp{=>}'s. In this example, the problem was in the @code{defun} for +@code{foo}. + +@item +@samp{(ZOQ Y)} This is the @dfn{original source} form responsible for +the error. Original source means that the form directly appeared in +the original input to the compiler, i.e. in the lambda passed to +@code{compile} or in the top level form read from the source file. In +this example, the expansion of the @code{zoq} macro was responsible +for the error. + +@item +@samp{--> ROQ PLOQ +} This is the @dfn{processing path} that the +compiler used to produce the errorful code. The processing path is a +representation of the evaluated forms enclosing the actual source that +the compiler encountered when processing the original source. The +path is the first element of each form, or the form itself if the form +is not a list. These forms result from the expansion of macros or +source-to-source transformation done by the compiler. In this +example, the enclosing evaluated forms are the calls to @code{roq}, +@code{ploq} and @code{+}. These calls resulted from the expansion of +the @code{zoq} macro. + +@item +@samp{==> Y} This is the @dfn{actual source} responsible for the +error. If the actual source appears in the explanation, then we print +the next enclosing evaluated form, instead of printing the actual +source twice. (This is the form that would otherwise have been the +last form of the processing path.) In this example, the problem is +with the evaluation of the reference to the variable @code{y}. + +@item +@samp{caught WARNING: Result is a SYMBOL, not a NUMBER.} This is the +@dfn{explanation} of the problem. In this example, the problem is that +@code{y} evaluates to a symbol, but is in a context where a number is +required (the argument to @code{+}). + +@end enumerate + +Note that each part of the error message is distinctively marked: + +@itemize + +@item + @samp{file:} and @samp{in:} mark the file and definition, +respectively. + +@item +The original source is an indented form with no prefix. + +@item +Each line of the processing path is prefixed with @samp{-->} + +@item +The actual source form is indented like the original source, but is +marked by a preceding @samp{==>} line. + + +@item +The explanation is prefixed with the error severity, which can be +@samp{caught ERROR:}, @samp{caught WARNING:}, @samp{caught +STYLE-WARNING:}, or @samp{note:}. + +@end itemize + +Each part of the error message is more specific than the preceding +one. If consecutive error messages are for nearby locations, then the +front part of the error messages would be the same. In this case, the +compiler omits as much of the second message as in common with the +first. For example: + +@example +file: /tmp/foo.lisp + +in: DEFUN FOO +(ZOQ Y) +--> ROQ +==> +(PLOQ (+ Y 3)) +caught STYLE-WARNING: +undefined function: PLOQ + +==> +(ROQ (PLOQ (+ Y 3))) +caught STYLE-WARNING: +undefined function: ROQ +@end example + +In this example, the file, definition and original source are +identical for the two messages, so the compiler omits them in the +second message. If consecutive messages are entirely identical, then +the compiler prints only the first message, followed by: @samp{[Last +message occurs @var{repeats} times]} where @var{repeats} is the number +of times the message was given. + +If the source was not from a file, then no file line is printed. If +the actual source is the same as the original source, then the +processing path and actual source will be omitted. If no forms +intervene between the original source and the actual source, then the +processing path will also be omitted. + + +@node The Original and Actual Source, Error Severity, The Parts of the Error Message, Error Messages +@comment node-name, next, previous, up +@subsection The Original and Actual Source + +The @emph{original source} displayed will almost always be a list. If +the actual source for an error message is a symbol, the original +source will be the immediately enclosing evaluated list form. So even +if the offending symbol does appear in the original source, the +compiler will print the enclosing list and then print the symbol as +the actual source (as though the symbol were introduced by a macro.) + +When the @emph{actual source} is displayed (and is not a symbol), it +will always be code that resulted from the expansion of a macro or a +source-to-source compiler optimization. This is code that did not +appear in the original source program; it was introduced by the +compiler. + +Keep in mind that when the compiler displays a source form in an error +message, it always displays the most specific (innermost) responsible +form. For example, compiling this function + +@lisp +(defun bar (x) + (let (a) + (declare (fixnum a)) + (setq a (foo x)) + a)) +@end lisp + +gives this error message + +@example +in: DEFUN BAR +(LET (A) (DECLARE (FIXNUM A)) (SETQ A (FOO X)) A) +caught WARNING: The binding of A is not a FIXNUM: +NIL +@end example + +This error message is not saying ``there is a problem somewhere in +this @code{let}'' -- it is saying that there is a problem with the +@code{let} itself. In this example, the problem is that @code{a}'s +@code{nil} initial value is not a @code{fixnum}. + + +@subsection The Processing Path +@cindex Processing path +@cindex Macroexpansion +@cindex Source-to-source transformation + +The processing path is mainly useful for debugging macros, so if you +don't write macros, you can probably ignore it. Consider this example: + +@lisp +(defun foo (n) + (dotimes (i n *undefined*))) +@end lisp + +Compiling results in this error message: + +@example +in: DEFUN FOO +(DOTIMES (I N *UNDEFINED*)) +--> DO BLOCK LET TAGBODY RETURN-FROM +==> +(PROGN *UNDEFINED*) +caught STYLE-WARNING: +undefined variable: *UNDEFINED* +@end example + +Note that @code{do} appears in the processing path. This is because +@code{dotimes} expands into: + +@lisp +(do ((i 0 (1+ i)) (#:g1 n)) + ((>= i #:g1) *undefined*) + (declare (type unsigned-byte i))) +@end lisp + +The rest of the processing path results from the expansion of +@code{do}: + +@lisp +(block nil + (let ((i 0) (#:g1 n)) + (declare (type unsigned-byte i)) + (tagbody (go #:g3) + #:g2 (psetq i (1+ i)) + #:g3 (unless (>= i #:g1) (go #:g2)) + (return-from nil (progn *undefined*))))) +@end lisp + +In this example, the compiler descended into the @code{block}, +@code{let}, @code{tagbody} and @code{return-from} to +reach the @code{progn} printed as the actual source. This is a +place where the ``actual source appears in explanation'' rule +was applied. The innermost actual source form was the symbol +@code{*undefined*} itself, but that also appeared in the +explanation, so the compiler backed out one level. + + +@node Error Severity, Errors During Macroexpansion, The Original and Actual Source, Error Messages +@comment node-name, next, previous, up +@subsection Error Severity +@cindex Severity of compiler errors +@cindex compiler error severity + +There are four levels of compiler error severity: @emph{error}, +@emph{warning}, @emph{style warning}, and @emph{note}. The first three +levels correspond to condition classes which are defined in the ANSI +standard for Common Lisp and which have special significance to the +@code{compile} and @code{compile-file} functions. These levels of +compiler error severity occur when the compiler handles conditions of +these classes. The fourth level of compiler error severity, +@emph{note}, is used for problems which are too mild for the standard +condition classes, typically hints about how efficiency might be +improved. + + +@node Errors During Macroexpansion, Read Errors, Error Severity, Error Messages +@comment node-name, next, previous, up +@subsection Errors During Macroexpansion +@cindex Macroexpansion, errors during + +The compiler handles errors that happen during macroexpansion, turning +them into compiler errors. If you want to debug the error (to debug a +macro), you can set @code{*break-on-signals*} to @code{error}. For +example, this definition: + +@lisp +(defun foo (e l) + (do ((current l (cdr current)) + ((atom current) nil)) + (when (eq (car current) e) (return current)))) +@end lisp + +gives this error: + +@example +in: DEFUN FOO +(DO ((CURRENT L #) (# NIL)) (WHEN (EQ # E) (RETURN CURRENT)) ) +caught ERROR: +(in macroexpansion of (DO # #)) +(hint: For more precise location, try *BREAK-ON-SIGNALS*.) +DO step variable is not a symbol: (ATOM CURRENT) +@end example + + +@node Read Errors, , Errors During Macroexpansion, Error Messages +@comment node-name, next, previous, up +@subsection Read Errors +@cindex Read errors, compiler + +SBCL's compiler (unlike CMUCL's) does not attempt to recover from read +errors when reading a source file, but instead just reports the +offending character position and gives up on the entire source file. + + +@c + + +@node Handling of Types, Compiler Policy, Error Messages, The Compiler +@comment node-name, next, previous, up +@section The Compiler's Handling of Types + +The most unusual features of the SBCL compiler (which is very similar +to the original CMUCL compiler, also known as @dfn{Python}) is its +unusually sophisticated understanding of the Common Lisp type system +and its unusually conservative approach to the implementation of type +declarations. These two features reward the use of type declarations +throughout development, even when high performance is not a +concern. Also, as discussed in the chapter on performance +(@pxref{Efficiency}), the use of appropriate type declarations can be +very important for performance as well. + +The SBCL compiler, like the related compiler in CMUCL, treats type +declarations much differently than other Lisp compilers. By default +(@emph{i.e.}, at ordinary levels of the @code{safety} compiler +optimization parameter), the compiler doesn't blindly believe most +type declarations; it considers them assertions about the program that +should be checked. + +The SBCL compiler also has a greater knowledge of the +Common Lisp type system than other compilers. Support is incomplete +only for the @code{not}, @code{and} and @code{satisfies} +types. + +@c + + +@menu +* Implementation Limitations:: +* Type Errors at Compile Time:: +* Precise Type Checking:: +* Weakened Type Checking:: +* Getting Existing Programs to Run:: +@end menu + +@node Implementation Limitations, Type Errors at Compile Time, Handling of Types, Handling of Types +@comment node-name, next, previous, up +@subsection Implementation Limitations + + +Ideally, the compiler would consider @emph{all} type declarations to +be assertions, so that adding type declarations to a program, no +matter how incorrect they might be, would @emph{never} cause undefined +behavior. As of SBCL version 0.8.1, the compiler is known to fall +short of this goal in two areas: + + @itemize + +@item +@code{Proclaim}ed constraints on argument and result types of a +function are supposed to be checked by the function. If the function +type is proclaimed before function definition, type checks are +inserted by the compiler, but the standard allows the reversed order, +in which case the compiler will trust the declaration. + +@item +The compiler cannot check types of an unknown number of values; if the +number of generated values is unknown, but the number of consumed is +known, only consumed values are checked. + +@item +There are a few poorly characterized but apparently very uncommon +situations where a type declaration in an unexpected location will be +trusted and never checked by the compiler. + +@end itemize + +These are important bugs, but are not necessarily easy to fix, so they +may, alas, remain in the system for a while. + + +@node Type Errors at Compile Time, Precise Type Checking, Implementation Limitations, Handling of Types +@comment node-name, next, previous, up +@subsection Type Errors at Compile Time +@cindex Compile time type errors +@cindex Type checking, at compile time + +If the compiler can prove at compile time that some portion of the +program cannot be executed without a type error, then it will give a +warning at compile time. It is possible that the offending code would +never actually be executed at run-time due to some higher level +consistency constraint unknown to the compiler, so a type warning +doesn't always indicate an incorrect program. For example, consider +this code fragment: + + +@lisp +(defun raz (foo) + (let ((x (case foo + (:this 13) + (:that 9) + (:the-other 42)))) + (declare (fixnum x)) + (foo x))) +@end lisp + +Compilation produces this warning: + +@example +in: DEFUN RAZ +(CASE FOO (:THIS 13) (:THAT 9) (:THE-OTHER 42)) +--> LET COND IF COND IF COND IF +==> +(COND) +caught WARNING: This is not a FIXNUM: +NIL +@end example + +In this case, the warning means that if @code{foo} isn't any of +@code{:this}, @code{:that} or @code{:the-other}, then @code{x} will be +initialized to @code{nil}, which the @code{fixnum} declaration makes +illegal. The warning will go away if @code{ecase} is used instead of +@code{case}, or if @code{:the-other} is changed to @code{t}. + +This sort of spurious type warning happens moderately often in the +expansion of complex macros and in inline functions. In such cases, +there may be dead code that is impossible to correctly execute. The +compiler can't always prove this code is dead (could never be +executed), so it compiles the erroneous code (which will always signal +an error if it is executed) and gives a warning. + + +Type warnings are inhibited when the @code{sb-ext:inhibit-warnings} +optimization quality is @code{3}. @xref{Compiler Policy}. This +can be used in a local declaration to inhibit type warnings in a code +fragment that has spurious warnings. + + +@node Precise Type Checking, Weakened Type Checking, Type Errors at Compile Time, Handling of Types +@comment node-name, next, previous, up +@subsection Precise Type Checking +@cindex Precise type checking +@cindex Type checking, precise + +With the default compilation policy, all type declarations are +precisely checked, except in a few situations where they are simply +ignored instead. Precise checking means that the check is done as +though @code{typep} had been called with the exact type specifier that +appeared in the declaration. In SBCL, adding type declarations makes +code safer. (Except that remaining bugs in the compiler's handling of +types unfortunately provide some exceptions to this rule, see +@ref{Implementation Limitations}). + +If a variable is declared to be @code{(integer 3 17)} then its value +must always be an integer between @code{3} and @code{17}. If multiple +type declarations apply to a single variable, then all the +declarations must be correct; it is as though all the types were +intersected producing a single @code{and} type specifier. + +Argument and result type declarations are automatically enforced. If +you declare the type of a function argument, a type check will be done +when that function is called. In a function call, the called function +does the argument type checking. + +The types of structure slots are also checked. The value of a +structure slot must always be of the type indicated in any +@code{:type} slot option. + +In traditional Common Lisp compilers, not all type assertions are +checked, and type checks are not precise. Traditional compilers +blindly trust explicit type declarations, but may check the argument +type assertions for built-in functions. Type checking is not precise, +since the argument type checks will be for the most general type legal +for that argument. In many systems, type declarations suppress what +little type checking is being done, so adding type declarations makes +code unsafe. This is a problem since it discourages writing type +declarations during initial coding. In addition to being more error +prone, adding type declarations during tuning also loses all the +benefits of debugging with checked type assertions. + +To gain maximum benefit from the compiler's type checking, you should +always declare the types of function arguments and structure slots as +precisely as possible. This often involves the use of @code{or}, +@code{member}, and other list-style type specifiers. + + +@node Weakened Type Checking, Getting Existing Programs to Run, Precise Type Checking, Handling of Types +@comment node-name, next, previous, up +@subsection Weakened Type Checking +@cindex Weakened type checking +@cindex Type checking, weakened + +At one time, CMUCL supported another level of type checking, +``weakened type checking'', when the value for the @code{speed} +optimization quality is greater than @code{safety}, and @code{safety} +is not @code{0}. The CMUCL manual still has a description of it, but +even the CMU CL code no longer corresponds to the manual. Some of this +partial safety checking lingers on in SBCL, but it's not a supported +feature, and should not be relied on. If you ask the compiler to +optimize @code{speed} to a higher level than @code{safety}, your +program is performing without a safety net, because SBCL may at its +option believe any or all type declarations with either partial or +nonexistent runtime checking. + + +@node Getting Existing Programs to Run, , Weakened Type Checking, Handling of Types +@comment node-name, next, previous, up +@subsection Getting Existing Programs to Run +@cindex Existing programs, to run +@cindex Types, portability +@cindex Compatibility with other Lisps +@c (should also have an entry in the non-ANSI-isms section)--> + +Since SBCL's compiler, like CMUCL's compiler, does much more +comprehensive type checking than most Lisp compilers, SBCL may detect +type errors in programs that have been debugged using other +compilers. These errors are mostly incorrect declarations, although +compile-time type errors can find actual bugs if parts of the program +have never been tested. + +Some incorrect declarations can only be detected by run-time type +checking. It is very important to initially compile a program with +full type checks (high @code{safety} optimization) and then test this +safe version. After the checking version has been tested, then you can +consider weakening or eliminating type checks. @emph{This applies +even to previously debugged programs,} because the SBCL compiler does +much more type inference than other Common Lisp compilers, so an +incorrect declaration can do more damage. + +The most common problem is with variables whose constant initial value +doesn't match the type declaration. Incorrect constant initial values +will always be flagged by a compile-time type error, and they are +simple to fix once located. Consider this code fragment: + +@lisp +(prog (foo) + (declare (fixnum foo)) + (setq foo ...) + ...) +@end lisp + +Here @code{foo} is given an initial value of @code{nil}, but is +declared to be a @code{fixnum}. Even if it is never read, the initial +value of a variable must match the declared type. There are two ways +to fix this problem. Change the declaration + +@lisp +(prog (foo) + (declare (type (or fixnum null) foo)) + (setq foo ...) + ...) +@end lisp + +or change the initial value + +@lisp +(prog ((foo 0)) + (declare (fixnum foo)) + (setq foo ...) + ...) +@end lisp + +It is generally preferable to change to a legal initial value rather +than to weaken the declaration, but sometimes it is simpler to weaken +the declaration than to try to make an initial value of the +appropriate type. + +Another declaration problem occasionally encountered is incorrect +declarations on @code{defmacro} arguments. This can happen when a +function is converted into a macro. Consider this macro: + +@lisp +(defmacro my-1+ (x) + (declare (fixnum x)) + `(the fixnum (1+ ,x))) +@end lisp + +Although legal and well-defined Common Lisp code, this meaning of this +definition is almost certainly not what the writer intended. For +example, this call is illegal: + +@lisp +(my-1+ (+ 4 5)) +@end lisp + +This call is illegal because the argument to the macro is @code{(+ 4 +5)}, which is a @code{list}, not a @code{fixnum}. Because of macro +semantics, it is hardly ever useful to declare the types of macro +arguments. If you really want to assert something about the type of +the result of evaluating a macro argument, then put a @code{the} in +the expansion: + +@lisp +(defmacro my-1+ (x) + `(the fixnum (1+ (the fixnum ,x)))) +@end lisp + +In this case, it would be stylistically preferable to change this +macro back to a function and declare it inline. +@c + +Some more subtle problems are caused by incorrect declarations that +can't be detected at compile time. Consider this code: + +@lisp +(do ((pos 0 (position #\a string :start (1+ pos)))) + ((null pos)) + (declare (fixnum pos)) + ...) +@end lisp + +Although @code{pos} is almost always a @code{fixnum}, it is @code{nil} +at the end of the loop. If this example is compiled with full type +checks (the default), then running it will signal a type error at the +end of the loop. If compiled without type checks, the program will go +into an infinite loop (or perhaps @code{position} will complain +because @code{(1+ nil)} isn't a sensible start.) Why? Because if you +compile without type checks, the compiler just quietly believes the +type declaration. Since the compiler believes that @code{pos} is +always a @code{fixnum}, it believes that @code{pos} is never +@code{nil}, so @code{(null pos)} is never true, and the loop exit test +is optimized away. Such errors are sometimes flagged by unreachable +code notes, but it is still important to initially compile and test +any system with full type checks, even if the system works fine when +compiled using other compilers. + +In this case, the fix is to weaken the type declaration to @code{(or +fixnum null)} @footnote{Actually, this declaration is unnecessary in +SBCL, since it already knows that @code{position} returns a +non-negative @code{fixnum} or @code{nil}.}. + +Note that there is usually little performance penalty for weakening a +declaration in this way. Any numeric operations in the body can still +assume that the variable is a @code{fixnum}, since @code{nil} is not a +legal numeric argument. Another possible fix would be to say: + +@lisp +(do ((pos 0 (position #\a string :start (1+ pos)))) + ((null pos)) + (let ((pos pos)) + (declare (fixnum pos)) + ...)) +@end lisp + +This would be preferable in some circumstances, since it would allow a +non-standard representation to be used for the local @code{pos} +variable in the loop body. +@c + + +@node Compiler Policy, Open Coding and Inline Expansion, Handling of Types, The Compiler +@comment node-name, next, previous, up +@section Compiler Policy + +As of version 0.6.4, SBCL still uses most of the CMUCL code for +compiler policy. The CMUCL code has many features and high-quality +documentation, but the two unfortunately do not match. So this area of +the compiler and its interface needs to be cleaned up. Meanwhile, here +is some rudimentary documentation on the current behavior of the +system. + +Compiler policy is controlled by the @code{optimize} declaration. The +compiler supports the ANSI optimization qualities, and also an +extension @code{sb-ext:inhibit-warnings}. + +Ordinarily, when the @code{speed} quality is high, the compiler emits +notes to notify the programmer about its inability to apply various +optimizations. Setting @code{sb-ext:inhibit-warnings} to a value at +least as large as the @code{speed} quality inhibits this +notification. This can be useful to suppress notes about code which is +known to be unavoidably inefficient. (For example, the compiler issues +notes about having to use generic arithmetic instead of fixnum +arithmetic, which is not helpful for code which by design supports +arbitrary-sized integers instead of being limited to fixnums.) + +@quotation +Note: The basic functionality of the @code{optimize +inhibit-warnings} extension will probably be supported in all future +versions of the system, but it will probably be renamed when the +compiler and its interface are cleaned up. The current name is +misleading, because it mostly inhibits optimization notes, not +warnings. And making it an optimization quality is misleading, because +it shouldn't affect the resulting code at all. It may become a +declaration identifier with a name like +@code{sb-ext:inhibit-notes}, so that what's currently written. + +@lisp +(declaim (optimize (sb-ext:inhibit-warnings 2))) +@end lisp + +would become something like + +@lisp +(declaim (sb-ext:inhibit-notes 2)) +@end lisp + +@end quotation + +In early versions of SBCL, a @code{speed} value of zero was used to +enable byte compilation, but since version 0.7.0, SBCL only supports +native compilation. + +When @code{safety} is zero, almost all runtime checking of types, +array bounds, and so forth is suppressed. + +When @code{safety} is less than @code{speed}, any and all type checks +may be suppressed. At some point in the past, CMUCL had a more nuanced +interpretation of this (@pxref{Weakened Type Checking}). However, SBCL +doesn't support that interpretation, and setting @code{safety} less +than @code{speed} may have roughly the same effect as setting +@code{safety} to zero. + +The value of @code{space} mostly influences the compiler's decision +whether to inline operations, which tend to increase the size of +programs. Use the value @code{0} with caution, since it can cause the +compiler to inline operations so indiscriminately that the net effect +is to slow the program by causing cache misses or swapping. + +@c + + +@node Open Coding and Inline Expansion, , Compiler Policy, The Compiler +@comment node-name, next, previous, up +@section Open Coding and Inline Expansion +@cindex Open-coding +@cindex inline expansion +@cindex static functions + +Since Common Lisp forbids the redefinition of standard functions, the +compiler can have special knowledge of these standard functions +embedded in it. This special knowledge is used in various ways (open +coding, inline expansion, source transformation), but the implications +to the user are basically the same: + +@itemize + +@item +Attempts to redefine standard functions may be frustrated, since the +function may never be called. Although it is technically illegal to +redefine standard functions, users sometimes want to implicitly +redefine these functions when they are debugging using the +@code{trace} macro. Special-casing of standard functions can be +inhibited using the @code{notinline} declaration. + +@item +The compiler can have multiple alternate implementations of standard +functions that implement different trade-offs of speed, space and +safety. This selection is based on the compiler policy, @ref{Compiler +Policy}. + +@end itemize + +When a function call is @emph{open coded}, inline code whose effect is +equivalent to the function call is substituted for that function +call. When a function call is @emph{closed coded}, it is usually left +as is, although it might be turned into a call to a different function +with different arguments. As an example, if @code{nthcdr} were to be +open coded, then + +@lisp +(nthcdr 4 foobar) +@end lisp + +might turn into + +@lisp +(cdr (cdr (cdr (cdr foobar)))) +@end lisp + +or even + +@lisp +(do ((i 0 (1+ i)) + (list foobar (cdr foobar))) + ((= i 4) list)) +@end lisp + +If @code{nth} is closed coded, then + +@lisp +(nth x l) +@end lisp + +might stay the same, or turn into something like + +@lisp +(car (nthcdr x l)) +@end lisp + +In general, open coding sacrifices space for speed, but some functions +(such as @code{car}) are so simple that they are always +open-coded. Even when not open-coded, a call to a standard function +may be transformed into a different function call (as in the last +example) or compiled as @emph{static call}. Static function call uses +a more efficient calling convention that forbids redefinition. + diff --git a/doc/manual/debugger.texinfo b/doc/manual/debugger.texinfo new file mode 100644 index 0000000..3713a9a --- /dev/null +++ b/doc/manual/debugger.texinfo @@ -0,0 +1,1167 @@ +@node The Debugger, Efficiency, The Compiler, Top +@comment node-name, next, previous, up +@chapter The Debugger +@cindex Debugger + +The SBCL debugger (as the CMUCL debugger it was derived from) has very +good support for source-level debugging of compiled code. Although +some other debuggers allow access of variables by name, this seems to +be the first Lisp debugger that: + +@itemize + +@item +Tells you when a variable doesn't have a value because it hasn't been +initialized yet or has already been deallocated, or + +@item +Can display the precise source location corresponding to a code +location in the debugged program. + +@end itemize + +These features allow the debugging of compiled code to be made almost +indistinguishable from interpreted code debugging. + + +@menu +* Starting the Debugger:: +* The Debugger Command Loop:: +* Controlling Printing in the Debugger:: +* Stack Frames:: +* Variable Access:: +* Source Location Printing:: +* Debugger Policy Control:: +* Exiting Commands:: +* Information Commands:: +* Function Tracing:: +@end menu + +@node Starting the Debugger, The Debugger Command Loop, The Debugger, The Debugger +@comment node-name, next, previous, up +@section Starting the Debugger + +The debugger is an interactive command loop that allows a user to examine +the function call stack. The debugger is invoked when: + +@itemize + +@item +A @code{serious-condition} is signaled, and it is not handled, or + +@item +@code{error} is called, and the condition it signals is not handled, +or + +@item +the debugger is explicitly entered with the Lisp @code{break} or +@code{debug} functions. + +@end itemize + +When you enter the TTY debugger, it looks something like this: + +@example +debugger invoked on a TYPE-ERROR in thread 11184: + The value 3 is not of type LIST. +restarts (invokable by number or by possibly-abbreviated name): + 0: [ABORT ] Reduce debugger level (leaving debugger, returning to toplevel). + 1: [TOPLEVEL] Restart at toplevel READ/EVAL/PRINT loop. +(CAR 1 3)[:EXTERNAL] +0] +@end example + +The first group of lines describe what the error was that put us in +the debugger. In this case @code{car} was called on @code{3}. After +@samp{restarts} is a list of all the ways that we can restart +execution after this error. In this case, both options return to +top-level. After printing its banner, the debugger prints the current +frame and the debugger prompt. + + +@node The Debugger Command Loop, Controlling Printing in the Debugger, Starting the Debugger, The Debugger +@comment node-name, next, previous, up +@section The Debugger Command Loop +@cindex Evaluation, in the debugger + +The debugger is an interactive read-eval-print loop much like the normal +top-level, but some symbols are interpreted as debugger commands instead +of being evaluated. A debugger command starts with the symbol name of +the command, possibly followed by some arguments on the same line. Some +commands prompt for additional input. Debugger commands can be +abbreviated by any unambiguous prefix: @command{help} can be typed as +@samp{h}, @samp{he}, etc. For convenience, some commands have +ambiguous one-letter abbreviations: @samp{f} for @command{frame}. + +The package is not significant in debugger commands; any symbol with the +name of a debugger command will work. If you want to show the value of +a variable that happens also to be the name of a debugger command, you +can use the @command{list-locals} command or the @code{sb-debug:var} +function, or you can wrap the variable in a @code{progn} to hide it from +the command loop. + +The debugger prompt is ``@code{@var{frame}]}'', where @var{frame} is +the number of the current frame. Frames are numbered starting from +zero at the top (most recent call), increasing down to the bottom. +The current frame is the frame that commands refer to. The current +frame also provides the lexical environment for evaluation of +non-command forms. + + The debugger evaluates forms in the lexical environment of the +functions being debugged. The debugger can only access variables. +You can't @code{go} or @code{return-from} into a function, and you +can't call local functions. Special variable references are evaluated +with their current value (the innermost binding around the debugger +invocation) -- you don't get the value that the special had in the +current frame. For more information on debugger variable access, see +@ref{Variable Access}. + + +@node Controlling Printing in the Debugger, Stack Frames, The Debugger Command Loop, The Debugger +@comment node-name, next, previous, up +@section Controlling Printing in the Debugger + +In the debugger, it is possible to override the printing behaviour of +the REPL. + +@defvr {Variable} *debug-print-variable-alist* + +An association list describing new bindings for special variables +(typically *PRINT-FOO* variables) to be used within the debugger, e.g. +@lisp +((*PRINT-LENGTH* . 10) (*PRINT-LEVEL* . 6) (*PRINT-PRETTY* . NIL)) +@end lisp +The variables in the @code{car} position are bound to the values in +the @code{cdr} during the execution of some debug commands. When +evaluating arbitrary expressions in the debugger, the normal values of +the printer control variables are in effect. @c FIXME: is this correct? +@code{*debug-print-variable-alist*} does not contain any bindings +initially. + +@end defvr + +@node Stack Frames, Variable Access, Controlling Printing in the Debugger, The Debugger +@comment node-name, next, previous, up +@section Stack Frames +@cindex Stack frames + +A @dfn{stack frame} is the run-time representation of a call to a +function; the frame stores the state that a function needs to remember +what it is doing. Frames have: + +@itemize + +@item +@dfn{Variables} (@pxref{Variable Access}), which are the values being operated +on, and + +@item +@dfn{Arguments} to the call (which are really just particularly +interesting variables), and + +@item +A current location (@pxref{Source Location Printing}), which is the place in +the program where the function was running when it stopped to call +another function, or because of an interrupt or error. + +@end itemize + + +@menu +* Stack Motion:: +* How Arguments are Printed:: +* Function Names:: +* Funny Frames:: +* Debug Tail Recursion:: +* Unknown Locations and Interrupts:: +@end menu + +@node Stack Motion, How Arguments are Printed, Stack Frames, Stack Frames +@comment node-name, next, previous, up +@subsection Stack Motion + +These commands move to a new stack frame and print the name of the +function and the values of its arguments in the style of a Lisp +function call: + +@deffn {Debugger Command} up +Move up to the next higher frame. More recent function calls are +considered to be higher on the stack. +@end deffn + +@deffn {Debugger Command} down +Move down to the next lower frame. +@end deffn + +@deffn {Debugger Command} top +Move to the highest frame, that is, the frame where the debugger was +entered. +@end deffn + +@deffn {Debugger Command} bottom +Move to the lowest frame. +@end deffn + +@deffn {Debugger Command} frame [@var{n}] +Move to the frame with the specified number. Prompts for the number if not +supplied. The frame with number 0 is the frame where the debugger +was entered. +@end deffn + + +@node How Arguments are Printed, Function Names, Stack Motion, Stack Frames +@comment node-name, next, previous, up +@subsection How Arguments are Printed + +A frame is printed to look like a function call, but with the actual +argument values in the argument positions. So the frame for this call +in the source: + +@lisp +(myfun (+ 3 4) 'a) +@end lisp + +would look like this: + +@example +(MYFUN 7 A) +@end example + +All keyword and optional arguments are displayed with their actual +values; if the corresponding argument was not supplied, the value will +be the default. So this call: + +@lisp +(subseq "foo" 1) +@end lisp + +would look like this: + +@example +(SUBSEQ "foo" 1 3) +@end example + +And this call: + +@lisp +(string-upcase "test case") +@end lisp + +would look like this: + +@example +(STRING-UPCASE "test case" :START 0 :END NIL) +@end example + +The arguments to a function call are displayed by accessing the +argument variables. Although those variables are initialized to the +actual argument values, they can be set inside the function; in this +case the new value will be displayed. + +@code{&rest} arguments are handled somewhat differently. The value of +the rest argument variable is displayed as the spread-out arguments to +the call, so: + +@lisp +(format t "~A is a ~A." "This" 'test) +@end lisp + +would look like this: + +@example +(FORMAT T "~A is a ~A." "This" 'TEST) +@end example + +Rest arguments cause an exception to the normal display of keyword +arguments in functions that have both @code{&rest} and @code{&key} +arguments. In this case, the keyword argument variables are not +displayed at all; the rest arg is displayed instead. So for these +functions, only the keywords actually supplied will be shown, and the +values displayed will be the argument values, not values of the +(possibly modified) variables. + +If the variable for an argument is never referenced by the function, +it will be deleted. The variable value is then unavailable, so the +debugger prints @samp{#} instead of the value. Similarly, +if for any of a number of reasons the value of the variable is +unavailable or not known to be available (@pxref{Variable Access}), +then @samp{#} will be printed instead of the argument +value. + +Printing of argument values is controlled by +@code{*debug-print-variable-alist*}. @xref{Controlling Printing in +the Debugger}. + + +@node Function Names, Funny Frames, How Arguments are Printed, Stack Frames +@comment node-name, next, previous, up +@subsection Function Names + +If a function is defined by @code{defun}, @code{labels}, or +@code{flet}, then the debugger will print the actual function name +after the open parenthesis, like: + +@example +(STRING-UPCASE "test case" :START 0 :END NIL) +((SETF AREF) #\a "for" 1) +@end example + +Otherwise, the function name is a string, and will be printed in +quotes: + +@example +("DEFUN MYFUN" BAR) +("DEFMACRO DO" (DO ((I 0 (1+ I))) ((= I 13))) NIL) +("SETQ *GC-NOTIFY-BEFORE*") +@end example + +This string name is derived from the @code{def@var{mumble}} form +that encloses or expanded into the lambda, or the outermost enclosing +form if there is no @code{def@var{mumble}}. + + +@node Funny Frames, Debug Tail Recursion, Function Names, Stack Frames +@comment node-name, next, previous, up +@subsection Funny Frames +@cindex External entry points +@cindex Entry points, external +@cindex Block compilation, debugger implications +@cindex External, stack frame kind +@cindex Optional, stack frame kind +@cindex Cleanup, stack frame kind + +Sometimes the evaluator introduces new functions that are used to +implement a user function, but are not directly specified in the +source. The main place this is done is for checking argument type and +syntax. Usually these functions do their thing and then go away, and +thus are not seen on the stack in the debugger. But when you get some +sort of error during lambda-list processing, you end up in the +debugger on one of these funny frames. + +These funny frames are flagged by printing +``@code{[@var{keyword}]}'' after the parentheses. For example, +this call: + +@lisp +(car 'a 'b) +@end lisp + +will look like this: + +@example +(CAR 2 A)[:EXTERNAL] +@end example + +And this call: + +@lisp +(string-upcase "test case" :end) +@end lisp + +would look like this: + +@example +("SB!INT:&MORE processing" "test case" 1053984 1)[:OPTIONAL] +@end example + +As you can see, these frames have only a vague resemblance to the +original call. Fortunately, the error message displayed when you +enter the debugger will usually tell you what problem is (in these +cases, too many arguments and odd keyword arguments.) Also, if you go +down the stack to the frame for the calling function, you can display +the original source. @xref{Source Location Printing}. + +@c FIXME: is the block-compilation part correct for SBCL? + +With recursive or block compiled functions, an @code{:EXTERNAL} frame +may appear before the frame representing the first call to the +recursive function or entry to the compiled block. This is a +consequence of the way the compiler does block compilation: there is +nothing odd with your program. You will also see @code{:CLEANUP} +frames during the execution of @code{unwind-protect} cleanup +code. Note that inline expansion and open-coding affect what frames +are present in the debugger, see @ref{Debugger Policy Control}. +@comment FIXME: link here to section about open coding once it exists. +@c @ref{open-coding} + + +@node Debug Tail Recursion, Unknown Locations and Interrupts, Funny Frames, Stack Frames +@comment node-name, next, previous, up +@subsection Debug Tail Recursion +@cindex Tail recursion +@cindex Recursion, tail + +Both the compiler and the interpreter are ``properly tail recursive.'' +If a function call is in a tail-recursive position, the stack frame +will be deallocated @emph{at the time of the call}, rather than after +the call returns. Consider this backtrace: + +@example +(BAR ...) +(FOO ...) +@end example + +Because of tail recursion, it is not necessarily the case that +@code{FOO} directly called @code{BAR}. It may be that @code{FOO} +called some other function @code{FOO2} which then called @code{BAR} +tail-recursively, as in this example: + +@lisp +(defun foo () + ... + (foo2 ...) + ...) + +(defun foo2 (...) + ... + (bar ...)) + +(defun bar (...) + ...) +@end lisp + +Usually the elimination of tail-recursive frames makes debugging more +pleasant, since theses frames are mostly uninformative. If there is +any doubt about how one function called another, it can usually be +eliminated by finding the source location in the calling frame. +@xref{Source Location Printing}. + +The elimination of tail-recursive frames can be prevented by disabling +tail-recursion optimization, which happens when the @code{debug} +optimization quality is greater than @code{2}. +@xref{Debugger Policy Control}. + +@comment FIXME: reinstate this link once the chapter is in the manual. +@c For a more thorough discussion of tail recursion, @ref{tail-recursion}. + + +@node Unknown Locations and Interrupts, , Debug Tail Recursion, Stack Frames +@comment node-name, next, previous, up +@subsection Unknown Locations and Interrupts +@cindex Unknown code locations +@cindex Locations, unknown +@cindex Interrupts +@cindex Errors, run-time + +The debugger operates using special debugging information attached to +the compiled code. This debug information tells the debugger what it +needs to know about the locations in the code where the debugger can +be invoked. If the debugger somehow encounters a location not +described in the debug information, then it is said to be +@dfn{unknown}. If the code location for a frame is unknown, then some +variables may be inaccessible, and the source location cannot be +precisely displayed. + +There are three reasons why a code location could be unknown: + +@itemize + +@item +There is inadequate debug information due to the value of the @code{debug} +optimization quality. @xref{Debugger Policy Control}. + +@item +The debugger was entered because of an interrupt such as @key{C-c}. + +@item +A hardware error such as ``@samp{bus error}'' occurred in code that was +compiled unsafely due to the value of the @code{safety} optimization +quality. +@comment FIXME: reinstate link when section on optimize qualities exists. +@c @xref{optimize-declaration}. + +@end itemize + +In the last two cases, the values of argument variables are +accessible, but may be incorrect. For more details on when variable +values are accessible, @ref{Variable Value Availability}. + +It is possible for an interrupt to happen when a function call or +return is in progress. The debugger may then flame out with some +obscure error or insist that the bottom of the stack has been reached, +when the real problem is that the current stack frame can't be +located. If this happens, return from the interrupt and try again. + + +@node Variable Access, Source Location Printing, Stack Frames, The Debugger +@comment node-name, next, previous, up +@section Variable Access +@cindex Debug variables +@cindex Variables, debugger access + +There are two ways to access the current frame's local variables in +the debugger: @command{list-locals} and @code{sb-debug:var}. + +The debugger doesn't really understand lexical scoping; it has just +one namespace for all the variables in the current stack frame. If a +symbol is the name of multiple variables in the same function, then +the reference appears ambiguous, even though lexical scoping specifies +which value is visible at any given source location. If the scopes of +the two variables are not nested, then the debugger can resolve the +ambiguity by observing that only one variable is accessible. + +When there are ambiguous variables, the evaluator assigns each one a +small integer identifier. The @code{sb-debug:var} function uses this +identifier to distinguish between ambiguous variables. The +@command{list-locals} command prints the identifier. In the +following example, there are two variables named @code{X}. The first +one has identifier 0 (which is not printed), the second one has +identifier 1. + +@example +X = 1 +X#1 = 2 +@end example + +@deffn {Debugger Command} list-locals [@var{prefix}] +This command prints the name and value of all variables in the current +frame whose name has the specified @var{prefix}. @var{prefix} may be +a string or a symbol. If no @var{prefix} is given, then all available +variables are printed. If a variable has a potentially ambiguous +name, then the name is printed with a ``@code{#@var{identifier}}'' +suffix, where @var{identifier} is the small integer used to make the +name unique. +@end deffn + +@defun sb-debug:var @var{name} &optional @var{identifier} +This function returns the value of the variable in the current frame +with the specified @var{name}. If supplied, @var{identifier} +determines which value to return when there are ambiguous variables. + +When @var{name} is a symbol, it is interpreted as the symbol name of +the variable, i.e. the package is significant. If @var{name} is an +uninterned symbol (gensym), then return the value of the uninterned +variable with the same name. If @var{name} is a string, +@code{sb-debug:var} interprets it as the prefix of a variable name +that must unambiguously complete to the name of a valid variable. + +@var{identifier} is used to disambiguate the variable name; use +@command{list-locals} to find out the identifiers. +@end defun + + +@menu +* Variable Value Availability:: +* Note On Lexical Variable Access:: +@end menu + +@node Variable Value Availability, Note On Lexical Variable Access, Variable Access, Variable Access +@comment node-name, next, previous, up +@subsection Variable Value Availability +@cindex Availability of debug variables +@cindex Validity of debug variables +@cindex Debug optimization quality + +The value of a variable may be unavailable to the debugger in portions +of the program where Lisp says that the variable is defined. If a +variable value is not available, the debugger will not let you read or +write that variable. With one exception, the debugger will never +display an incorrect value for a variable. Rather than displaying +incorrect values, the debugger tells you the value is unavailable. + +The one exception is this: if you interrupt (e.g., with @key{C-c}) or +if there is an unexpected hardware error such as ``@samp{bus error}'' +(which should only happen in unsafe code), then the values displayed +for arguments to the interrupted frame might be +incorrect.@footnote{Since the location of an interrupt or hardware +error will always be an unknown location, non-argument variable values +will never be available in the interrupted frame. @xref{Unknown +Locations and Interrupts}.} This exception applies only to the +interrupted frame: any frame farther down the stack will be fine. + +The value of a variable may be unavailable for these reasons: + +@itemize + +@item +The value of the @code{debug} optimization quality may have omitted debug +information needed to determine whether the variable is available. +Unless a variable is an argument, its value will only be available when +@code{debug} is at least @code{2}. + +@item +The compiler did lifetime analysis and determined that the value was no longer +needed, even though its scope had not been exited. Lifetime analysis is +inhibited when the @code{debug} optimization quality is @code{3}. + +@item +The variable's name is an uninterned symbol (gensym). To save space, the +compiler only dumps debug information about uninterned variables when the +@code{debug} optimization quality is @code{3}. + +@item +The frame's location is unknown (@pxref{Unknown Locations and +Interrupts}) because the debugger was entered due to an interrupt or +unexpected hardware error. Under these conditions the values of +arguments will be available, but might be incorrect. This is the +exception mentioned above. + +@item +The variable (or the code referencing it) was optimized out +of existence. Variables with no reads are always optimized away. The +degree to which the compiler deletes variables will depend on the +value of the @code{compilation-speed} optimization quality, but most +source-level optimizations are done under all compilation policies. + +@item +The variable is never set and its definition looks like +@lisp +(LET ((var1 var2)) + ...) +@end lisp +In this case, @code{var1} is substituted with @code{var2}. + +@item +The variable is never set and is referenced exactly once. In this +case, the reference is substituted with the variable initial value. + +@end itemize + +Since it is especially useful to be able to get the arguments to a +function, argument variables are treated specially when the +@code{speed} optimization quality is less than @code{3} and the +@code{debug} quality is at least @code{1}. With this compilation +policy, the values of argument variables are almost always available +everywhere in the function, even at unknown locations. For +non-argument variables, @code{debug} must be at least @code{2} for +values to be available, and even then, values are only available at +known locations. + + +@node Note On Lexical Variable Access, , Variable Value Availability, Variable Access +@comment node-name, next, previous, up +@subsection Note On Lexical Variable Access + +When the debugger command loop establishes variable bindings for +available variables, these variable bindings have lexical scope and +dynamic extent.@footnote{The variable bindings are actually created +using the Lisp @code{symbol-macrolet} special form.} You can close +over them, but such closures can't be used as upward funargs. + +You can also set local variables using @code{setq}, but if the +variable was closed over in the original source and never set, then +setting the variable in the debugger may not change the value in all +the functions the variable is defined in. Another risk of setting +variables is that you may assign a value of a type that the compiler +proved the variable could never take on. This may result in bad +things happening. + + +@node Source Location Printing, Debugger Policy Control, Variable Access, The Debugger +@comment node-name, next, previous, up +@section Source Location Printing +@cindex Source location printing, debugger + +One of the debugger's capabilities is source level debugging of +compiled code. These commands display the source location for the +current frame: + +@deffn {Debugger Command} source [@var{context}] +This command displays the file that the current frame's function was +defined from (if it was defined from a file), and then the source form +responsible for generating the code that the current frame was +executing. If @var{context} is specified, then it is an integer +specifying the number of enclosing levels of list structure to print. +@end deffn + +The source form for a location in the code is the innermost list present +in the original source that encloses the form responsible for generating +that code. If the actual source form is not a list, then some enclosing +list will be printed. For example, if the source form was a reference +to the variable @code{*some-random-special*}, then the innermost +enclosing evaluated form will be printed. Here are some possible +enclosing forms: + +@lisp +(let ((a *some-random-special*)) + ...) + +(+ *some-random-special* ...) +@end lisp + +If the code at a location was generated from the expansion of a macro +or a source-level compiler optimization, then the form in the original +source that expanded into that code will be printed. Suppose the file +@file{/usr/me/mystuff.lisp} looked like this: + +@lisp +(defmacro mymac () + '(myfun)) + +(defun foo () + (mymac) + ...) +@end lisp + +If @code{foo} has called @code{myfun}, and is waiting for it to +return, then the @command{source} command would print: + +@example +; File: /usr/me/mystuff.lisp + +(MYMAC) +@end example + +Note that the macro use was printed, not the actual function call form, +@code{(myfun)}. + +If enclosing source is printed by giving an argument to +@command{source} or @command{vsource}, then the actual source form is +marked by wrapping it in a list whose first element is +@samp{#:***HERE***}. In the previous example, @code{source 1} would +print: + +@example +; File: /usr/me/mystuff.lisp + +(DEFUN FOO () + (#:***HERE*** + (MYMAC)) + ...) +@end example + + +@menu +* How the Source is Found:: +* Source Location Availability:: +@end menu + +@node How the Source is Found, Source Location Availability, Source Location Printing, Source Location Printing +@comment node-name, next, previous, up +@subsection How the Source is Found + +If the code was defined from Lisp by @code{compile} or +@code{eval}, then the source can always be reliably located. If the +code was defined from a @file{fasl} file created by +@code{compile-file}, then the debugger gets the source forms it +prints by reading them from the original source file. This is a +potential problem, since the source file might have moved or changed +since the time it was compiled. + +The source file is opened using the @code{truename} of the source file +pathname originally given to the compiler. This is an absolute pathname +with all logical names and symbolic links expanded. If the file can't +be located using this name, then the debugger gives up and signals an +error. + +If the source file can be found, but has been modified since the time it was +compiled, the debugger prints this warning: + +@example +; File has been modified since compilation: +; @var{filename} +; Using form offset instead of character position. +@end example + +where @var{filename} is the name of the source file. It then proceeds +using a robust but not foolproof heuristic for locating the source. +This heuristic works if: + +@itemize + +@item +No top-level forms before the top-level form containing the source +have been added or deleted, and + +@item +The top-level form containing the source has not been modified much. +(More precisely, none of the list forms beginning before the source +form have been added or deleted.) + +@end itemize + +If the heuristic doesn't work, the displayed source will be wrong, but will +probably be near the actual source. If the ``shape'' of the top-level form in +the source file is too different from the original form, then an error will be +signaled. When the heuristic is used, the the source location commands are +noticeably slowed. + +Source location printing can also be confused if (after the source was +compiled) a read-macro you used in the code was redefined to expand +into something different, or if a read-macro ever returns the same +@code{eq} list twice. If you don't define read macros and don't use +@code{##} in perverted ways, you don't need to worry about this. + + +@node Source Location Availability, , How the Source is Found, Source Location Printing +@comment node-name, next, previous, up +@subsection Source Location Availability +@cindex Debug optimization quality +@cindex Block, basic +@cindex Block, start location + +Source location information is only available when the @code{debug} +optimization quality is at least @code{2}. If source location +information is unavailable, the source commands will give an error +message. + +If source location information is available, but the source location +is unknown because of an interrupt or unexpected hardware error +(@pxref{Unknown Locations and Interrupts}), then the command will +print: + +@example +Unknown location: using block start. +@end example + +and then proceed to print the source location for the start of the +@emph{basic block} enclosing the code location. It's a bit +complicated to explain exactly what a basic block is, but here are +some properties of the block start location: + +@itemize + +@item The block start location may be the same as the true location. + +@item The block start location will never be later in the the +program's flow of control than the true location. + +@item No conditional control structures (such as @code{if}, +@code{cond}, @code{or}) will intervene between the block start and the +true location (but note that some conditionals present in the original +source could be optimized away.) Function calls @emph{do not} end +basic blocks. + +@item The head of a loop will be the start of a block. + +@item The programming language concept of ``block structure'' and the +Lisp @code{block} special form are totally unrelated to the compiler's +basic block. + +@end itemize + +In other words, the true location lies between the printed location and the +next conditional (but watch out because the compiler may have changed the +program on you.) + + +@node Debugger Policy Control, Exiting Commands, Source Location Printing, The Debugger +@comment node-name, next, previous, up +@section Debugger Policy Control +@cindex Policy, debugger +@cindex Debug optimization quality +@cindex Optimize declaration +@cindex Inline expansion +@cindex Semi-inline expansion + +The compilation policy specified by @code{optimize} declarations +affects the behavior seen in the debugger. The @code{debug} quality +directly affects the debugger by controlling the amount of debugger +information dumped. Other optimization qualities have indirect but +observable effects due to changes in the way compilation is done. + +Unlike the other optimization qualities (which are compared in relative value +to evaluate tradeoffs), the @code{debug} optimization quality is directly +translated to a level of debug information. This absolute interpretation +allows the user to count on a particular amount of debug information being +available even when the values of the other qualities are changed during +compilation. These are the levels of debug information that correspond to the +values of the @code{debug} quality: + +@table @code + +@item 0 +Only the function name and enough information to allow the stack to +be parsed. + +@item > 0 +Any level greater than @code{0} gives level @code{0} plus all argument +variables. Values will only be accessible if the argument variable is +never set and @code{speed} is not @code{3}. SBCL allows any real +value for optimization qualities. It may be useful to specify +@code{0.5} to get backtrace argument display without argument +documentation. + +@item 1 +Level @code{1} provides argument documentation (printed arglists) and +derived argument/result type information. This makes @code{describe} +more informative, and allows the compiler to do compile-time argument +count and type checking for any calls compiled at run-time. This is +the default. + +@item 2 +Level @code{1} plus all interned local variables, source location +information, and lifetime information that tells the debugger when +arguments are available (even when @code{speed} is @code{3} or the +argument is set). + +@item > 2 +Any level greater than @code{2} gives level @code{2} and in addition +disables tail-call optimization, so that the backtrace will contain +frames for all invoked functions, even those in tail positions. + +@item 3 +Level @code{2} plus all uninterned variables. In addition, lifetime +analysis is disabled (even when @code{speed} is @code{3}), ensuring +that all variable values are available at any known location within +the scope of the binding. This has a speed penalty in addition to the +obvious space penalty. + +@item > (max speed space) +If @code{debug} is greater than both @code{speed} and @code{space}, +the command @command{return} can be used to continue execution by +returning a value from the current stack frame. + +@end table + +As you can see, if the @code{speed} quality is @code{3}, debugger performance is +degraded. This effect comes from the elimination of argument variable +special-casing (@pxref{Variable Value Availability}). Some degree of +speed/debuggability tradeoff is unavoidable, but the effect is not too drastic +when @code{debug} is at least @code{2}. + +In addition to @code{inline} and @code{notinline} declarations, the +relative values of the @code{speed} and @code{space} qualities also +change whether functions are inline expanded. +@comment FIXME: link to section about inline expansion when it exists +@c (\pxlref{inline-expansion}.) +If a function is inline expanded, then +there will be no frame to represent the call, and the arguments will +be treated like any other local variable. Functions may also be +``semi-inline'', in which case there is a frame to represent the call, +but the call is to an optimized local version of the function, not to +the original function. + + +@node Exiting Commands, Information Commands, Debugger Policy Control, The Debugger +@comment node-name, next, previous, up +@section Exiting Commands + +These commands get you out of the debugger. + +@deffn {Debugger Command} toplevel +Throw to top level. +@end deffn + +@deffn {Debugger Command} restart [@var{n}] +Invokes the @var{n}th restart case as displayed by the @code{error} +command. If @var{n} is not specified, the available restart cases are +reported. +@end deffn + +@deffn {Debugger Command} continue +Calls @code{continue} on the condition given to @code{debug}. If there is no +restart case named @var{continue}, then an error is signaled. +@end deffn + +@deffn {Debugger Command} abort +Calls @code{abort} on the condition given to @code{debug}. This is +useful for popping debug command loop levels or aborting to top level, +as the case may be. +@end deffn + +@deffn {Debugger Command} return @var{value} +Returns @var{value} from the current stack frame. This command is +available when the @code{debug} optimization quality is greater than +both @code{speed} and @code{space}. Care must be taken that the value +is of the same type as SBCL expects the stack frame to return. +@end deffn + + +@node Information Commands, Function Tracing, Exiting Commands, The Debugger +@comment node-name, next, previous, up +@section Information Commands + +Most of these commands print information about the current frame or +function, but a few show general information. + +@deffn {Debugger Command} help +@deffnx {Debugger Command} ? +Displays a synopsis of debugger commands. +@end deffn + +@deffn {Debugger Command} describe +Calls @code{describe} on the current function and displays the number of +local variables. +@end deffn + +@deffn {Debugger Command} print +Displays the current function call as it would be displayed by moving to +this frame. +@end deffn + +@deffn {Debugger Command} error +Prints the condition given to @code{invoke-debugger} and the active +proceed cases. +@end deffn + +@deffn {Debugger Command} backtrace [@var{n}] +Displays all the frames from the current to the bottom. Only shows +@var{n} frames if specified. The printing is controlled by @code{*debug-print-variable-alist*}. +@end deffn + +@comment FIXME (rudi 2004-03-31): sbcl doesn't support breakpoints +@comment and stepping as of version 0.8.9. The `list-locations' +@comment command works, but executing a function leads to an error +@comment when a breakpoint is hit. When stepping works, the +@comment commented-out section below should be reinstated and the +@comment example output updated to correspont to sbcl's behaviour. + +@c @node Breakpoint Commands, , Information Commands, The Debugger +@c @comment node-name, next, previous, up +@c @section Breakpoint Commands +@c @cindex Breakpoints + +@c SBCL supports setting of breakpoints inside compiled functions and +@c stepping of compiled code. Breakpoints can only be set at at known +@c locations (@pxref{Unknown Locations and Interrupts}), so these +@c commands are largely useless unless the @code{debug} optimize quality +@c is at least @code{2} (@pxref{Debugger Policy Control}). These +@c commands manipulate breakpoints: + +@c @deffn {Debugger Command} breakpoint @var{location} [@var{option} @var{value}]* +@c Set a breakpoint in some function. @var{location} may be an integer +@c code location number (as displayed by @command{list-locations}) or a +@c keyword. The keyword can be used to indicate setting a breakpoint at +@c the function start (@code{:start}, @code{:s}) or function end +@c (@code{:end}, @code{:e}). The @command{breakpoint} command has +@c @code{:condition}, @code{:break}, @code{:print} and @code{:function} +@c options which work similarly to the @code{trace} options. +@c @end deffn + +@c @deffn {Debugger Command} list-locations [@var{function}] +@c @deffnx {Debugger Command} ll [@var{function}] +@c List all the code locations in the current frame's function, or in +@c @var{function} if it is supplied. The display format is the code +@c location number, a colon and then the source form for that location: + +@c @example +@c 3: (1- N) +@c @end example + +@c If consecutive locations have the same source, then a numeric range +@c like @code{3-5:} will be printed. For example, a default function +@c call has a known location both immediately before and after the call, +@c which would result in two code locations with the same source. The +@c listed function becomes the new default function for breakpoint +@c setting (via the @command{breakpoint}) command. +@c @end deffn + +@c @deffn {Debugger Command} list-breakpoints +@c @deffnx {Debugger Command} lb +@c List all currently active breakpoints with their breakpoint number. +@c @end deffn + +@c @deffn {Debugger Command} delete-breakpoint [@var{number}] +@c @deffnx {Debugger Command} db [@var{number}] +@c Delete a breakpoint specified by its breakpoint number. If no number +@c is specified, delete all breakpoints. +@c @end deffn + +@c @deffn {Debugger Command} step +@c Step to the next possible breakpoint location in the current function. +@c This always steps over function calls, instead of stepping into them. +@c @end deffn + + +@c @menu +@c * Breakpoint Example:: +@c @end menu + +@c @node Breakpoint Example, , Breakpoint Commands, Breakpoint Commands +@c @comment node-name, next, previous, up +@c @subsection Breakpoint Example + +@c Consider this definition of the factorial function: + +@c @lisp +@c (defun ! (n) +@c (if (zerop n) +@c 1 +@c (* n (! (1- n))))) +@c @end lisp + +@c This debugger session demonstrates the use of breakpoints: + +@c @example +@c * (break) ; invoke debugger + +@c debugger invoked on a SIMPLE-CONDITION in thread 11184: break + +@c restarts (invokable by number or by possibly-abbreviated name): +@c 0: [CONTINUE] Return from BREAK. +@c 1: [ABORT ] Reduce debugger level (leaving debugger, returning to toplevel). +@c 2: [TOPLEVEL] Restart at toplevel READ/EVAL/PRINT loop. +@c ("varargs entry for top level local call BREAK" "break") +@c 0] ll #'! + +@c 0-1: (SB-INT:NAMED-LAMBDA ! (N) (BLOCK ! (IF (ZEROP N) 1 (* N (! #))))) +@c 2: (BLOCK ! (IF (ZEROP N) 1 (* N (! (1- N))))) +@c 3: (ZEROP N) +@c 4: (* N (! (1- N))) +@c 5: (1- N) +@c 6: (! (1- N)) +@c 7-8: (* N (! (1- N))) +@c 9-10: (IF (ZEROP N) 1 (* N (! (1- N)))) +@c 0] br 4 + +@c (* N (! (1- N))) +@c 1: 4 in ! +@c added +@c 0] toplevel + +@c FIXME: SBCL errored out, and not in the expected way ... Copying the +@c output verbatim from the CMUCL manual for now. + +@c common-lisp-user> (! 10) ; Call the function + +@c *Breakpoint hit* + +@c Restarts: +@c 0: [CONTINUE] Return from BREAK. +@c 1: [ABORT ] Return to Top-Level. + +@c Debug (type H for help) + +@c (! 10) ; We are now in first call (arg 10) before the multiply +@c Source: (* N (! (1- N))) +@c 3] st + +@c *Step* + +@c (! 10) ; We have finished evaluation of (1- n) +@c Source: (1- N) +@c 3] st + +@c *Breakpoint hit* + +@c Restarts: +@c 0: [CONTINUE] Return from BREAK. +@c 1: [ABORT ] Return to Top-Level. + +@c Debug (type H for help) + +@c (! 9) ; We hit the breakpoint in the recursive call +@c Source: (* N (! (1- N))) +@c 3] +@c @end example + + +@node Function Tracing, , Information Commands, The Debugger +@comment node-name, next, previous, up +@section Function Tracing +@cindex Tracing +@cindex Function, tracing + +The tracer causes selected functions to print their arguments and +their results whenever they are called. Options allow conditional +printing of the trace information and conditional breakpoints on +function entry or exit. + +@comment rudi 2004-03-26: The docstring for `trace' is quite comprehensive, +@comment so refer to it (see also ``OAOO'') +The docstrings for @code{trace} and @code{untrace} explain SBCL's +tracing facility. + +@comment FIXME rudi 2004-03-26: revive the documentation of variables +@comment describing trace behaviour: *trace-encapsulate-default*, +@comment *max-trace-indentation* and friends. Some of these are +@comment mentioned (perhaps under different names) in the cmucl +@comment manual. + +@comment FIXME rudi 2004-03-26: encapsulate is (per TODO file as of +@comment 0.8.9) in a state of flux. When it's sorted out, revive the +@comment cmucl documentation. + + + diff --git a/doc/manual/efficiency.texinfo b/doc/manual/efficiency.texinfo new file mode 100644 index 0000000..0a82814 --- /dev/null +++ b/doc/manual/efficiency.texinfo @@ -0,0 +1,152 @@ +@node Efficiency, Beyond The ANSI Standard, The Debugger, Top +@comment node-name, next, previous, up +@chapter Efficiency + +FIXME: The material in the CMUCL manual about getting good +performance from the compiler should be reviewed, reformatted in +Texinfo, lightly edited for SBCL, and substituted into this +manual. In the meantime, the original CMUCL manual is still 95+% +correct for the SBCL version of the Python compiler. See the +sections + +@itemize +@item Advanced Compiler Use and Efficiency Hints +@item Advanced Compiler Introduction +@item More About Types in Python +@item Type Inference +@item Source Optimization +@item Tail Recursion +@item Local Call +@item Block Compilation +@item Inline Expansion +@item Object Representation +@item Numbers +@item General Efficiency Hints +@item Efficiency Notes +@end itemize + +Besides this information from the CMUCL manual, there are a few other +points to keep in mind. + +@itemize + +@item +The CMUCL manual doesn't seem to state it explicitly, but Python has a +mental block about type inference when assignment is involved. Python +is very aggressive and clever about inferring the types of values +bound with @code{let}, @code{let*}, inline function call, and so +forth. However, it's much more passive and dumb about inferring the +types of values assigned with @code{setq}, @code{setf}, and +friends. It would be nice to fix this, but in the meantime don't +expect that just because it's very smart about types in most respects +it will be smart about types involved in assignments. (This doesn't +affect its ability to benefit from explicit type declarations +involving the assigned variables, only its ability to get by without +explicit type declarations.) + +@c + +@item +Since the time the CMUCL manual was written, CMUCL (and thus SBCL) has +gotten a generational garbage collector. This means that there are +some efficiency implications of various patterns of memory usage which +aren't discussed in the CMUCL manual. (Some new material should be +written about this.) + +@item +SBCL has some important known efficiency problems. Perhaps the most +important are + +@itemize @minus + +@item +There is no support for the ANSI @code{dynamic-extent} declaration, +not even for closures or @code{&rest} lists. + +@item +The garbage collector is not particularly efficient, at least on +platforms without the generational collector (as of SBCL 0.8.9, all +except x86). + +@item +Various aspects of the PCL implementation of CLOS are more inefficient +than necessary. + +@end itemize + +@end itemize + +Finally, note that Common Lisp defines many constructs which, in +the infamous phrase, ``could be compiled efficiently by a +sufficiently smart compiler''. The phrase is infamous because +making a compiler which actually is sufficiently smart to find all +these optimizations systematically is well beyond the state of the art +of current compiler technology. Instead, they're optimized on a +case-by-case basis by hand-written code, or not optimized at all if +the appropriate case hasn't been hand-coded. Some cases where no such +hand-coding has been done as of SBCL version 0.6.3 include + +@itemize + +@item +@code{(reduce #'f x)} where the type of @code{x} is known at compile +time + +@item +various bit vector operations, e.g. @code{(position 0 +some-bit-vector)} + +@end itemize + +If your system's performance is suffering because of some construct +which could in principle be compiled efficiently, but which the SBCL +compiler can't in practice compile efficiently, consider writing a +patch to the compiler and submitting it for inclusion in the main +sources. Such code is often reasonably straightforward to write; +search the sources for the string ``@code{deftransform}'' to find many +examples (some straightforward, some less so). + +@menu +* Modular arithmetic:: +@end menu + +@node Modular arithmetic, , Efficiency, Efficiency +@comment node-name, next, previous, up +@section Modular arithmetic + +Some numeric functions have a property: @var{N} lower bits of the +result depend only on @var{N} lower bits of (all or some) +arguments. If the compiler sees an expression of form @code{(logand +@var{exp} @var{mask})}, where @var{exp} is a tree of such ``good'' +functions and @var{mask} is known to be of type @code{(unsigned-byte +@var{w})}, where @var{w} is a ``good'' width, all intermediate results +will be cut to @var{w} bits (but it is not done for variables and +constants!). This often results in an ability to use simple machine +instructions for the functions. + +Consider an example. + +@lisp +(defun i (x y) + (declare (type (unsigned-byte 32) x y)) + (ldb (byte 32 0) (logxor x (lognot y)))) +@end lisp + +The result of @code{(lognot y)} will be negative and of type +@code{(signed-byte 33)}, so a naive implementation on a 32-bit +platform is unable to use 32-bit arithmetic here. But modular +arithmetic optimizer is able to do it: because the result is cut down +to 32 bits, the compiler will replace @code{logxor} and @code{lognot} +with versions cutting results to 32 bits, and because terminals +(here---expressions @code{x} and @code{y}) are also of type +@code{(unsigned-byte 32)}, 32-bit machine arithmetic can be used. + + +As of SBCL 0.8.5 ``good'' functions are @code{+}, @code{-}; +@code{logand}, @code{logior}, @code{logxor}, @code{lognot} and their +combinations; and @code{ash} with the positive second +argument. ``Good'' widths are 32 on HPPA, MIPS, PPC, Sparc and X86 and +64 on Alpha. While it is possible to support smaller widths as well, +currently it is not implemented. diff --git a/doc/manual/ffi.texinfo b/doc/manual/ffi.texinfo new file mode 100644 index 0000000..68d5cce --- /dev/null +++ b/doc/manual/ffi.texinfo @@ -0,0 +1,1246 @@ +@node The Foreign Function Interface, Function Index, Beyond The ANSI Standard, Top +@comment node-name, next, previous, up +@chapter The Foreign Function Interface + +This chapter describes SBCL's interface to C programs and +libraries (and, since C interfaces are a sort of @emph{ingua +franca} of the Unix world, to other programs and libraries in +general.) + +@quotation +Note: In the modern Lisp world, the usual term for this functionality +is Foreign Function Interface, or @acronym{FFI}, where despite the +mention of ``function'' in this term, @acronym{FFI} also +refers to direct manipulation of C data structures as well as +functions. The traditional CMUCL terminology is Alien Interface, and +while that older terminology is no longer used much in the system +documentation, it still reflected in names in the implementation, +notably in the name of the @code{SB-ALIEN} package. +@end quotation + +@menu +* Introduction to the Foreign Function Interface:: +* Foreign Types:: +* Operations On Foreign Values:: +* Foreign Variables:: +* Foreign Data Structure Examples:: +* Loading Unix Object Files:: +* Foreign Function Calls:: +* Step-By-Step Example of the Foreign Function Interface:: +@end menu + +@node Introduction to the Foreign Function Interface, Foreign Types, The Foreign Function Interface, The Foreign Function Interface +@comment node-name, next, previous, up +@section Introduction to the Foreign Function Interface +@c AKA "Introduction to Aliens" in the CMU CL manual + +Because of Lisp's emphasis on dynamic memory allocation and garbage +collection, Lisp implementations use non-C-like memory representations +for objects. This representation mismatch creates friction when a Lisp +program must share objects with programs which expect C data. There +are three common approaches to establishing communication: + +@itemize +@item +The burden can be placed on the foreign program (and programmer) by +requiring the knowledge and use of the representations used internally +by the Lisp implementation. This can require a considerable amount of +``glue'' code on the C side, and that code tends to be sensitively +dependent on the internal implementation details of the Lisp system. + +@item +The Lisp system can automatically convert objects back and forth +between the Lisp and foreign representations. This is convenient, but +translation becomes prohibitively slow when large or complex data +structures must be shared. This approach is supported by the SBCL +@acronym{FFI}, and used automatically by the when passing integers and +strings. + +@item +The Lisp program can directly manipulate foreign objects through the +use of extensions to the Lisp language. + +@end itemize + +SBCL, like CMUCL before it, relies primarily on the automatic +conversion and direct manipulation approaches. The @code{SB-ALIEN} +package provices a facility wherein foreign values of simple scalar +types are automatically converted and complex types are directly +manipulated in their foreign representation. Additionally the +lower-level System Area Pointers (or @acronym{SAP}s) can be used where +necessary to provide untyped access to foreign memory. + +Any foreign objects that can't automatically be converted into Lisp +values are represented by objects of type @code{alien-value}. Since +Lisp is a dynamically typed language, even foreign objects must have a +run-time type; this type information is provided by encapsulating the +raw pointer to the foreign data within an @code{alien-value} object. + +The type language and operations on foreign types are +intentionally similar to those of the C language. + +@node Foreign Types, Operations On Foreign Values, Introduction to the Foreign Function Interface, The Foreign Function Interface +@comment node-name, next, previous, up +@section Foreign Types +@c AKA "Alien Types" in the CMU CL manual + +Alien types have a description language based on nested list +structure. For example the C type + +@example +struct foo @{ + int a; + struct foo *b[100]; +@}; +@end example + +has the corresponding SBCL @acronym{FFI} type + +@lisp +(struct foo + (a int) + (b (array (* (struct foo)) 100))) +@end lisp + + +@menu +* Defining Foreign Types:: +* Foreign Types and Lisp Types:: +* Foreign Type Specifiers:: +@end menu + +@node Defining Foreign Types, Foreign Types and Lisp Types, Foreign Types, Foreign Types +@comment node-name, next, previous, up +@subsection Defining Foreign Types + +Types may be either named or anonymous. With structure and union +types, the name is part of the type specifier, allowing recursively +defined types such as: + +@lisp +(struct foo (a (* (struct foo)))) +@end lisp + +An anonymous structure or union type is specified by using the name +@code{nil}. The @code{with-alien} macro defines a local scope which +``captures'' any named type definitions. Other types are not +inherently named, but can be given named abbreviations using the +@code{define-alien-type} macro. + +@node Foreign Types and Lisp Types, Foreign Type Specifiers, Defining Foreign Types, Foreign Types +@comment node-name, next, previous, up +@subsection Foreign Types and Lisp Types + +The foreign types form a subsystem of the SBCL type system. An +@code{alien} type specifier provides a way to use any foreign type as a +Lisp type specifier. For example, + +@lisp +(typep @var{foo} '(alien (* int))) +@end lisp + +can be used to determine whether @var{foo} is a pointer to a foreign +@code{int}. @code{alien} type specifiers can be used in the same ways +as ordinary Lisp type specifiers (like @code{string}.) Alien type +declarations are subject to the same precise type checking as any +other declaration. @xref{Precise Type Checking}. + +Note that the type identifiers used in the foreign type system overlap +with native Lisp type specifiers in some cases. For example, the type +specifier @code{(alien single-float)} is identical to +@code{single-float}, since foreign floats are automatically converted +to Lisp floats. When @code{type-of} is called on an alien value that +is not automatically converted to a Lisp value, then it will return an +@code{alien} type specifier. + +@node Foreign Type Specifiers, , Foreign Types and Lisp Types, Foreign Types +@comment node-name, next, previous, up +@subsection Foreign Type Specifiers + +Note: All foreign type names are exported from the @code{sb-alien} +package. Some foreign type names are also symbols in +the @code{common-lisp} package, in which case they are +reexported from the @code{sb-alien} package, so that +e.g. it is legal to refer to @code{sb-alien:single-float}. + +These are the basic foreign type specifiers: + +@itemize +@item +The foreign type specifier @code{(* @var{foo})} describes a pointer to +an object of type @var{foo}. A pointed-to type @var{foo} of @code{t} +indicates a pointer to anything, similar to @code{void *} in +ANSI C. A null alien pointer can be detected with the +@code{sb-alien:null-alien} function. + +@item +The foreign type specifier @code{(array @var{foo} &optional +dimensions)} describes array of the specified @code{dimensions}, +holding elements of type @var{foo}. Note that (unlike in C) @code{(* +@var{foo})} and @code{(array @var{foo})} are considered to be +different types when type checking is done. If equivalence of pointer +and array types is desired, it may be explicitly coerced using +@code{sb-alien:cast}. + +Arrays are accessed using @code{sb-alien:deref}, passing the indices +as additional arguments. Elements are stored in column-major order +(as in C), so the first dimension determines only the size of the +memory block, and not the layout of the higher dimensions. An array +whose first dimension is variable may be specified by using @code{nil} +as the first dimension. Fixed-size arrays can be allocated as array +elements, structure slots or @code{sb-alien:with-alien} +variables. Dynamic arrays can only be allocated using +@code{sb-alien:make-alien}. + +@item +The foreign type specifier @code{(sb-alien:struct @var{name} &rest +@var{fields})} describes a structure type with the specified +@var{name} and @var{fields}. Fields are allocated at the same offsets +used by the implementation's C compiler. If @var{name} is @code{nil} +then the structure is anonymous. + +If a named foreign @code{struct} specifier is passed to +@code{define-alien-type} or @code{with-alien}, then this defines, +respectively, a new global or local foreign structure type. If no +@var{fields} are specified, then the fields are taken +from the current (local or global) alien structure type definition of +@var{name}. + +@item +The foreign type specifier @code{(sb-alien:union @var{name} &rest +@var{fields})} is similar to @code{sb-alien:struct}, but describes a +union type. All fields are allocated at the same offset, and the size +of the union is the size of the largest field. The programmer must +determine which field is active from context. + +@item +The foreign type specifier @code{(sb-alien:enum @var{name} &rest +@var{specs})} describes an enumeration type that maps between integer +values and keywords. If @var{name} is @code{nil}, then the type is +anonymous. Each element of the @var{specs} list is either a Lisp +keyword, or a list @code{(@var{keyword} @var{value})}. @var{value} is +an integer. If @var{value} is not supplied, then it defaults to one +greater than the value for the preceding spec (or to zero if it is the +first spec). + +@item +The foreign type specifier @code{(sb-alien:signed &optional +@var{bits})} specifies a signed integer with the specified number of +@var{bits} precision. The upper limit on integer +precision is determined by the machine's word size. If +@var{bits} is not specified, the maximum size will be +used. + +@item +The foreign type specifier @code{(integer &optional @var{bits})} +is equivalent to the corresponding type specifier using +@code{sb-alien:signed} instead of @code{integer}. + +@item +The foreign type specifier @code{(sb-alien:unsigned &optional +@var{bits})} is like corresponding type specifier using +@code{sb-alien:signed} except that the variable is treated as an +unsigned integer. + +@item +The foreign type specifier @code{(boolean &optional @var{bits})} is +similar to an enumeration type, but maps from Lisp @code{nil} and +@code{t} to C @code{0} and @code{1} respectively. @var{bits} +determines the amount of storage allocated to hold the truth value. + +@item +The foreign type specifier @code{single-float} describes a +floating-point number in IEEE single-precision format. + +@item +The foreign type specifier @code{double-float} describes a +floating-point number in IEEE double-precision format. + +@item +The foreign type specifier @code{(function @var{result-type} &rest +@var{arg-types})} describes a foreign function that takes arguments of +the specified @var{arg-types} and returns a result of type +@var{result-type}. Note that the only context where a foreign +@code{function} type is directly specified is in the argument to +@code{sb-alien:alien-funcall}. In all other contexts, foreign +functions are represented by foreign function pointer types: @code{(* +(function @dots{}))}. + +@item +The foreign type specifier @code{sb-alien:system-area-pointer} +describes a pointer which is represented in Lisp as a +@code{system-area-pointer} object. SBCL exports this type from +@code{sb-alien} because CMUCL did, but tentatively (as of the first +draft of this section of the manual, SBCL 0.7.6) it is deprecated, +since it doesn't seem to be required by user code. + +@item +The foreign type specifier @code{sb-alien:void} is used in function +types to declare that no useful value is returned. Using +@code{alien-funcall} to call a @code{void} foreign function will +return zero values. + +@item +The foreign type specifier @code{sb-alien:c-string} is similar to +@code{(* char)}, but is interpreted as a null-terminated string, and +is automatically converted into a Lisp string when accessed; or if the +pointer is C @code{NULL} or @code{0}, then accessing it gives Lisp +@code{nil}. Lisp strings are stored with a trailing NUL +termination, so no copying (either by the user or the implementation) +is necessary when passing them to foreign code. + +Assigning a Lisp string to a @code{c-string} structure field or +variable stores the contents of the string to the memory already +pointed to by that variable. When a foreign object of type @code{(* +char)} is assigned to a @code{c-string}, then the +@code{c-string} pointer is assigned to. This allows +@code{c-string} pointers to be initialized. For example: + +@lisp +(cl:in-package "CL-USER") ; which USEs package "SB-ALIEN" + +(define-alien-type nil (struct foo (str c-string))) + +(defun make-foo (str) + (let ((my-foo (make-alien (struct foo)))) + (setf (slot my-foo 'str) (make-alien char (length str)) + (slot my-foo 'str) str) + my-foo)) +@end lisp + +Storing Lisp @code{NIL} in a @code{c-string} writes C @code{NULL} to +the variable. + +@item +@code{sb-alien} also exports translations of these C type +specifiers as foreign type specifiers: @code{sb-alien:char}, +@code{sb-alien:short}, @code{sb-alien:int}, +@code{sb-alien:long}, @code{sb-alien:unsigned-char}, +@code{sb-alien:unsigned-short}, +@code{sb-alien:unsigned-int}, +@code{sb-alien:unsigned-long}, @code{sb-alien:float}, and +@code{sb-alien:double}. + +@end itemize + +@node Operations On Foreign Values, Foreign Variables, Foreign Types, The Foreign Function Interface +@comment node-name, next, previous, up +@section Operations On Foreign Values +@c AKA "Alien Operations" in the CMU CL manual + +This section describes how to read foreign values as Lisp values, how +to coerce foreign values to different kinds of foreign values, and how +to dynamically allocate and free foreign variables. + +@menu +* Accessing Foreign Values:: +* Coercing Foreign Values:: +* Foreign Dynamic Allocation:: +@end menu + +@node Accessing Foreign Values, Coercing Foreign Values, Operations On Foreign Values, Operations On Foreign Values +@comment node-name, next, previous, up +@subsection Accessing Foreign Values + +@defun sb-alien:deref @var{pointer-or-array} &rest @var{indices} + +The @code{sb-alien:deref} function returns the value pointed to by a +foreign pointer, or the value of a foreign array element. When +dereferencing a pointer, an optional single index can be specified to +give the equivalent of C pointer arithmetic; this index is scaled by +the size of the type pointed to. When dereferencing an array, the +number of indices must be the same as the number of dimensions in the +array type. @code{deref} can be set with @code{setf} to assign a new +value. + +@end defun + +@defun sb-alien:slot @var{struct-or-union} &rest @var{slot-names} + +The @code{sb-alien:slot} function extracts the value of the slot named +@var{slot-name} from a foreign @code{struct} or @code{union}. If +@var{struct-or-union} is a pointer to a structure or union, then it is +automatically dereferenced. @code{sb-alien:slot} can be set with +@code{setf} to assign a new value. Note that @var{slot-name} is +evaluated, and need not be a compile-time constant (but only constant +slot accesses are efficiently compiled). +@end defun + + +@subsubsection Untyped memory + +As noted at the beginning of the chapter, the System Area Pointer +facilities allow untyped access to foreign memory. @acronym{SAP}s can +be converted to and from the usual typed foreign values using +@code{sap-alien} and @code{alien-sap} (described elsewhere), and also +to and from integers - raw machine addresses. They should thus be +used with caution; corrupting the Lisp heap or other memory with +@acronym{SAP}s is trivial. + +@defun sb-sys:int-sap @var{machine-address} + +Creates a @acronym{SAP} pointing at the virtual address +@var{machine-address}. +@end defun + +@defun sb-sys:sap-ref-32 @var{sap} @var{offset} + +Access the value of the memory location at @var{offset} bytes from +@var{sap}. This form may also be used with @code{setf} to alter the +memory at that location. +@end defun + +@defun sb-sys:sap= @var{sap1} @var{sap2} + +Compare @var{sap1} and @var{sap2} for equality. +@end defun + +Similarly named functions exist for accessing other sizes of word, +other comparisons, and other conversions. The reader is invited to +use @code{apropos} and @code{describe} for more details + +@lisp +(apropos "sap" :sb-sys) +@end lisp + + +@node Coercing Foreign Values, Foreign Dynamic Allocation, Accessing Foreign Values, Operations On Foreign Values +@comment node-name, next, previous, up +@subsection Coercing Foreign Values + +@defun sb-alien:addr @var{alien-expr} + +The @code{sb-alien:addr} macro returns a pointer to the location +specified by @var{alien-expr}, which must be either a foreign +variable, a use of @code{sb-alien:deref}, a use of +@code{sb-alien:slot}, or a use of @code{sb-alien:extern-alien}. +@end defun + +@defun sb-alien:cast @var{foreign-value} @var{new-type} + +The @code{sb-alien:cast} macro converts @var{foreign-value} to a new +foreign value with the specified @var{new-type}. Both types, old and +new, must be foreign pointer, array or function types. Note that the +resulting Lisp foreign variable object is not @code{eq} to the +argument, but it does refer to the same foreign data bits. +@end defun + +@defun sb-alien:sap-alien @var{sap} @var{type} + +The @code{sb-alien:sap-alien} function converts @var{sap} (a system +area pointer) to a foreign value with the specified +@var{type}. @var{type} is not evaluated. + +The @var{type} must be some foreign pointer, array, or record type. +@end defun + +@defun sb-alien:alien-sap @var{foreign-value} @var{type} + +The @code{sb-alien:alien-sap} function returns the @acronym{SAP} which +points to @var{alien-value}'s data. + +The @var{foreign-value} must be of some foreign pointer, array, or +record type. +@end defun + + +@node Foreign Dynamic Allocation, , Coercing Foreign Values, Operations On Foreign Values +@comment node-name, next, previous, up +@subsection Foreign Dynamic Allocation + +Lisp code can call the C standard library functions @code{malloc} and +@code{free} to dynamically allocate and deallocate foreign +variables. The Lisp code shares the same allocator with foreign C +code, so it's OK for foreign code to call @code{free} on the result of +Lisp @code{sb-alien:make-alien}, or for Lisp code to call +@code{sb-alien:free-alien} on foreign objects allocated by C +code. + +@defmac sb-alien:make-alien @var{type} @var{size} + +The @code{sb-alien:make-alien} macro +returns a dynamically allocated foreign value of the specified +@var{type} (which is not evaluated.) The allocated memory is not +initialized, and may contain arbitrary junk. If supplied, +@var{size} is an expression to evaluate to compute the size of the +allocated object. There are two major cases: + +@itemize +@item +When @var{type} is a foreign array type, an array of that type is +allocated and a pointer to it is returned. Note that you must use +@code{deref} to change the result to an array before you can use +@code{deref} to read or write elements: + +@lisp +(cl:in-package "CL-USER") ; which USEs package "SB-ALIEN" +(defvar *foo* (make-alien (array char 10))) +(type-of *foo*) @result{} (alien (* (array (signed 8) 10))) +(setf (deref (deref foo) 0) 10) @result{} 10 +@end lisp + +If supplied, @var{size} is used as the first dimension for the + array. + +@item +When @var{type} is any other foreign type, then an object for that +type is allocated, and a pointer to it is returned. So +@code{(make-alien int)} returns a @code{(* int)}. If @var{size} is +specified, then a block of that many objects is allocated, with the +result pointing to the first one. + +@end itemize + +@end defmac + +@defun sb-alien:free-alien @var{foreign-value} + +The @code{sb-alien:free-alien} function +frees the storage for @var{foreign-value}, +which must have been allocated with Lisp @code{make-alien} +or C @code{malloc}. + +See also the @code{sb-alien:with-alien} macro, which allocates foreign +values on the stack. +@end defun + +@node Foreign Variables, Foreign Data Structure Examples, Operations On Foreign Values, The Foreign Function Interface +@comment node-name, next, previous, up +@section Foreign Variables +@c AKA "Alien Variables" in the CMU CL manual + +Both local (stack allocated) and external (C global) foreign variables +are supported. + +@menu +* Local Foreign Variables:: +* External Foreign Variables:: +@end menu + +@node Local Foreign Variables, External Foreign Variables, Foreign Variables, Foreign Variables +@comment node-name, next, previous, up +@subsection Local Foreign Variables + +@defmac sb-alien:with-alien @var{var-definitions} &body @var{body} + +The @code{with-alien} macro establishes local foreign variables with +the specified alien types and names. This form is analogous to +defining a local variable in C: additional storage is allocated, and +the initial value is copied. This form is less analogous to +@code{LET}-allocated Lisp variables, since the variables can't be +captured in closures: they live only for the dynamic extent of the +body, and referring to them outside is a gruesome error. + +The @var{var-definitions} argument is a list of +variable definitions, each of the form +@lisp +(@var{name} @var{type} &optional @var{initial-value}) +@end lisp + +The names of the variables are established as symbol-macros; the +bindings have lexical scope, and may be assigned with @code{setq} or +@code{setf}. + +The @code{with-alien} macro also establishes a new scope for named +structures and unions. Any @var{type} specified for a variable may +contain named structure or union types with the slots specified. +Within the lexical scope of the binding specifiers and body, a locally +defined foreign structure type @var{foo} can be referenced by its name +using @code{(struct @var{foo})}. +@end defmac + +@node External Foreign Variables, , Local Foreign Variables, Foreign Variables +@comment node-name, next, previous, up +@subsection External Foreign Variables + +External foreign names are strings, and Lisp names are symbols. When +an external foreign value is represented using a Lisp variable, there +must be a way to convert from one name syntax into the other. The +macros @code{extern-alien}, @code{define-alien-variable} and +@code{define-alien-routine} use this conversion heuristic: + +@itemize + +@item +Alien names are converted to Lisp names by uppercasing and replacing +underscores with hyphens. + +@item +Conversely, Lisp names are converted to alien names by lowercasing and +replacing hyphens with underscores. + +@item +Both the Lisp symbol and alien string names may be separately +specified by using a list of the form + +@lisp +(alien-string lisp-symbol) +@end lisp + +@end itemize + +@defmac sb-alien:define-alien-variable @var{name} @var{type} + +The @code{define-alien-variable} macro defines @var{name} as an +external foreign variable of the specified foreign @code{type}. +@var{name} and @code{type} are not evaluated. The Lisp name of the +variable (see above) becomes a global alien variable. Global alien +variables are effectively ``global symbol macros''; a reference to the +variable fetches the contents of the external variable. Similarly, +setting the variable stores new contents -- the new contents must be +of the declared @code{type}. Someday, they may well be implemented +using the @acronym{ANSI} @code{define-symbol-macro} mechanism, but as +of SBCL 0.7.5, they are still implemented using an older more-or-less +parallel mechanism inherited from CMUCL. + +For example, to access a C-level counter @var{foo}, one could write + +@lisp +(define-alien-variable "foo" int) +;; Now it is possible to get the value of the C variable foo simply by +;; referencing that Lisp variable: +(print foo) +(setf foo 14) +(incf foo) +@end lisp +@end defmac + +@defun sb-alien:get-errno + +Since in modern C libraries, the @code{errno} ``variable'' is typically +no longer a variable, but some bizarre artificial construct +which behaves superficially like a variable within a given thread, +it can no longer reliably be accessed through the ordinary +@code{define-alien-variable} mechanism. Instead, SBCL provides +the operator @code{sb-alien:get-errno} to allow Lisp code to read it. +@end defun + +@defmac sb-alien:extern-alien @var{name} @var{type} + +The @code{extern-alien} macro returns an alien with the specified +@var{type} which points to an externally defined value. @var{name} is +not evaluated, and may be either a string or a symbol. @var{type} is +an unevaluated alien type specifier. +@end defmac + +@node Foreign Data Structure Examples, Loading Unix Object Files, Foreign Variables, The Foreign Function Interface +@comment node-name, next, previous, up +@section Foreign Data Structure Examples +@c AKA "Alien Data Structure Example" in the CMU CL manual + +Now that we have alien types, operations and variables, we can +manipulate foreign data structures. This C declaration + +@example +struct foo @{ + int a; + struct foo *b[100]; +@}; +@end example + +can be translated into the following alien type: + +@lisp +(define-alien-type nil + (struct foo + (a int) + (b (array (* (struct foo)) 100)))) +@end lisp + +Once the @code{foo} alien type has been defined as above, the C +expression + +@example +struct foo f; +f.b[7].a; +@end example + +can be translated in this way: + +@lisp +(with-alien ((f (struct foo))) + (slot (deref (slot f 'b) 7) 'a) + ;; + ;; Do something with f... + ) +@end lisp + +Or consider this example of an external C variable and some accesses: + +@example +struct c_struct @{ + short x, y; + char a, b; + int z; + c_struct *n; +@}; +extern struct c_struct *my_struct; +my_struct->x++; +my_struct->a = 5; +my_struct = my_struct->n; +@end example + +which can be manipulated in Lisp like this: + +@lisp +(define-alien-type nil + (struct c-struct + (x short) + (y short) + (a char) + (b char) + (z int) + (n (* c-struct)))) +(define-alien-variable "my_struct" (* c-struct)) +(incf (slot my-struct 'x)) +(setf (slot my-struct 'a) 5) +(setq my-struct (slot my-struct 'n)) +@end lisp + +@node Loading Unix Object Files, Foreign Function Calls, Foreign Data Structure Examples, The Foreign Function Interface +@comment node-name, next, previous, up +@section Loading Unix Object Files + +Foreign object files can be loaded into the running Lisp process by +calling the functions @code{load-foreign} or @code{load-1-foreign}. + +The @code{sb-alien:load-1-foreign} function is the more primitive of +the two operations. It loads a single object file into the currently +running Lisp. The external symbols defining routines and variables are +made available for future external references (e.g. by +@code{extern-alien}). Forward references to foreign symbols aren't +supported: @code{load-1-foreign} must be run before any of the defined +symbols are referenced. + +@code{sb-alien:load-foreign} is built in terms of +@code{load-1-foreign} and some other machinery like +@code{sb-ext:run-program}. It accepts a list of files and libraries, +and runs the linker on the files and libraries, creating an absolute +Unix object file which is then processed by @code{load-1-foreign}. + +@quotation +Note: As of SBCL 0.7.5, all foreign code (code loaded with +@code{load-1-function} or @code{load-function}) is lost when a Lisp +core is saved with @code{sb-ext:save-lisp-and-die}, and no attempt is +made to restore it when the core is loaded. Historically this has been +an annoyance both for SBCL users and for CMUCL users. It's hard to +solve this problem completely cleanly, but some generally-reliable +partial solution might be useful. Once someone in either camp gets +sufficiently annoyed to create it, SBCL is likely to adopt some +mechanism for automatically restoring foreign code when a saved core +is loaded. +@end quotation + + +@node Foreign Function Calls, Step-By-Step Example of the Foreign Function Interface, Loading Unix Object Files, The Foreign Function Interface +@comment node-name, next, previous, up +@section Foreign Function Calls + +The foreign function call interface allows a Lisp program to call +many functions written in languages that use the C calling convention. + +Lisp sets up various signal handling routines and other environment +information when it first starts up, and expects these to be in place +at all times. The C functions called by Lisp should not change the +environment, especially the signal handlers: the signal handlers +installed by Lisp typically have interesting flags set (e.g to request +machine context information, or for signal delivery on an alternate +stack) which the Lisp runtime relies on for correct operation. +Precise details of how this works may change without notice between +versions; the source, or the brain of a friendly SBCL developer, is +the only documentation. Users of a Lisp built with the +@code{:sb-thread} feature should also read the section about threads, +@ref{Threading}. + +@menu +* The alien-funcall Primitive:: +* The define-alien-routine Macro:: +* define-alien-routine Example:: +* Calling Lisp From C:: +@end menu + +@node The alien-funcall Primitive, The define-alien-routine Macro, Foreign Function Calls, Foreign Function Calls +@comment node-name, next, previous, up +@subsection The @code{alien-funcall} Primitive + +@defun sb-alien:alien-funcall @var{alien-function} &rest @var{arguments} + +The @code{alien-funcall} function is the foreign function call +primitive: @var{alien-function} is called with the supplied +@var{arguments} and its C return value is returned as a Lisp value. +The @var{alien-function} is an arbitrary run-time expression; to refer +to a constant function, use @code{extern-alien} or a value defined by +@code{define-alien-routine}. + +The type of @code{alien-function} must be @code{(alien (function +...))} or @code{(alien (* (function ...)))}. The function type is +used to determine how to call the function (as though it was declared +with a prototype.) The type need not be known at compile time, but +only known-type calls are efficiently compiled. Limitations: + +@itemize + +@item +Structure type return values are not implemented. + +@item +Passing of structures by value is not implemented. + +@end itemize + +@end defun + +Here is an example which allocates a @code{(struct foo)}, calls a +foreign function to initialize it, then returns a Lisp vector of all +the @code{(* (struct foo))} objects filled in by the foreign call: + +@lisp +;; Allocate a foo on the stack. +(with-alien ((f (struct foo))) + ;; Call some C function to fill in foo fields. + (alien-funcall (extern-alien "mangle_foo" (function void (* foo))) + (addr f)) + ;; Find how many foos to use by getting the A field. + (let* ((num (slot f 'a)) + (result (make-array num))) + ;; Get a pointer to the array so that we don't have to keep extracting it: + (with-alien ((a (* (array (* (struct foo)) 100)) (addr (slot f 'b)))) + ;; Loop over the first N elements and stash them in the result vector. + (dotimes (i num) + (setf (svref result i) (deref (deref a) i))) + ;; Voila. + result))) +@end lisp + +@node The define-alien-routine Macro, define-alien-routine Example, The alien-funcall Primitive, Foreign Function Calls +@comment node-name, next, previous, up +@subsection The @code{define-alien-routine} Macro + +@defmac sb-alien:define-alien-routine @var{name} @var{result-type} &rest @var{arg-specifiers} + +The @code{define-alien-routine} macro is a convenience for +automatically generating Lisp interfaces to simple foreign functions. +The primary feature is the parameter style specification, which +translates the C pass-by-reference idiom into additional return +values. + +@var{name} is usually a string external symbol, but may also be a +symbol Lisp name or a list of the foreign name and the Lisp name. If +only one name is specified, the other is automatically derived as for +@code{extern-alien}. @var{result-type} is the alien type of the +return value. + +Each element of the @var{arg-specifiers} list +specifies an argument to the foreign function, and is +of the form +@lisp +(aname atype &optional style) +@end lisp + +@var{aname} is the symbol name of the argument to the constructed +function (for documentation). @var{atype} is the alien type of +corresponding foreign argument. The semantics of the actual call are +the same as for @code{alien-funcall}. @var{style} specifies how this +argument should be handled at call and return time, and should be one +of the following: + +@itemize + +@item +@code{:in} specifies that the argument is passed by value. This is the +default. @code{:in} arguments have no corresponding return value from +the Lisp function. + +@item +@code{:copy} is similar to @code{:in}, but the argument is copied to a +pre-allocated object and a pointer to this object is passed to the +foreign routine. + +@item +@code{:out} specifies a pass-by-reference output value. The type of +the argument must be a pointer to a fixed-sized object (such as an +integer or pointer). @code{:out} and @code{:in-out} style cannot be +used with pointers to arrays, records or functions. An object of the +correct size is allocated on the stack, and its address is passed to +the foreign function. When the function returns, the contents of this +location are returned as one of the values of the Lisp function (and +the location is automatically deallocated). + +@item +@code{:in-out} is a combination of @code{:copy} and @code{:out}. The +argument is copied to a pre-allocated object and a pointer to this +object is passed to the foreign routine. On return, the contents of +this location is returned as an additional value. + +@end itemize + +@quotation +Note: Any efficiency-critical foreign interface function should be inline +expanded, which can be done by preceding the +@code{define-alien-routine} call with: + +@lisp +(declaim (inline lisp-name)) +@end lisp + +In addition to avoiding the Lisp call overhead, this allows +pointers, word-integers and floats to be passed using non-descriptor +representations, avoiding consing.) +@end quotation + +@end defmac + +@node define-alien-routine Example, Calling Lisp From C, The define-alien-routine Macro, Foreign Function Calls +@comment node-name, next, previous, up +@subsection @code{define-alien-routine} Example + +Consider the C function @code{cfoo} with the following calling +convention: + +@example +void +cfoo (str, a, i) + char *str; + char *a; /* update */ + int *i; /* out */ +@{ + /* body of cfoo(...) */ +@} +@end example + +This can be described by the following call to +@code{define-alien-routine}: + +@lisp +(define-alien-routine "cfoo" void + (str c-string) + (a char :in-out) + (i int :out)) +@end lisp + +The Lisp function @code{cfoo} will have two arguments (@var{str} and +@var{a}) and two return values (@var{a} and @var{i}). + +@node Calling Lisp From C, , define-alien-routine Example, Foreign Function Calls +@comment node-name, next, previous, up +@subsection Calling Lisp From C + +Calling Lisp functions from C is sometimes possible, but is extremely +hackish and poorly supported as of SBCL 0.7.5. See @code{funcall0} +@dots{} @code{funcall3} in the runtime system. The arguments must be +valid SBCL object descriptors (so that e.g. fixnums must be +left-shifted by 2.) As of SBCL 0.7.5, the format of object descriptors +is documented only by the source code and, in parts, by the old CMUCL +@file{INTERNALS} documentation. + +Note that the garbage collector moves objects, and won't be +able to fix up any references in C variables. There are three +mechanisms for coping with this: + +@enumerate +@item +The @code{sb-ext:purify} moves all live Lisp +data into static or read-only areas such that it will never be moved +(or freed) again in the life of the Lisp session + +@item +@code{sb-sys:with-pinned-objects} is a macro which arranges for some +set of objects to be pinned in memory for the dynamic extent of its +body forms. On ports which use the generational garbage collector (as +of SBCL 0.8.3, only the x86) this has a page granularity - i.e. the +entire 4k page or pages containing the objects will be locked down. On +other ports it is implemented by turning off GC for the duration (so +could be said to have a whole-world granularity). + +@item +Disable GC, using the @code{without-gcing} macro or @code{gc-off} +call. +@end enumerate + +@c + + +@node Step-By-Step Example of the Foreign Function Interface, , Foreign Function Calls, The Foreign Function Interface +@comment node-name, next, previous, up +@section Step-By-Step Example of the Foreign Function Interface + +This section presents a complete example of an interface to a somewhat +complicated C function. + +Suppose you have the following C function which you want to be able to +call from Lisp in the file @file{test.c} + +@example +struct c_struct +@{ + int x; + char *s; +@}; + +struct c_struct *c_function (i, s, r, a) + int i; + char *s; + struct c_struct *r; + int a[10]; +@{ + int j; + struct c_struct *r2; + + printf("i = %d\n", i); + printf("s = %s\n", s); + printf("r->x = %d\n", r->x); + printf("r->s = %s\n", r->s); + for (j = 0; j < 10; j++) printf("a[%d] = %d.\n", j, a[j]); + r2 = (struct c_struct *) malloc (sizeof(struct c_struct)); + r2->x = i + 5; + r2->s = "a C string"; + return(r2); +@}; +@end example + +It is possible to call this C function from Lisp using the file +@file{test.lisp} containing + +@lisp +(cl:defpackage "TEST-C-CALL" (:use "CL" "SB-ALIEN" "SB-C-CALL")) +(cl:in-package "TEST-C-CALL") + +;;; Define the record C-STRUCT in Lisp. +(define-alien-type nil + (struct c-struct + (x int) + (s c-string))) + +;;; Define the Lisp function interface to the C routine. It returns a +;;; pointer to a record of type C-STRUCT. It accepts four parameters: +;;; I, an int; S, a pointer to a string; R, a pointer to a C-STRUCT +;;; record; and A, a pointer to the array of 10 ints. +;;; +;;; The INLINE declaration eliminates some efficiency notes about heap +;;; allocation of alien values. +(declaim (inline c-function)) +(define-alien-routine c-function + (* (struct c-struct)) + (i int) + (s c-string) + (r (* (struct c-struct))) + (a (array int 10))) + +;;; a function which sets up the parameters to the C function and +;;; actually calls it +(defun call-cfun () + (with-alien ((ar (array int 10)) + (c-struct (struct c-struct))) + (dotimes (i 10) ; Fill array. + (setf (deref ar i) i)) + (setf (slot c-struct 'x) 20) + (setf (slot c-struct 's) "a Lisp string") + + (with-alien ((res (* (struct c-struct)) + (c-function 5 "another Lisp string" (addr c-struct) ar))) + (format t "~&back from C function~%") + (multiple-value-prog1 + (values (slot res 'x) + (slot res 's)) + + ;; Deallocate result. (after we are done referring to it: + ;; "Pillage, *then* burn.") + (free-alien res))))) +@end lisp + +To execute the above example, it is necessary to compile the C +routine, e.g.: @samp{cc -c test.c} (In order to enable incremental +loading with some linkers, you may need to say @samp{cc -G 0 -c +test.c}) + +Once the C code has been compiled, you can start up Lisp and load it +in: @samp{sbcl} Lisp should start up with its normal prompt. + +Within Lisp, compile the Lisp file. (This step can be done +separately. You don't have to recompile every time.) +@samp{(compile-file "test.lisp")} + +Within Lisp, load the foreign object file to define the necessary +symbols: @samp{(load-foreign "test.o")}. This must be done before +loading any code that refers to these symbols. + +Now you can load the compiled Lisp (``fasl'') file into Lisp: +@samp{(load "test.fasl")} +And once the Lisp file is loaded, you can call the +Lisp routine that sets up the parameters and calls the C +function: +@samp{(test-c-call::call-cfun)} + +The C routine should print the following information to standard output: + +@example +i = 5 +s = another Lisp string +r->x = 20 +r->s = a Lisp string +a[0] = 0. +a[1] = 1. +a[2] = 2. +a[3] = 3. +a[4] = 4. +a[5] = 5. +a[6] = 6. +a[7] = 7. +a[8] = 8. +a[9] = 9. +@end example + +After return from the C function, +the Lisp wrapper function should print the following output: + +@example +back from C function +@end example + +And upon return from the Lisp wrapper function, +before the next prompt is printed, the +Lisp read-eval-print loop should print the following return values: + +@example +10 +"a C string" +@end example diff --git a/doc/manual/intro.texinfo b/doc/manual/intro.texinfo new file mode 100644 index 0000000..10297f6 --- /dev/null +++ b/doc/manual/intro.texinfo @@ -0,0 +1,234 @@ +@node Introduction, The Compiler, Top, Top +@comment node-name, next, previous, up +@chapter Introduction + +SBCL is a mostly-conforming implementation of the ANSI Common Lisp +standard. This manual focuses on behavior which is specific to SBCL, +not on behavior which is common to all implementations of ANSI Common +Lisp. + +@menu +* More Common Lisp Information:: +* More SBCL Information:: +* Overview:: +@end menu + +@node More Common Lisp Information, More SBCL Information, Introduction, Introduction +@comment node-name, next, previous, up +@section Where To Go For More Information about Common Lisp in General + +Regardless of your ability level, two very useful resources for +working with any implementation of Common Lisp are the ILISP package +for Emacs at @uref{http://ilisp.cons.org} and the Common Lisp HyperSpec +at @uref{http://www.lispworks.com/reference/HyperSpec/index.html}. + +If you're not a programmer and you're trying to learn, many +introductory Lisp books are available. However, we don't have any +standout favorites. If you can't decide, try checking the Usenet +comp.lang.lisp FAQ for recent recommendations. + +If you are an experienced programmer in other languages but need to +learn about Lisp, three books stand out. + +@itemize + +@item +@emph{ANSI Common Lisp}, by Paul Graham, will teach you +about most of the language. (And later it might also be worth checking +out @emph{On Lisp}, by the same author.) + +@item +@emph{Paradigms Of Artificial Intelligence +Programming}, by Peter Norvig, also has some good information on +general Common Lisp programming, and many nontrivial examples. +Whether or not your work is AI, it's a very good book to look at. + +@item +Neither of the books above emphasizes CLOS, but @emph{Object-Oriented +Programming In Common Lisp} by Sonya Keene does. Even if you're very +knowledgeable about object oriented programming in the abstract, it's +worth looking at this book if you want to do any OO in Common +Lisp. Some abstractions in CLOS (especially multiple dispatch) go +beyond anything you'll see in most OO systems, and there are a number +of lesser differences as well. This book tends to help with the +culture shock. + +@end itemize + + +@node More SBCL Information, Overview, More Common Lisp Information, Introduction +@comment node-name, next, previous, up +@section Where To Go For More Information About SBCL + +Before you read this user manual, you should probably read two other +things. + +@itemize + +@item +You should know how to program in Common Lisp. If you don't already +know how, you should probably read a book on it. @xref{More Common +Lisp Information}. + +@item +The Unix ``man page'' for SBCL will tell you +how to start the SBCL environment, so you can get to the classic +``hello, world'' level of knowledge. It's the file called +@file{sbcl.1} in the SBCL distribution. If SBCL is installed on your +system, you can read a formatted copy by executing the command +@samp{man sbcl}. + +@end itemize + + +Besides this user manual and the Unix man page, some other +SBCL-specific information is available: + +@itemize + +@item +The SBCL home page at @uref{http://sbcl.sourceforge.net/} has some +general information, plus links to mailing lists devoted to SBCL, and +to archives of these mailing lists. + +@item +Documentation for non-ANSI extensions for various commands is +available online from the SBCL executable itself. The extensions for +functions which have their own command prompts (e.g. the debugger, and +@code{inspect}) are documented in text available by typing +@command{help} at their command prompts. The extensions for functions +which don't have their own command prompt (like @code{trace} does) are +described in their documentation strings, unless your SBCL was +compiled with an option not to include documentation strings, in which +case the doc strings are only readable in the source code. + +@item +Some low-level information describing the programming details of the +conversion from CMUCL to SBCL is available in the +@file{doc/FOR-CMUCL-DEVELOPERS} file in the SBCL +distribution. + +@end itemize + + +@node Overview, , More SBCL Information, Introduction +@comment node-name, next, previous, up +@section Overview Of SBCL, How It Works And Where It Came From + +You can work productively with SBCL without knowing anything +understanding anything about where it came from, how it is +implemented, or how it extends the ANSI Common Lisp standard. However, +a little knowledge can be helpful in order to understand error +messages, to troubleshoot problems, to understand why some parts of +the system are better debugged than others, and to anticipate which +known bugs, known performance problems, and missing extensions are +likely to be fixed, tuned, or added. + +SBCL is descended from CMUCL, which is itself descended from Spice +Lisp, including early implementations for the Mach operating system on +the IBM RT, back in the 1980s. Design decisions from that time are +still reflected in the current implementation: + +@itemize + +@item +The system expects to be loaded into a fixed-at-compile-time location +in virtual memory, and also expects the location of all of its heap +storage to be specified at compile time. + +@item +The system overcommits memory, allocating large amounts of address +space from the system (often more than the amount of virtual memory +available) and then failing if ends up using too much of the allocated +storage. + +@item +A word is a 32-bit quantity. The system has been ported to many +processor architectures without altering this basic principle. Some +hacks allow the system to run on the Alpha chip (a 64-bit +architecture) but even there 32-bit words are used. The assumption +that a word is 32 bits wide is implicit in hundreds of places in the +system. + +@item +The system is implemented as a C program which is responsible for +supplying low-level services and loading a Lisp @file{.core} +file. + +@end itemize + +SBCL also inherited some newer architectural features from CMUCL. The +most important is that on some architectures it has a generational +garbage collector (``GC''), which has various implications (mostly +good) for performance. These are discussed another chapter, +@ref{Efficiency}. + +SBCL has diverged from CMUCL in that SBCL is now essentially a +``compiler-only implementation'' of Common Lisp. A Common Lisp +implementation is permitted to implement both a compiler and an +interpreter, and there's some special support in the standard +(e.g. the distinction between @code{functionp} and +@code{compiled-function-p}) to help support that. But SBCL has only a +vestigial, rudimentary true interpreter. In SBCL, the @code{eval} +function only truly ``interprets'' a few special classes of forms, +such as symbols which are @code{boundp}. More complicated forms are +evaluated by calling @code{compile} and then calling @code{funcall} on +the returned result. + + +The direct ancestor of SBCL is the X86 port of CMUCL. This port was in +some ways the most cobbled-together of all the CMUCL ports, since a +number of strange changes had to be made to support the register-poor +X86 architecture. Some things (like tracing and debugging) do not work +particularly well there. SBCL should be able to improve in these areas +(and has already improved in some other areas), but it takes a while. + +On the x86, SBCL like the X86 port of CMUCL, uses a +@emph{conservative} GC. This means that it doesn't maintain a strict +separation between tagged and untagged data, instead treating some +untagged data (e.g. raw floating point numbers) as possibly-tagged +data and so not collecting any Lisp objects that they point to. This +has some negative consequences for average time efficiency (though +possibly no worse than the negative consequences of trying to +implement an exact GC on a processor architecture as register-poor as +the X86) and also has potentially unlimited consequences for +worst-case memory efficiency. In practice, conservative garbage +collectors work reasonably well, not getting anywhere near the worst +case. But they can occasionally cause odd patterns of memory usage. + +The fork from CMUCL was based on a major rewrite of the system +bootstrap process. CMUCL has for many years tolerated a very unusual +``build'' procedure which doesn't actually build the complete system +from scratch, but instead progressively overwrites parts of a running +system with new versions. This quasi-build procedure can cause various +bizarre bootstrapping hangups, especially when a major change is made +to the system. It also makes the connection between the current source +code and the current executable more tenuous than in other software +systems -- it's easy to accidentally ``build'' a CMUCL system +containing characteristics not reflected in the current version of the +source code. + +Other major changes since the fork from CMUCL include + +@itemize + +@item +SBCL has dropped support for many CMUCL extensions, (e.g. IP +networking, remote procedure call, Unix system interface, and X11 +interface). Most of these are now available as contributed or +third-party modules. + +@item +SBCL has deleted or deprecated some nonstandard features and code +complexity which helped efficiency at the price of +maintainability. For example, the SBCL compiler no longer implements +memory pooling internally (and so is simpler and more maintainable, +but generates more garbage and runs more slowly), and various +block-compilation efficiency-increasing extensions to the language +have been deleted or are no longer used in the implementation of SBCL +itself. + +@end itemize + + + diff --git a/doc/manual/sbcl.texinfo b/doc/manual/sbcl.texinfo new file mode 100644 index 0000000..eda689d --- /dev/null +++ b/doc/manual/sbcl.texinfo @@ -0,0 +1,197 @@ +\input texinfo @c -*-texinfo-*- +@c %**start of header +@setfilename sbcl.info +@settitle SBCL User Manual +@c %**end of header + + +@set EDITION 0.1 +@set VERSION 0.8.9 +@set UPDATED 24 March 2004 +@set UPDATE-MONTH March 2004 + + +@copying + +@quotation +This manual is part of the SBCL software system. See the @file{README} file +for more information. + +This manual is largely derived from the manual for the CMUCL system, +which was produced at Carnegie Mellon University and later released +into the public domain. This manual is in the public domain and is +provided with absolutely no warranty. See the @file{COPYING} and +@file{CREDITS} files for more information. +@end quotation + +@end copying + +@titlepage + +@title SBCL User Manual +@subtitle SBCL Version @value{VERSION} +@subtitle @value{UPDATE-MONTH} +@c @author The CMUCL and SBCL teams + + +@c The following two commands start the copyright page. +@page +@vskip 0pt plus 1filll +@insertcopying + +@end titlepage + +@contents + +@ifnottex + +@node Top, Introduction, (dir), (dir) +@comment node-name, next, previous, up +@top SBCL + +@insertcopying + +@menu +* Introduction:: +* The Compiler:: +* The Debugger:: +* Efficiency:: +* Beyond The ANSI Standard:: +* The Foreign Function Interface:: +* Function Index:: +* Concept Index:: +* Colophon:: + +@detailmenu + --- The Detailed Node Listing --- + +Introduction + +* More Common Lisp Information:: +* More SBCL Information:: +* Overview:: + +The Compiler + +* Error Messages:: +* Handling of Types:: +* Compiler Policy:: +* Open Coding and Inline Expansion:: + +Error Messages + +* The Parts of the Error Message:: +* The Original and Actual Source:: +* Error Severity:: +* Errors During Macroexpansion:: +* Read Errors:: + +The Compiler's Handling of Types + +* Implementation Limitations:: +* Type Errors at Compile Time:: +* Precise Type Checking:: +* Weakened Type Checking:: +* Getting Existing Programs to Run:: + +The Debugger + +* Starting the Debugger:: +* The Debugger Command Loop:: +* Controlling Printing in the Debugger:: +* Stack Frames:: +* Variable Access:: +* Source Location Printing:: +* Debugger Policy Control:: +* Exiting Commands:: +* Information Commands:: +* Function Tracing:: + +Stack Frames + +* Stack Motion:: +* How Arguments are Printed:: +* Function Names:: +* Funny Frames:: +* Debug Tail Recursion:: +* Unknown Locations and Interrupts:: + +Variable Access + +* Variable Value Availability:: +* Note On Lexical Variable Access:: + +Source Location Printing + +* How the Source is Found:: +* Source Location Availability:: + +Efficiency + +* Modular arithmetic:: + +Beyond The ANSI Standard + +* Non-Conformance With The ANSI Standard:: +* Idiosyncrasies:: +* Extensions:: + +Extensions + +* Things Which Might Be In The Next ANSI Standard:: +* Threading:: +* Support For Unix:: +* Customization Hooks for Users:: +* Tools To Help Developers:: +* Interface To Low-Level SBCL Implementation:: +* Efficiency Hacks:: + +The Foreign Function Interface + +* Introduction to the Foreign Function Interface:: +* Foreign Types:: +* Operations On Foreign Values:: +* Foreign Variables:: +* Foreign Data Structure Examples:: +* Loading Unix Object Files:: +* Foreign Function Calls:: +* Step-By-Step Example of the Foreign Function Interface:: + +Foreign Types + +* Defining Foreign Types:: +* Foreign Types and Lisp Types:: +* Foreign Type Specifiers:: + +Operations On Foreign Values + +* Accessing Foreign Values:: +* Coercing Foreign Values:: +* Foreign Dynamic Allocation:: + +Foreign Variables + +* Local Foreign Variables:: +* External Foreign Variables:: + +Foreign Function Calls + +* The alien-funcall Primitive:: +* The define-alien-routine Macro:: +* define-alien-routine Example:: +* Calling Lisp From C:: + +@end detailmenu +@end menu + +@end ifnottex + +@include intro.texinfo +@include compiler.texinfo +@include debugger.texinfo +@include efficiency.texinfo +@include beyond-ansi.texinfo +@include ffi.texinfo +@include backmatter.texinfo + +@bye diff --git a/version.lisp-expr b/version.lisp-expr index 4ee3c85..bd1c7b3 100644 --- a/version.lisp-expr +++ b/version.lisp-expr @@ -17,4 +17,4 @@ ;;; checkins which aren't released. (And occasionally for internal ;;; versions, especially for internal versions off the main CVS ;;; branch, it gets hairier, e.g. "0.pre7.14.flaky4.13".) -"0.8.9.12" +"0.8.9.13"