X-Git-Url: http://repo.macrolet.net/gitweb/?a=blobdiff_plain;f=doc%2Fmanual%2Fbeyond-ansi.texinfo;h=98253433a00ac266561b886c6a408247ab7fcba7;hb=920b5eb02b1e1fd1c6c28395cade04e81fbee2bb;hp=3a60c5757ef31584e8ba0298c484c4ce5e11a284;hpb=1a68f34a511841986710cc0012417a8633ab7241;p=sbcl.git diff --git a/doc/manual/beyond-ansi.texinfo b/doc/manual/beyond-ansi.texinfo index 3a60c57..9825343 100644 --- a/doc/manual/beyond-ansi.texinfo +++ b/doc/manual/beyond-ansi.texinfo @@ -1,312 +1,644 @@ -@node Beyond The ANSI Standard +@node Beyond the ANSI Standard @comment node-name, next, previous, up -@chapter Beyond The ANSI Standard - -SBCL is mostly an implementation of the ANSI standard for -Common Lisp. However, there's some important behavior which extends -or clarifies the standard, and various behavior which outright -violates the standard. +@chapter Beyond the ANSI Standard +SBCL is derived from CMUCL, which implements many extensions to the +ANSI standard. SBCL doesn't support as many extensions as CMUCL, but +it still has quite a few. @xref{Contributed Modules}. @menu -* Non-Conformance With The ANSI Standard:: -* Idiosyncrasies:: -* Extensions:: +* Reader Extensions:: +* Package-Local Nicknames:: +* Package Variance:: +* Garbage Collection:: +* Metaobject Protocol:: +* Extensible Sequences:: +* Support For Unix:: +* Customization Hooks for Users:: +* Tools To Help Developers:: +* Resolution of Name Conflicts:: +* Hash Table Extensions:: +* Random Number Generation:: +* Miscellaneous Extensions:: +* Stale Extensions:: +* Efficiency Hacks:: @end menu -@node Non-Conformance With The ANSI Standard +@node Reader Extensions @comment node-name, next, previous, up -@section Non-Conformance With The ANSI Standard +@section Reader Extensions +@cindex Reader Extensions -Essentially every type of non-conformance is considered a bug. (The -exceptions involve internal inconsistencies in the standard.) In SBCL -0.7.6, the master record of known bugs is in the @file{BUGS} file in -the distribution. Some highlight information about bugs may also be -found in the manual page. The recommended way to report bugs is -through the sbcl-help or sbcl-devel mailing lists. For mailing list -addresses, @xref{More SBCL Information}. +SBCL supports extended package prefix syntax, which allows specifying +an alternate package instead of @code{*package*} for the reader to use +as the default package for interning symbols: +@lisp +:: +@end lisp + +Example: -@node Idiosyncrasies -@comment node-name, next, previous, up -@section Idiosyncrasies - -The information in this section describes some of the ways that SBCL -deals with choices that the ANSI standard leaves to the -implementation. - -Declarations are generally treated as assertions. This general -principle, and its implications, and the bugs which still keep the -compiler from quite satisfying this principle, are discussed in -@ref{The Compiler}. - -SBCL is essentially a compiler-only implementation of Common -Lisp. That is, for all but a few special cases, @code{eval} creates a -lambda expression, calls @code{compile} on the lambda expression to -create a compiled function, and then calls @code{funcall} on the -resulting function object. This is explicitly allowed by the ANSI -standard, but leads to some oddities, e.g. collapsing @code{functionp} -and @code{compiled-function-p} into the same predicate. - -@findex defconstant -SBCL is quite strict about ANSI's definition of -@code{defconstant}. ANSI says that doing @code{defconstant} of the -same symbol more than once is undefined unless the new value is -@code{eql} to the old value. Conforming to this specification is a -nuisance when the ``constant'' value is only constant under some -weaker test like @code{string=} or @code{equal}. It's especially -annoying because, in SBCL, @code{defconstant} takes effect not only at -load time but also at compile time, so that just compiling and loading -reasonable code like @lisp -(defconstant +foobyte+ '(1 4)) + 'foo::(bar quux zot) == '(foo::bar foo::quux foo::zot) @end lisp -runs into this undefined behavior. Many implementations of Common Lisp -try to help the programmer around this annoyance by silently accepting -the undefined code and trying to do what the programmer probably -meant. SBCL instead treats the undefined behavior as an error. Often -such code can be rewritten in portable ANSI Common Lisp which has the -desired behavior. E.g., the code above can be given an exactly -defined meaning by replacing @code{defconstant} either with -@code{defparameter} or with a customized macro which does the right -thing, possibly along the lines of the @code{defconstant-eqx} macro -used internally in the implementation of SBCL itself. In -circumstances where this is not appropriate, the programmer can handle -the condition type @code{sb-ext:defconstant-uneql}, and choose either -the @command{continue} or @command{abort} restart as appropriate. - -SBCL gives style warnings about various kinds of perfectly legal code, -e.g. + +Doesn't alter @code{*package*}: if @code{foo::bar} would cause a +read-time package lock violation, so does @code{foo::(bar)}. + +@node Package-Local Nicknames +@comment node-name, next, previous, up +@section Package-Local Nicknames +@cindex Package-Local Nicknames + +SBCL allows giving packages local nicknames: they allow short and +easy-to-use names to be used without fear of name conflict associated +with normal nicknames. + +A local nickname is valid only when inside the package for which it +has been specified. Different packages can use same local nickname for +different global names, or different local nickname for same global +name. + +Symbol @code{:package-local-nicknames} in @code{*features*} denotes the +support for this feature. + +@findex @cl{defpackage} +@defmac @cl{defpackage} name [[option]]* @result{} package + +Options are extended to include @itemize - -@item -@code{defmethod} without a preceding @code{defgeneric}; - -@item -multiple @code{defun}s of the same symbol in different units; - @item -special variables not named in the conventional @code{*foo*} style, -and lexical variables unconventionally named in the @code{*foo*} style +@code{:local-nicknames (@var{local-nickname} @var{actual-package-name})*} +The package has the specified local nicknames for the corresponding +actual packages. @end itemize -This causes friction with people who point out that other ways of -organizing code (especially avoiding the use of @code{defgeneric}) are -just as aesthetically stylish. However, these warnings should be read -not as ``warning, bad aesthetics detected, you have no style'' but -``warning, this style keeps the compiler from understanding the code -as well as you might like.'' That is, unless the compiler warns about -such conditions, there's no way for the compiler to warn about some -programming errors which would otherwise be easy to overlook. (Related -bug: The warning about multiple @code{defun}s is pointlessly annoying -when you compile and then load a function containing @code{defun} -wrapped in @code{eval-when}, and ideally should be suppressed in that -case, but still isn't as of SBCL 0.7.6.) - - -@node Extensions +Example: + +@lisp +(defpackage :bar (:intern "X")) +(defpackage :foo (:intern "X")) +(defpackage :quux (:use :cl) (:local-nicknames (:bar :foo) (:foo :bar))) +(find-symbol "X" :foo) ; => FOO::X +(find-symbol "X" :bar) ; => BAR::X +(let ((*package* (find-package :quux))) + (find-symbol "X" :foo)) ; => BAR::X +(let ((*package* (find-package :quux))) + (find-symbol "X" :bar)) ; => FOO::X +@end lisp +@end defmac + +@include fun-sb-ext-package-local-nicknames.texinfo +@include fun-sb-ext-package-locally-nicknamed-by-list.texinfo +@include fun-sb-ext-add-package-local-nickname.texinfo +@include fun-sb-ext-remove-package-local-nickname.texinfo + +@node Package Variance @comment node-name, next, previous, up -@section Extensions +@section Package Variance -SBCL is derived from CMUCL, which implements many extensions to the -ANSI standard. SBCL doesn't support as many extensions as CMUCL, but -it still has quite a few. @xref{Contributed Modules}. +Common Lisp standard specifies that ``If the new definition is at +variance with the current state of that package, the consequences are +undefined;'' SBCL by default signals a full warning and retains as +much of the package state as possible. +This can be adjusted using @code{sb-ext:*on-package-variance*}: -@menu -* Things Which Might Be In The Next ANSI Standard:: -* Threading:: -* Support For Unix:: -* Customization Hooks for Users:: -* Tools To Help Developers:: -* Interface To Low-Level SBCL Implementation:: -* Stale Extensions:: -* Efficiency Hacks:: -@end menu +@include var-sb-ext-star-on-package-variance-star.texinfo -@node Things Which Might Be In The Next ANSI Standard +@node Garbage Collection @comment node-name, next, previous, up -@subsection Things Which Might Be In The Next ANSI Standard - -SBCL provides extensive support for calling external C code, @ref{The -Foreign Function Interface}. +@section Garbage Collection +@cindex Garbage collection SBCL provides additional garbage collection functionality not -specified by ANSI. Weak pointers allow references to objects to be -maintained without keeping them from being GCed (garbage -collected). And ``finalization'' hooks are available to cause code to -be executed when an object has been GCed. -@c - -SBCL supports @dfn{Gray streams}, user-overloadable CLOS classes whose -instances can be used as Lisp streams (e.g. passed as the first -argument to @code{format}). Additionally, the bundled contrib module -@dfn{sb-simple-streams} implements a subset of the Franz Allegro -simple-streams proposal. - -SBCL supports a MetaObject Protocol which is intended to be compatible +specified by ANSI. + +@include var-sb-ext-star-after-gc-hooks-star.texinfo +@include fun-sb-ext-gc.texinfo + +@subsection Finalization +@cindex Finalization + +Finalization allows code to be executed after an object has been +garbage collected. This is useful for example for releasing foreign +memory associated with a Lisp object. + +@include fun-sb-ext-finalize.texinfo +@include fun-sb-ext-cancel-finalization.texinfo + +@subsection Weak Pointers +@cindex Weak pointers + +Weak pointers allow references to objects to be maintained without +keeping them from being garbage collected: useful for building caches +among other things. + +Hash tables can also have weak keys and values: @pxref{Hash Table +Extensions}. + +@include fun-sb-ext-make-weak-pointer.texinfo +@include fun-sb-ext-weak-pointer-value.texinfo + +@subsection Introspection and Tuning + +@include var-sb-ext-star-gc-run-time-star.texinfo +@include fun-sb-ext-bytes-consed-between-gcs.texinfo +@include fun-sb-ext-dynamic-space-size.texinfo +@include fun-sb-ext-get-bytes-consed.texinfo +@include fun-sb-ext-gc-logfile.texinfo +@include fun-sb-ext-generation-average-age.texinfo +@include fun-sb-ext-generation-bytes-allocated.texinfo +@include fun-sb-ext-generation-bytes-consed-between-gcs.texinfo +@include fun-sb-ext-generation-minimum-age-before-gc.texinfo +@include fun-sb-ext-generation-number-of-gcs-before-promotion.texinfo +@include fun-sb-ext-generation-number-of-gcs.texinfo + +@node Metaobject Protocol +@comment node-name, next, previous, up +@section Metaobject Protocol + +@subsection AMOP Compatibility of Metaobject Protocol + +SBCL supports a metaobject protocol which is intended to be compatible with AMOP; present exceptions to this (as distinct from current bugs) are: @itemize - + +@item +@findex @sbmop{compute-effective-method} +@code{compute-effective-method} only returns one value, not two. + +There is no record of what the second return value was meant to +indicate, and apparently no clients for it. + +@item +@tindex @cl{generic-function} +@tindex @cl{standard-generic-function} +@tindex @sbmop{funcallable-standard-object} +@tindex @cl{standard-object} +@tindex @cl{function} +The direct superclasses of @code{sb-mop:funcallable-standard-object} are +@code{(function standard-object)}, not @code{(standard-object function)}. + +This is to ensure that the @code{standard-object} class is the last of +the standardized classes before @code{t} appearing in the class +precedence list of @code{generic-function} and +@code{standard-generic-function}, as required by section 1.4.4.5 of the +ANSI specification. + +@item +@findex @cl{ensure-generic-function} +@findex @sbmop{generic-function-declarations} +the arguments @code{:declare} and @code{:declarations} to +@code{ensure-generic-function} are both accepted, with the leftmost +argument defining the declarations to be stored and returned by +@code{generic-function-declarations}. + +Where AMOP specifies @code{:declarations} as the keyword argument to +@code{ensure-generic-function}, the Common Lisp standard specifies +@code{:declare}. Portable code should use @code{:declare}. + @item -the abstract @code{metaobject} class is not present in the class -hierarchy; - +@findex @sbmop{validate-superclass} +@findex @sbmop{finalize-inheritance} +@tindex @cl{standard-class} +@tindex @sbmop{funcallable-standard-class} +@tindex @cl{function} +@findex @sbmop{class-prototype} +although SBCL obeys the requirement in AMOP that +@code{validate-superclass} should treat @code{standard-class} and +@code{funcallable-standard-class} as compatible metaclasses, we +impose an additional requirement at class finalization time: a class +of metaclass @code{funcallable-standard-class} must have +@code{function} in its superclasses, and a class of metaclass +@code{standard-class} must not. + +@findex @cl{typep} +@findex @cl{class-of} +@findex @cl{subtypep} +After a class has been finalized, it is associated with a class +prototype which is accessible by a standard mop function +@code{sb-mop:class-prototype}. The user can then ask whether this +object is a @code{function} or not in several different ways: whether it +is a function according to @code{typep}; whether its @code{class-of} is +@code{subtypep} @code{function}, or whether @code{function} appears in +the superclasses of the class. The additional consistency requirement +comes from the desire to make all of these answers the same. + +The following class definitions are bad, and will lead to errors +either immediately or if an instance is created: +@lisp +(defclass bad-object (funcallable-standard-object) + () + (:metaclass standard-class)) +@end lisp +@lisp +(defclass bad-funcallable-object (standard-object) + () + (:metaclass funcallable-standard-class)) +@end lisp +The following definition is acceptable: +@lisp +(defclass mixin () + ((slot :initarg slot))) +(defclass funcallable-object (funcallable-standard-object mixin) + () + (:metaclass funcallable-standard-class)) +@end lisp +and leads to a class whose instances are funcallable and have one slot. + +@tindex @sbmop{funcallable-standard-object} +Note that this requirement also applies to the class +@code{sb-mop:funcallable-standard-object}, which has metaclass +@code{sb-mop:funcallable-standard-class} rather than +@code{standard-class} as AMOP specifies. + @item -the @code{standard-object} and @code{funcallable-standard-object} -classes are disjoint; - +the requirement that ``No portable class @math{C_p} may inherit, by +virtue of being a direct or indirect subclass of a specified class, any +slot for which the name is a symbol accessible in the +@code{common-lisp-user} package or exported by any package defined in +the ANSI Common Lisp standard.'' is interpreted to mean that the +standardized classes themselves should not have slots named by external +symbols of public packages. + +The rationale behind the restriction is likely to be similar to the ANSI +Common Lisp restriction on defining functions, variables and types named +by symbols in the Common Lisp package: preventing two independent pieces +of software from colliding with each other. + @item -@code{compute-effective-method} only returns one value, not two; - +@findex @sbmop{slot-value-using-class} +@findex @setf{@sbmop{slot-value-using-class}} +@findex @sbmop{slot-boundp-using-class} +specializations of the @code{new-value} argument to @code{(setf +sb-mop:slot-value-using-class)} are not allowed: all user-defined +methods must have a specializer of the class @code{t}. + +This prohibition is motivated by a separation of layers: the +@code{slot-value-using-class} family of functions is intended for use in +implementing different and new slot allocation strategies, rather than +in performing application-level dispatching. Additionally, with this +requirement, there is a one-to-one mapping between metaclass, class and +slot-definition-class tuples and effective methods of @code{(setf +slot-value-using-class)}, which permits optimization of @code{(setf +slot-value-using-class)}'s discriminating function in the same manner as +for @code{slot-value-using-class} and @code{slot-boundp-using-class}. + +Note that application code may specialize on the @code{new-value} +argument of slot accessors. + @item -the system-supplied @code{:around} method for @code{compute-slots} -specialized on @code{funcallable-standard-class} does not respect the -requested order from a user-supplied primary method. +@findex @cl{defclass} +@findex @sbmop{ensure-class} +@findex @sbmop{ensure-class-using-class} +@findex @cl{find-class} +@findex @cl{class-name} +the class named by the @code{name} argument to @code{ensure-class}, if +any, is only redefined if it is the proper name of that class; +otherwise, a new class is created. + +This is consistent with the description of @code{ensure-class} in AMOP +as the functional version of @code{defclass}, which has this behaviour; +however, it is not consistent with the weaker requirement in AMOP, which +states that any class found by @code{find-class}, no matter what its +@code{class-name}, is redefined. @end itemize +@subsection Metaobject Protocol Extensions -@node Threading -@comment node-name, next, previous, up -@subsection Threading (a.k.a Multiprocessing) +In addition, SBCL supports extensions to the Metaobject protocol from +AMOP; at present, they are: + +@itemize + +@item +@findex @cl{defmethod} +@findex @cl{find-class} +@findex @sbmop{intern-eql-specializer} +@findex @sbpcl{make-method-specializers-form} +@findex @sbmop{make-method-lambda} +compile-time support for generating specializer metaobjects from +specializer names in @code{defmethod} forms is provided by the +@code{make-method-specializers-form} function, which returns a form +which, when evaluated in the lexical environment of the +@code{defmethod}, returns a list of specializer metaobjects. This +operator suffers from similar restrictions to those affecting +@code{make-method-lambda}, namely that the generic function must be +defined when the @code{defmethod} form is expanded, so that the +correct method of @code{make-method-specializers-form} is invoked. +The system-provided method on @code{make-method-specializers-form} +generates a call to @code{find-class} for each symbol specializer +name, and a call to @code{intern-eql-specializer} for each @code{(eql +@var{x})} specializer name. + +@item +@findex @cl{find-method} +@findex @sbpcl{parse-specializer-using-class} +@findex @sbpcl{unparse-specializer-using-class} +run-time support for converting between specializer names and +specializer metaobjects, mostly for the purposes of +@code{find-method}, is provided by +@code{parse-specializer-using-class} and +@code{unparse-specializer-using-class}, which dispatch on their first +argument, the generic function associated with a method with the given +specializer. The system-provided methods on those methods convert +between classes and proper names and between lists of the form +@code{(eql @var{x})} and interned eql specializer objects. -SBCL (as of version 0.8.3, on Linux x86 only) supports a fairly -low-level threading interface that maps onto the host operating -system's concept of threads or lightweight processes. +@item +@vindex @sbpcl{+slot-unbound+} +@findex @sbmop{standard-instance-access} +@findex @sbmop{funcallable-standard-instance-access} +distinguishing unbound instance allocated slots from bound ones when +using @code{standard-instance-access} and +@code{funcallable-standard-instance-access} is possible by comparison +to the constant @code{+slot-unbound+}. -@subsubsection Lisp-level view +@end itemize -A rudimentary interface to creating and managing multiple threads can -be found in the @dfn{sb-thread} package. This is intended for public -consumption, so look at the exported symbols and their documentation -strings. +@node Extensible Sequences +@comment node-name, next, previous, up +@section Extensible Sequences -Dynamic bindings to symbols are per-thread. Signal handlers are -per-thread. +@menu +* Iterator Protocol:: +* Simple Iterator Protocol:: +@end menu -Mutexes and condition variables are available for managing access to -shared data: see +ANSI Common Lisp has a class @code{sequence} with subclasses @code{list} and +@code{vector} on which the ``sequence functions'' like @code{find}, +@code{subseq}, etc. operate. As an extension to the ANSI specification, +SBCL allows additional subclasses of @code{sequence} to be defined +@footnote{A motivation, rationale and additional examples for the design +of this extension can be found in the paper @cite{Rhodes, Christophe +(2007): User-extensible sequences in Common Lisp} available for download +at +@url{http://www.doc.gold.ac.uk/~mas01cr/papers/ilc2007/sequences-20070301.pdf}.}. +@tindex @cl{sequence} +@tindex @cl{vector} +@findex @cl{find} +@findex @cl{subseq} + +Users of this extension just make instances of @cl{sequence} subclasses +and transparently operate on them using sequence functions: +@lisp +(coerce (subseq (make-instance 'my-sequence) 5 10) 'list) +@end lisp +From this perspective, no distinction between builtin and user-defined +@code{sequence} subclasses should be necessary. +@findex @cl{coerce} +@findex @cl{subseq} +@findex @cl{make-instance} +@tindex @cl{list} + +Providers of the extension, that is of user-defined @code{sequence} +subclasses, have to adhere to a ``sequence protocol'' which consists of +a set of generic functions in the @code{sequence} package. +@c +A minimal @code{sequence} subclass has to specify @code{standard-object} and +@code{sequence} as its superclasses and has to be the specializer of the +@code{sequence} parameter of methods on at least the following generic +functions: +@tindex @cl{sequence} +@tindex @cl{standard-object} + +@include fun-sb-sequence-length.texinfo +@include fun-sb-sequence-elt.texinfo +@include fun-sb-sequence-setf-elt.texinfo +@include fun-sb-sequence-adjust-sequence.texinfo +@include fun-sb-sequence-make-sequence-like.texinfo + +@code{make-sequence-like} is needed for functions returning +freshly-allocated sequences such as @code{subseq} or +@code{copy-seq}. @code{adjust-sequence} is needed for functions which +destructively modify their arguments such as @code{delete}. In fact, all +other sequence functions can be implemented in terms of the above +functions and actually are, if no additional methods are +defined. However, relying on these generic implementations, in +particular not implementing the iterator protocol can incur a high +performance penalty @xref{Iterator Protocol}. +@tindex @cl{sequence} +@findex @sequence{make-sequence-like} +@findex @cl{subseq} +@findex @cl{copy-seq} +@findex @sequence{adjust-sequence} + +In addition to the mandatory functions above, methods on the following +sequence functions can be defined: + +@include fun-sb-sequence-emptyp.texinfo @itemize +@item +@code{sb-sequence:count}, @code{sb-sequence:count-if}, @code{sb-sequence:count-if-not} + +@item +@code{sb-sequence:find}, @code{sb-sequence:find-if}, @code{sb-sequence:find-if-not} @item -@code{(apropos "mutex" :sb-thread)} - +@code{sb-sequence:position}, @code{sb-sequence:position-if}, @code{sb-sequence:position-if-not} + @item -@code{(apropos "condition" :sb-thread)} - +@code{sb-sequence:subseq} + +@item +@code{sb-sequence:copy-seq} + +@item +@code{sb-sequence:fill} + +@item +@code{sb-sequence:nsubstitute}, @code{sb-sequence:nsubstitute-if}, +@code{sb-sequence:nsubstitute-if-not}, @code{sb-sequence:substitute}, +@code{sb-sequence:substitute-if}, @code{sb-sequence:substitute-if-not} + @item -and the @code{waitqueue} structure +@code{sb-sequence:replace} +@item +@code{sb-sequence:nreverse}, @code{sb-sequence:reverse} + +@item +@code{sb-sequence:reduce} + +@item +@code{sb-sequence:mismatch} + +@item +@code{sb-sequence:search} + +@item +@code{sb-sequence:delete}, @code{sb-sequence:delete-if}, @code{sb-sequence:delete-if-not}, +@code{sb-sequence:remove}, @code{sb-sequence:remove-if}, @code{sb-sequence:remove-if-not}, + +@item +@code{sb-sequence:delete-duplicates}, @code{sb-sequence:remove-duplicates} + +@item +@code{sb-sequence:sort}, @code{sb-sequence:stable-sort} @end itemize -and poke around in their documentation strings. - -@subsubsection Sessions - -If the user has multiple views onto the same Lisp image (for example, -using multiple terminals, or a windowing system, or network access) -they are typically set up as multiple @dfn{sessions} such that each -view has its own collection of foreground/background/stopped threads. -A thread which wishes to create a new session can use -@code{sb-thread:with-new-session} to remove itself from the current -session (which it shares with its parent and siblings) and create a -fresh one. See also @code{sb-thread:make-listener-thread}. - -Within a single session, threads arbitrate between themselves for the -user's attention. A thread may be in one of three notional states: -foreground, background, or stopped. When a background process -attempts to print a repl prompt or to enter the debugger, it will stop -and print a message saying that it has stopped. The user at his -leisure may switch to that thread to find out what it needs. If a -background thread enters the debugger, selecting any restart will put -it back into the background before it resumes. Arbitration for the -input stream is managed by calls to @code{sb-thread:get-foreground} -(which may block) and @code{sb-thread:release-foreground}. - -@code{sb-ext:quit} terminates all threads in the current session, but -leaves other sessions running. - - -@subsubsection Implementation (Linux x86) - -On Linux x86, threading is implemented using @code{clone()} and does -not involve pthreads. This is not because there is anything wrong -with pthreads @emph{per se}, but there is plenty wrong (from our -perspective) with LinuxThreads. SBCL threads are mapped 1:1 onto -Linux tasks which share a VM but nothing else - each has its own -process id and can be seen in e.g. @command{ps} output. - -Per-thread local bindings for special variables is achieved using the -%fs segment register to point to a per-thread storage area. This may -cause interesting results if you link to foreign code that expects -threading or creates new threads, and the thread library in question -uses %fs in an incompatible way. - -There are two implementation mechanisms for queueing. If SBCL was -built on an NPTL-capable Linux system (2.6 or some vendor 2.4 ports) -with the @code{:SB-FUTEX} feature, queuing will be done using the -@code{sys_futex()} system call if it's available at runtime. -Otherwise it will fall back to using @code{sigtimedwait()} to sleep -and a signal (@code{SIG_DEQUEUE}, one of the POSIX RT signals) to -wake. - -Garbage collection is done with the existing Conservative Generational -GC. Allocation is done in small (typically 8k) regions: each thread -has its own region so this involves no stopping. However, when a -region fills, a lock must be obtained while another is allocated, and -when a collection is required, all processes are stopped. This is -achieved by sending them signals, which may make for interesting -behaviour if they are interrupted in system calls. The streams -interface is believed to handle the required system call restarting -correctly, but this may be a consideration when making other blocking -calls e.g. from foreign library code. - -Large amounts of the SBCL library have not been inspected for -thread-safety. Some of the obviously unsafe areas have large locks -around them, so compilation and fasl loading, for example, cannot be -parallelized. Work is ongoing in this area. - -A new thread by default is created in the same POSIX process group and -session as the thread it was created by. This has an impact on -keyboard interrupt handling: pressing your terminal's intr key -(typically @kbd{Control-C}) will interrupt all processes in the -foreground process group, including Lisp threads that SBCL considers -to be notionally `background'. This is undesirable, so background -threads are set to ignore the SIGINT signal. - -@code{sb-thread:make-listener-thread} in addition to creating a new -Lisp session makes a new POSIX session, so that pressing -@kbd{Control-C} in one window will not interrupt another listener - -this has been found to be embarrassing. +In the spirit of @code{dolist}, generic sequences can be traversed using +the macro +@findex @cl{dolist} + +@include macro-sb-sequence-dosequence.texinfo + +@node Iterator Protocol +@comment node-name, next, previous, up +@subsection Iterator Protocol + +The iterator protocol allows subsequently accessing some or all elements +of a sequence in forward or reverse direction. Users first call +@code{make-sequence-iterator} to create an iteration state and +receive functions to query and mutate it. These functions allow, among +other things, moving to, retrieving or modifying elements of the +sequence. An iteration state consists of a state object, a limit object, +a from-end indicator and the following six functions to query or mutate +this state: +@findex @sequence{make-sequence-iterator} +@deffn {Function} @code{step function} sequence iterator from-end +Moves the iterator one position forward or backward in the associated +sequence depending on the iteration direction. +@end deffn +@deffn {Function} @code{endp function} sequence iterator limit from-end +Returns non-@code{nil} when the iterator has reached the end of the +associated sequence with respect to the iteration direction. +@end deffn +@deffn {Function} @code{element function} sequence iterator +Returns the sequence element associated to the current position of the +iteration. +@end deffn +@deffn {Function} @code{setf element function} new-value sequence iterator +Destructively modifies the associates sequence by replacing the sequence +element associated to the current iteration position with a new value. +@end deffn +@deffn {Function} @code{index function} sequence iterator +Returns the position of the iteration in the associated sequence. +@end deffn +@deffn {Function} @code{copy function} sequence iterator +Returns a copy of the iteration state which can be mutated independently +of the copied iteration state. +@end deffn + +An iterator is created by calling: + +@include fun-sb-sequence-make-sequence-iterator.texinfo + +Note that @code{make-sequence-iterator} calls +@code{make-simple-sequence-iterator} when there is no specialized +method for a particular @code{sequence} subclass. @xref{Simple Iterator +Protocol}. +@findex @sequence{make-sequence-iterator} +@findex @sequence{make-simple-sequence-iterator} +@tindex @cl{sequence} + +The following convenience macros simplify traversing sequences using +iterators: + +@include macro-sb-sequence-with-sequence-iterator.texinfo +@include macro-sb-sequence-with-sequence-iterator-functions.texinfo + +@node Simple Iterator Protocol +@comment node-name, next, previous, up +@subsection Simple Iterator Protocol +For cases in which the full flexibility and performance of the general +sequence iterator protocol is not required, there is a simplified +sequence iterator protocol consisting of a few generic functions which +can be specialized for iterator classes: + +@include fun-sb-sequence-iterator-step.texinfo +@include fun-sb-sequence-iterator-endp.texinfo +@include fun-sb-sequence-iterator-element.texinfo +@include fun-sb-sequence-setf-iterator-element.texinfo +@include fun-sb-sequence-iterator-index.texinfo +@include fun-sb-sequence-iterator-copy.texinfo + +Iterator objects implementing the above simple iteration protocol are +created by calling the following generic function: + +@include fun-sb-sequence-make-simple-sequence-iterator.texinfo @node Support For Unix @comment node-name, next, previous, up -@subsection Support For Unix +@section Support For Unix + +@menu +* Command-line arguments:: +* Querying the process environment:: +* Running external programs:: +@end menu + +@node Command-line arguments +@subsection Command-line arguments +@vindex @sbext{@earmuffs{posix-argv}} The UNIX command line can be read from the variable -@code{sb-ext:*posix-argv*}. The UNIX environment can be queried with -the @code{sb-ext:posix-getenv} function. +@code{sb-ext:*posix-argv*}. + +@node Querying the process environment +@subsection Querying the process environment + +The UNIX environment can be queried with the +@code{sb-ext:posix-getenv} function. @include fun-sb-ext-posix-getenv.texinfo -The SBCL system can be terminated with @code{sb-ext:quit}, (but see -notes in @ref{Threading} about the interaction between this feature and -sessions) optionally returning a specified numeric value to the -calling Unix process. The normal Unix idiom of terminating on end of -file on input is also supported. +@node Running external programs +@subsection Running external programs + +External programs can be run with @code{sb-ext:run-program}. +@footnote{In SBCL versions prior to 1.0.13, @code{sb-ext:run-program} +searched for executables in a manner somewhat incompatible with other +languages. As of this version, SBCL uses the system library routine +@code{execvp(3)}, and no longer contains the function, +@code{find-executable-in-search-path}, which implemented the old +search. Users who need this function may find it +in @file{run-program.lisp} versions 1.67 and earlier in SBCL's CVS +repository here +@url{http://sbcl.cvs.sourceforge.net/sbcl/sbcl/src/code/run-program.lisp?view=log}. However, +we caution such users that this search routine finds executables that +system library routines do not.} + +@include fun-sb-ext-run-program.texinfo + +When @code{sb-ext:run-program} is called with @code{wait} equal to +NIL, an instance of class @var{sb-ext:process} is returned. The +following functions are available for use with processes: + +@include fun-sb-ext-process-p.texinfo + +@include fun-sb-ext-process-input.texinfo + +@include fun-sb-ext-process-output.texinfo + +@include fun-sb-ext-process-error.texinfo + +@include fun-sb-ext-process-alive-p.texinfo -@include fun-sb-ext-quit.texinfo +@include fun-sb-ext-process-status.texinfo + +@include fun-sb-ext-process-wait.texinfo + +@include fun-sb-ext-process-exit-code.texinfo + +@include fun-sb-ext-process-core-dumped.texinfo + +@include fun-sb-ext-process-close.texinfo + +@include fun-sb-ext-process-kill.texinfo @node Customization Hooks for Users @comment node-name, next, previous, up -@subsection Customization Hooks for Users +@section Customization Hooks for Users The toplevel repl prompt may be customized, and the function that reads user input may be replaced completely. @@ -326,40 +658,148 @@ mechanisms as follows: @include fun-common-lisp-ed.texinfo @include var-sb-ext-star-ed-functions-star.texinfo -@node Tools To Help Developers +Conditions of type @code{warning} and @code{style-warning} are +sometimes signaled at runtime, especially during execution of Common +Lisp defining forms such as @code{defun}, @code{defmethod}, etc. To +muffle these warnings at runtime, SBCL provides a variable +@code{sb-ext:*muffled-warnings*}: + +@include var-sb-ext-star-muffled-warnings-star.texinfo + +@node Tools To Help Developers @comment node-name, next, previous, up -@subsection Tools To Help Developers +@section Tools To Help Developers +@findex @cl{trace} +@findex @cl{inspect} SBCL provides a profiler and other extensions to the ANSI @code{trace} -facility. For more information, see @ref{macro-common-lisp-trace}. +facility. For more information, see @ref{Macro common-lisp:trace}. The debugger supports a number of options. Its documentation is -accessed by typing @kbd{help} at the debugger prompt. @xref{The -Debugger}. +accessed by typing @kbd{help} at the debugger prompt. @xref{Debugger}. Documentation for @code{inspect} is accessed by typing @kbd{help} at the @code{inspect} prompt. -@node Interface To Low-Level SBCL Implementation +@node Resolution of Name Conflicts +@section Resolution of Name Conflicts +@tindex @sbext{name-conflict} +@findex @sbext{name-conflict-symbols} + +The ANSI standard (section 11.1.1.2.5) requires that name conflicts in +packages be resolvable in favour of any of the conflicting symbols. In +the interactive debugger, this is achieved by prompting for the symbol +in whose favour the conflict should be resolved; for programmatic use, +the @code{sb-ext:resolve-conflict} restart should be invoked with one +argument, which should be a member of the list returned by the condition +accessor @code{sb-ext:name-conflict-symbols}. + +@node Hash Table Extensions @comment node-name, next, previous, up -@subsection Interface To Low-Level SBCL Implementation +@section Hash Table Extensions +@cindex Hash tables -SBCL has the ability to save its state as a file for later -execution. This functionality is important for its bootstrapping -process, and is also provided as an extension to the user. Note that -foreign libraries loaded via @code{load-1-foreign} don't survive this -process; a core should not be saved in this case. +Hash table extensions supported by SBCL are all controlled by keyword +arguments to @code{make-hash-table}. -@emph{FIXME: what should be done for foreign libraries?} +@include fun-common-lisp-make-hash-table.texinfo -@emph{FIXME: document load-1-foreign somewhere} +@include macro-sb-ext-define-hash-table-test.texinfo -@include fun-sb-ext-save-lisp-and-die.texinfo +@include macro-sb-ext-with-locked-hash-table.texinfo +@include fun-sb-ext-hash-table-synchronized-p.texinfo + +@include fun-sb-ext-hash-table-weakness.texinfo + +@node Random Number Generation +@comment node-name, next, previous, up +@section Random Number Generation +@cindex Random Number Generation + +The initial value of @code{*random-state*} is the same each time SBCL +is started. This makes it possible for user code to obtain repeatable +pseudo random numbers using only standard-provided functionality. See +@code{seed-random-state} below for an SBCL extension that allows to +seed the random number generator from given data for an additional +possibility to achieve this. Non-repeatable random numbers can always +be obtained using @code{(make-random-state t)}. + +The sequence of numbers produced by repeated calls to @code{random} +starting with the same random state and using the same sequence of +@code{limit} arguments is guaranteed to be reproducible only in the +same version of SBCL on the same platform, using the same code under +the same evaluator mode and compiler optimization qualities. Just two +examples of differences that may occur otherwise: calls to +@code{random} can be compiled differently depending on how much is +known about the @code{limit} argument at compile time, yielding +different results even if called with the same argument at run time, +and the results can differ depending on the machine's word size, for +example for limits that are fixnums under 64-bit word size but bignums +under 32-bit word size. + +@include fun-sb-ext-seed-random-state.texinfo + +Some notes on random floats: The standard doesn't prescribe a specific +method of generating random floats. The following paragraph describes +SBCL's current implementation and should be taken purely informational, +that is, user code should not depend on any of its specific properties. +The method used has been chosen because it is common, conceptually +simple and fast. + +To generate random floats, SBCL evaluates code that has an equivalent +effect as +@lisp +(* limit + (float (/ (random (expt 2 23)) (expt 2 23)) 1.0f0)) +@end lisp +(for single-floats) and correspondingly (with @code{52} and +@code{1.0d0} instead of @code{23} and @code{1.0f0}) for double-floats. +Note especially that this means that zero is a possible return value +occurring with probability @code{(expt 2 -23)} respectively +@code{(expt 2 -52)}. Also note that there exist twice as many +equidistant floats between 0 and 1 as are generated. For example, the +largest number that @code{(random 1.0f0)} ever returns is +@code{(float (/ (1- (expt 2 23)) (expt 2 23)) 1.0f0)} while +@code{(float (/ (1- (expt 2 24)) (expt 2 24)) 1.0f0)} is the +largest single-float less than 1. This is a side effect of the fact +that the implementation uses the fastest possible conversion from bits +to floats. + +SBCL currently uses the Mersenne Twister as its random number +generator, specifically the 32-bit version under both 32- and 64-bit +word size. The seeding algorithm has been improved several times by +the authors of the Mersenne Twister; SBCL uses the third version +(from 2002) which is still the most recent as of June 2012. The +implementation has been tested to provide output identical to the +recommended C implementation. + +While the Mersenne Twister generates random numbers of much better +statistical quality than other widely used generators, it uses only +linear operations modulo 2 and thus fails some statistical +tests@footnote{See chapter 7 "Testing widely used RNGs" in +@cite{TestU01: A C Library for Empirical Testing of Random Number +Generators} by Pierre L'Ecuyer and Richard Simard, ACM Transactions on +Mathematical Software, Vol. 33, article 22, 2007.}. +For example, the distribution of ranks of (sufficiently large) random +binary matrices is much distorted compared to the theoretically +expected one when the matrices are generated by the Mersenne Twister. +Thus, applications that are sensitive to this aspect should use a +different type of generator. + +@node Miscellaneous Extensions +@comment node-name, next, previous, up +@section Miscellaneous Extensions + +@include fun-sb-ext-array-storage-vector.texinfo +@include fun-sb-ext-delete-directory.texinfo +@include fun-sb-ext-get-time-of-day.texinfo +@include macro-sb-ext-wait-for.texinfo +@include fun-sb-ext-version-assert.texinfo @node Stale Extensions @comment node-name, next, previous, up -@subsection Stale Extensions +@section Stale Extensions SBCL has inherited from CMUCL various hooks to allow the user to tweak and monitor the garbage collection process. These are somewhat @@ -381,18 +821,18 @@ list. @node Efficiency Hacks @comment node-name, next, previous, up -@subsection Efficiency Hacks +@section Efficiency Hacks The @code{sb-ext:purify} function causes SBCL first to collect all -garbage, then to mark all uncollected objects as permanent, never -again attempting to collect them as garbage. This can cause a large -increase in efficiency when using a primitive garbage collector, or a -more moderate increase in efficiency when using a more sophisticated -garbage collector which is well suited to the program's memory usage -pattern. It also allows permanent code to be frozen at fixed -addresses, a precondition for using copy-on-write to share code -between multiple Lisp processes. it is less important with modern -generational garbage collectors. +garbage, then to mark all uncollected objects as permanent, never again +attempting to collect them as garbage. This can cause a large increase +in efficiency when using a primitive garbage collector, or a more +moderate increase in efficiency when using a more sophisticated garbage +collector which is well suited to the program's memory usage pattern. It +also allows permanent code to be frozen at fixed addresses, a +precondition for using copy-on-write to share code between multiple Lisp +processes. This is less important with modern generational garbage +collectors, but not all SBCL platforms use such a garbage collector. @include fun-sb-ext-purify.texinfo @@ -405,13 +845,3 @@ not checked. In short: don't use it. The @code{sb-ext:freeze-type} declaration declares that a type will never change, which can make type testing (@code{typep}, etc.) more efficient for structure types. - -The @code{sb-ext:constant-function} declaration specifies -that a function will always return the same value for the same -arguments, which may allow the compiler to optimize calls -to it. This is appropriate for functions like @code{sqrt}, but -is @emph{not} appropriate for functions like @code{aref}, -which can change their return values when the underlying data are -changed. -@c