Introduction
************

SBCL is a mostly-conforming implementation of the ANSI Common Lisp standard. This manual focuses on behavior which is specific to SBCL, not on behavior which is common to all implementations of ANSI Common Lisp.

ANSI Conformance
================

Essentially every type of non-conformance is considered a bug. (The exceptions involve internal inconsistencies in the standard.) In SBCL the master record of known bugs is in the BUGS file in the distribution.

The recommended way to report bugs is through the sbcl-help or sbcl-devel mailing lists. For mailing list addresses, see SBCL Homepage.

Extensions
==========

SBCL comes with numerous extensions, some in core and some in modules loadable with require. Unfortunately, not all of these extensions have proper documentation yet.

System Definition Tool
     asdf is a flexible and popular protocol-oriented system definition tool by Daniel Barlow. See the asdf manual (asdf), for more information.

Third-party Extension Installation Tool
     asdf-install is a tool that can be used to download and install third-party libraries and applications, automatically handling dependencies, etc.

Foreign Function Interface
     sb-alien package allows interfacing with C-code, loading shared object files, etc. See Foreign Function Interface.

     sb-grovel can be used to partially automate generation of foreign function interface definitions. See sb-grovel.

Recursive Event Loop
     SBCL provides a recursive event loop (serve-event) for doing non-blocking IO on multiple streams without using threads.

Metaobject Protocol
     sb-mop package provides a metaobject protocol for the Common Lisp Object System as described in Art of Metaobject Protocol.

Native Threads
     SBCL has native threads on x86/Linux, capable of taking advantage of SMP on multiprocessor machines. See Threading.

Network Interface
     sb-bsd-sockets is a low-level networking interface, providing both TCP and UDP sockets. xNetworking.

Introspective Facilities
     sb-introspect module offers numerous introspective extensions, including access to function lambda-lists.

Operating System Interface
     sb-ext contains a number of functions for running external processes, accessing environment variables, etc.

     sb-posix module provides a lispy interface to standard POSIX facilities.

Extensible Streams
     sb-gray is an implentation of Gray Streams. See Gray Streams.

     sb-simple-streams is an implementation of the simple streams API proposed by Franz Inc. See Simple Streams.

Profiling
     sb-profile is a exact per-function profiler. See Accurate Profiler.

     sb-sprof is a statistical profiler, capable of call-graph generation and instruction level profiling. See Statistical Profiler.

Customization Hooks
     SBCL contains a number of extra-standard customization hooks that can be used to tweak the behaviour of the system. See Customization Hooks for Users.

     sb-aclrepl provides an Allegro CL -style toplevel for SBCL, as an alternative to the classic CMUCL-style one. See sb-aclrepl.

CLTL2 Compatility Layer
     sb-cltl2 module provides compiler-let and environment access functionality described in Common Lisp The Language, 2nd Edition which were removed from the language during the ANSI standardization process.

Executable Fasl Packaging
     sb-executable can be used to concatenate multiple fasls into a single executable (though the presense of an SBCL runtime and core image is still required to run it).

Bitwise Rotation
     sb-rotate-byte provides an efficient primitive for bitwise rotation of integers, an operation required by eg. numerous cryptographic algorightms, but not available as a primitive in ANSI Common Lisp. See sb-rotate-byte.

Test Harness
     sb-rt module is a simple yet attractive regression and unit-test framework.

MD5 Sums
     sb-md5 is an implementation of the MD5 message digest algorithm for Common Lisp, using the modular arithmetic optimizations provided by SBCL. See sb-md5.

Idiosyncrasies
==============

The information in this section describes some of the ways that SBCL deals with choices that the ANSI standard leaves to the implementation.

Declarations
------------

Declarations are generally treated as assertions. This general principle, and its implications, and the bugs which still keep the compiler from quite satisfying this principle, are discussed in Declarations as Assertions.

Compiler-only Implementation
----------------------------

SBCL is essentially a compiler-only implementation of Common Lisp. That is, for all but a few special cases, eval creates a lambda expression, calls compile on the lambda expression to create a compiled function, and then calls funcall on the resulting function object. This is explicitly allowed by the ANSI standard, but leads to some oddities, e.g. collapsing functionp and compiled-function-p into the same predicate.

Defining Constants
------------------

SBCL is quite strict about ANSI's definition of defconstant. ANSI says that doing defconstant of the same symbol more than once is undefined unless the new value is eql to the old value. Conforming to this specification is a nuisance when the ``constant'' value is only constant under some weaker test like string= or equal.

It's especially annoying because, in SBCL, defconstant takes effect not only at load time but also at compile time, so that just compiling and loading reasonable code like

     (defconstant +foobyte+ '(1 4))

runs into this undefined behavior. Many implementations of Common Lisp try to help the programmer around this annoyance by silently accepting the undefined code and trying to do what the programmer probably meant.

SBCL instead treats the undefined behavior as an error. Often such code can be rewritten in portable ANSI Common Lisp which has the desired behavior. E.g., the code above can be given an exactly defined meaning by replacing defconstant either with defparameter or with a customized macro which does the right thing, eg.

     (defmacro define-constant (name value &optional doc)
       `(defconstant ,name (if (boundp ',name) (symbol-value ',name) ,value)
                           ,@(when doc (list doc))))

or possibly along the lines of the defconstant-eqx macro used internally in the implementation of SBCL itself. In circumstances where this is not appropriate, the programmer can handle the condition type sb-ext:defconstant-uneql, and choose either the continue or abort restart as appropriate.

Style Warnings
--------------

SBCL gives style warnings about various kinds of perfectly legal code, e.g.

  * defmethod without a preceding defgeneric;

  * multiple defuns of the same symbol in different units;

  * special variables not named in the conventional *foo* style, and lexical variables unconventionally named in the *foo* style

This causes friction with people who point out that other ways of organizing code (especially avoiding the use of defgeneric) are just as aesthetically stylish.  However, these warnings should be read not as ``warning, bad aesthetics detected, you have no style'' but ``warning, this style keeps the compiler from understanding the code as well as you might like.'' That is, unless the compiler warns about such conditions, there's no way for the compiler to warn about some programming errors which would otherwise be easy to overlook. (Related bug: The warning about multiple defuns is pointlessly annoying when you compile and then load a function containing defun wrapped in eval-when, and ideally should be suppressed in that case, but still isn't as of SBCL 0.7.6.)

Development Tools
=================

Editor Integration
------------------

Though SBCL can be used running ``bare'', the recommended mode of development is with an editor connected to SBCL, supporting not only basic lisp editing (paren-matching, etc), but providing among other features an integrated debugger, interactive compilation, and automated documentation lookup.

Currently SLIME (Historically, the ILISP package at http://ilisp.cons.org/ provided similar functionality, but it does not support modern SBCL versions.) (Superior Lisp Interaction Mode for Emacs) together with Emacs is recommended for use with SBCL, though other options exist as well.

SLIME can be downloaded from http://www.common-lisp.net/project/slime/.

Language Reference
------------------

CLHS (Common Lisp Hyperspec) is a hypertext version of the ANSI standard, made freely available by Xanalyst -- an invaluable reference.

See: http://www.lispworks.com/reference/HyperSpec/index.html

More SBCL Information
=====================

SBCL Homepage
-------------

The SBCL website at http://www.sbcl.org/ has some general information, plus links to mailing lists devoted to SBCL, and to archives of these mailing lists. Subscribing to the mailing lists sbcl-help and sbcl-announce is recommended: both are fairly low-volume, and help you keep abrest with SBCL development.

Additional Distributed Documentation
------------------------------------

Besides this user manual both SBCL source and binary distributions include some other SBCL-specific documentation files, which should be installed along with this manual in on your system, eg. in /usr/local/share/doc/sbcl/.

BUGS
     Lists known bugs in the distribution.

COPYING
     Licence and copyright summary.

CREDITS
     Authorship information on various parts of SBCL.

INSTALL
     Covers installing SBCL from both source and binary distributions on your system, and also has some installation related troubleshooting information.

NEWS
     Summarizes changes between various SBCL versions.

SUPPORT
     Lists SBCL developers available for-pay development of SBCL.

Online Documentation
--------------------

Documentation for non-ANSI extensions for various commands is available online from the SBCL executable itself. The extensions for functions which have their own command prompts (e.g. the debugger, and inspect) are documented in text available by typing help at their command prompts. The extensions for functions which don't have their own command prompt (such as trace) are described in their documentation strings, unless your SBCL was compiled with an option not to include documentation strings, in which case the documentation strings are only readable in the source code.

Internals Documentation
-----------------------

If you're interested in the development of the SBCL system itself, then subcribing to sbcl-devel is a good idea.

SBCL internals documentation -- besides comments in the source -- is currenly maitained as a wiki-like website: http://sbcl-internals.cliki.net/.

Some low-level information describing the programming details of the conversion from CMUCL to SBCL is available in the doc/FOR-CMUCL-DEVELOPERS file in the SBCL distribution, though it is not installed by default.

More Common Lisp Information
============================

Internet Community
------------------

The Common Lisp internet community is fairly diverse: news://comp.lang.lisp is fairly high volume newsgroup, but has a rather poor signal/noise ratio. Various special interest mailing lists and IRC tend to provide more content and less flames. http://www.lisp.org and http://www.cliki.net contain numerous pointers places in the net where lispers talks shop.

Third-party Libraries
---------------------

For a wealth of information about free Common Lisp libraries and tools we recommend checking out CLiki: http://www.cliki.net/.

Common Lisp Books
-----------------

If you're not a programmer and you're trying to learn, many introductory Lisp books are available. However, we don't have any standout favorites. If you can't decide, try checking the Usenet news://comp.lang.lisp FAQ for recent recommendations.

If you are an experienced programmer in other languages but need to learn about Common Lisp, some books stand out:

ANSI Common Lisp, by Paul Graham
     Introduces most of the language, though some parts (eg. CLOS) are covered only lightly.

On Lisp, by Paul Graham
     An in-depth treatment of macros, but not recommended as a first Common Lisp book, since it is slightly pre-ANSI so you need to be on your guard against non-standard usages, and since it doesn't really even try to cover the language as a whole, focusing solely on macros. Downloadable from http://www.paulgraham.com/onlisp.html.

Paradigms Of Artificial Intelligence Programming, by Peter Norvig
     Good information on general Common Lisp programming, and many nontrivial examples. Whether or not your work is AI, it's a very good book to look at.

Object-Oriented Programming In Common Lisp, by Sonya Keene
     None the books above emphasize CLOS, but this one does. Even if you're very knowledgeable about object oriented programming in the abstract, it's worth looking at this book if you want to do any OO in Common Lisp. Some abstractions in CLOS (especially multiple dispatch) go beyond anything you'll see in most OO systems, and there are a number of lesser differences as well. This book tends to help with the culture shock.

Art Of Metaobject Programming, by Gregor Kiczales et al.
     Currently to prime source of information on the Common Lisp Metaobject Protocol, which is supported by SBCL. Section 2 (Chapers 5 and 6) are freely available at http://www.lisp.org/mop/.

History and Implementation of SBCL
==================================

You can work productively with SBCL without knowing anything understanding anything about where it came from, how it is implemented, or how it extends the ANSI Common Lisp standard. However, a little knowledge can be helpful in order to understand error messages, to troubleshoot problems, to understand why some parts of the system are better debugged than others, and to anticipate which known bugs, known performance problems, and missing extensions are likely to be fixed, tuned, or added.

SBCL is descended from CMUCL, which is itself descended from Spice Lisp, including early implementations for the Mach operating system on the IBM RT, back in the 1980s. Some design decisions from that time are still reflected in the current implementation:

  * The system expects to be loaded into a fixed-at-compile-time location in virtual memory, and also expects the location of all of its heap storage to be specified at compile time.

  * The system overcommits memory, allocating large amounts of address space from the system (often more than the amount of virtual memory available) and then failing if ends up using too much of the allocated storage.

  * The system is implemented as a C program which is responsible for supplying low-level services and loading a Lisp .core file.

SBCL also inherited some newer architectural features from CMUCL. The most important is that on some architectures it has a generational garbage collector (``GC''), which has various implications (mostly good) for performance. These are discussed in another chapter, Efficiency.

SBCL has diverged from CMUCL in that SBCL is now essentially a ``compiler-only implementation'' of Common Lisp. This is a change in implementation strategy, taking advantage of the freedom ``any of these facilities might share the same execution strategy'' guaranteed in the ANSI specification section 3.1 (``Evaluation''). It does not mean SBCL can't be used interactively, and in fact the change is largely invisible to the casual user, since SBCL still can and does execute code interactively by compiling it on the fly. (It is visible if you know how to look, like using compiled-function-p; and it is visible in the way that that SBCL doesn't have many bugs which behave differently in interpreted code than in compiled code.) What it means is that in SBCL, the eval function only truly ``interprets'' a few easy kinds of forms, such as symbols which are boundp. More complicated forms are evaluated by calling compile and then calling funcall on the returned result.

The direct ancestor of SBCL is the x86 port of CMUCL. This port was in some ways the most cobbled-together of all the CMUCL ports, since a number of strange changes had to be made to support the register-poor x86 architecture. Some things (like tracing and debugging) do not work particularly well there. SBCL should be able to improve in these areas (and has already improved in some other areas), but it takes a while.

On the x86 SBCL -- like the x86 port of CMUCL -- uses a conservative GC. This means that it doesn't maintain a strict separation between tagged and untagged data, instead treating some untagged data (e.g. raw floating point numbers) as possibly-tagged data and so not collecting any Lisp objects that they point to. This has some negative consequences for average time efficiency (though possibly no worse than the negative consequences of trying to implement an exact GC on a processor architecture as register-poor as the X86) and also has potentially unlimited consequences for worst-case memory efficiency. In practice, conservative garbage collectors work reasonably well, not getting anywhere near the worst case. But they can occasionally cause odd patterns of memory usage.

The fork from CMUCL was based on a major rewrite of the system bootstrap process. CMUCL has for many years tolerated a very unusual ``build'' procedure which doesn't actually build the complete system from scratch, but instead progressively overwrites parts of a running system with new versions. This quasi-build procedure can cause various bizarre bootstrapping hangups, especially when a major change is made to the system. It also makes the connection between the current source code and the current executable more tenuous than in other software systems -- it's easy to accidentally ``build'' a CMUCL system containing characteristics not reflected in the current version of the source code.

Other major changes since the fork from CMUCL include

  * SBCL has removed many CMUCL extensions, (e.g. IP networking, remote procedure call, Unix system interface, and X11 interface) from the core system. Most of these are available as contributed modules (distributed with sbcl) or third-party modules instead.

  * SBCL has deleted or deprecated some nonstandard features and code complexity which helped efficiency at the price of maintainability. For example, the SBCL compiler no longer implements memory pooling internally (and so is simpler and more maintainable, but generates more garbage and runs more slowly), and various block-compilation efficiency-increasing extensions to the language have been deleted or are no longer used in the implementation of SBCL itself.