add support for package::form-read-in-package syntax

[sbcl.git] / doc / manual / beyond-ansi.texinfo

@node  Beyond the ANSI Standard
@comment  node-name,  next,  previous,  up
@chapter Beyond the ANSI Standard

SBCL is derived from CMUCL, which implements many extensions to the
ANSI standard. SBCL doesn't support as many extensions as CMUCL, but
it still has quite a few.  @xref{Contributed Modules}.

@menu
* Reader Extensions::
* Garbage Collection::
* Metaobject Protocol::
* Support For Unix::
* Customization Hooks for Users::
* Tools To Help Developers::
* Resolution of Name Conflicts::
* Hash Table Extensions::
* Miscellaneous Extensions::
* Stale Extensions::
* Efficiency Hacks::
@end menu

@node Reader Extensions
@comment  node-name,  next,  previous,  up
@section Reader Extensions
@cindex Reader Extensions

SBCL supports extended package prefix syntax, which allows specifying
the package to read an arbitrary form in:

@lisp
<package-name>::<form-in-package>
@end lisp

Example:

@lisp
  'foo::(bar quux zot) == '(foo::bar foo::quux foo::zot)
@end lisp

@node  Garbage Collection
@comment  node-name,  next,  previous,  up
@section Garbage Collection
@cindex Garbage collection

SBCL provides additional garbage collection functionality not
specified by ANSI.

@include var-sb-ext-star-after-gc-hooks-star.texinfo
@include fun-sb-ext-gc.texinfo

@subsection Finalization
@cindex Finalization

Finalization allows code to be executed after an object has been
garbage collected. This is useful for example for releasing foreign
memory associated with a Lisp object.

@include fun-sb-ext-finalize.texinfo
@include fun-sb-ext-cancel-finalization.texinfo

@subsection Weak Pointers
@cindex Weak pointers

Weak pointers allow references to objects to be maintained without
keeping them from being garbage collected: useful for building caches
among other things.

Hash tables can also have weak keys and values: @pxref{Hash Table
Extensions}.

@include fun-sb-ext-make-weak-pointer.texinfo
@include fun-sb-ext-weak-pointer-value.texinfo

@subsection Introspection and Tuning

@include var-sb-ext-star-gc-run-time-star.texinfo
@include fun-sb-ext-bytes-consed-between-gcs.texinfo
@include fun-sb-ext-dynamic-space-size.texinfo
@include fun-sb-ext-get-bytes-consed.texinfo
@include fun-sb-ext-gc-logfile.texinfo
@include fun-sb-ext-generation-average-age.texinfo
@include fun-sb-ext-generation-bytes-allocated.texinfo
@include fun-sb-ext-generation-bytes-consed-between-gcs.texinfo
@include fun-sb-ext-generation-minimum-age-before-gc.texinfo
@include fun-sb-ext-generation-number-of-gcs-before-promotion.texinfo
@include fun-sb-ext-generation-number-of-gcs.texinfo

@node Metaobject Protocol
@comment  node-name,  next,  previous,  up
@section Metaobject Protocol

@subsection AMOP Compatibility of Metaobject Protocol

SBCL supports a metaobject protocol which is intended to be compatible
with AMOP; present exceptions to this (as distinct from current bugs)
are:

@itemize

@item
@findex @sbmop{compute-effective-method}
@code{compute-effective-method} only returns one value, not two.

There is no record of what the second return value was meant to
indicate, and apparently no clients for it.

@item
@tindex @cl{generic-function}
@tindex @cl{standard-generic-function}
@tindex @sbmop{funcallable-standard-object}
@tindex @cl{standard-object}
@tindex @cl{function}
The direct superclasses of @code{sb-mop:funcallable-standard-object} are
@code{(function standard-object)}, not @code{(standard-object function)}.

This is to ensure that the @code{standard-object} class is the last of
the standardized classes before @code{t} appearing in the class
precedence list of @code{generic-function} and
@code{standard-generic-function}, as required by section 1.4.4.5 of the
ANSI specification.

@item
@findex @cl{ensure-generic-function}
@findex @sbmop{generic-function-declarations}
the arguments @code{:declare} and @code{:declarations} to
@code{ensure-generic-function} are both accepted, with the leftmost
argument defining the declarations to be stored and returned by
@code{generic-function-declarations}.

Where AMOP specifies @code{:declarations} as the keyword argument to
@code{ensure-generic-function}, the Common Lisp standard specifies
@code{:declare}.  Portable code should use @code{:declare}.

@item
@findex @sbmop{validate-superclass}
@findex @sbmop{finalize-inheritance}
@tindex @cl{standard-class}
@tindex @sbmop{funcallable-standard-class}
@tindex @cl{function}
@findex @sbmop{class-prototype}
although SBCL obeys the requirement in AMOP that
@code{validate-superclass} should treat @code{standard-class} and
@code{funcallable-standard-class} as compatible metaclasses, we
impose an additional requirement at class finalization time: a class
of metaclass @code{funcallable-standard-class} must have
@code{function} in its superclasses, and a class of metaclass
@code{standard-class} must not.

@findex @cl{typep}
@findex @cl{class-of}
@findex @cl{subtypep}
After a class has been finalized, it is associated with a class
prototype which is accessible by a standard mop function
@code{sb-mop:class-prototype}.  The user can then ask whether this
object is a @code{function} or not in several different ways: whether it
is a function according to @code{typep}; whether its @code{class-of} is
@code{subtypep} @code{function}, or whether @code{function} appears in
the superclasses of the class.  The additional consistency requirement
comes from the desire to make all of these answers the same.

The following class definitions are bad, and will lead to errors
either immediately or if an instance is created:
@lisp
(defclass bad-object (funcallable-standard-object)
  ()
  (:metaclass standard-class))
@end lisp
@lisp
(defclass bad-funcallable-object (standard-object)
  ()
  (:metaclass funcallable-standard-class))
@end lisp
The following definition is acceptable:
@lisp
(defclass mixin ()
  ((slot :initarg slot)))
(defclass funcallable-object (funcallable-standard-object mixin)
  ()
  (:metaclass funcallable-standard-class))
@end lisp
and leads to a class whose instances are funcallable and have one slot.

@tindex @sbmop{funcallable-standard-object}
Note that this requirement also applies to the class
@code{sb-mop:funcallable-standard-object}, which has metaclass
@code{sb-mop:funcallable-standard-class} rather than
@code{standard-class} as AMOP specifies.

@item
the requirement that ``No portable class @math{C_p} may inherit, by
virtue of being a direct or indirect subclass of a specified class, any
slot for which the name is a symbol accessible in the
@code{common-lisp-user} package or exported by any package defined in
the ANSI Common Lisp standard.'' is interpreted to mean that the
standardized classes themselves should not have slots named by external
symbols of public packages.

The rationale behind the restriction is likely to be similar to the ANSI
Common Lisp restriction on defining functions, variables and types named
by symbols in the Common Lisp package: preventing two independent pieces
of software from colliding with each other.

@item
@findex @sbmop{slot-value-using-class}
@findex @setf{@sbmop{slot-value-using-class}}
@findex @sbmop{slot-boundp-using-class}
specializations of the @code{new-value} argument to @code{(setf
sb-mop:slot-value-using-class)} are not allowed: all user-defined
methods must have a specializer of the class @code{t}.

This prohibition is motivated by a separation of layers: the
@code{slot-value-using-class} family of functions is intended for use in
implementing different and new slot allocation strategies, rather than
in performing application-level dispatching.  Additionally, with this
requirement, there is a one-to-one mapping between metaclass, class and
slot-definition-class tuples and effective methods of @code{(setf
slot-value-using-class)}, which permits optimization of @code{(setf
slot-value-using-class)}'s discriminating function in the same manner as
for @code{slot-value-using-class} and @code{slot-boundp-using-class}.

Note that application code may specialize on the @code{new-value}
argument of slot accessors.

@item
@findex @cl{defclass}
@findex @sbmop{ensure-class}
@findex @sbmop{ensure-class-using-class}
@findex @cl{find-class}
@findex @cl{class-name}
the class named by the @code{name} argument to @code{ensure-class}, if
any, is only redefined if it is the proper name of that class;
otherwise, a new class is created.

This is consistent with the description of @code{ensure-class} in AMOP
as the functional version of @code{defclass}, which has this behaviour;
however, it is not consistent with the weaker requirement in AMOP, which
states that any class found by @code{find-class}, no matter what its
@code{class-name}, is redefined.

@end itemize

@subsection Metaobject Protocol Extensions

In addition, SBCL supports extensions to the Metaobject protocol from
AMOP; at present, they are:

@itemize

@item
@findex @cl{defmethod}
@findex @cl{find-class}
@findex @sbmop{intern-eql-specializer}
@findex @sbpcl{make-method-specializers-form}
@findex @sbmop{make-method-lambda}
compile-time support for generating specializer metaobjects from
specializer names in @code{defmethod} forms is provided by the
@code{make-method-specializers-form} function, which returns a form
which, when evaluated in the lexical environment of the
@code{defmethod}, returns a list of specializer metaobjects.  This
operator suffers from similar restrictions to those affecting
@code{make-method-lambda}, namely that the generic function must be
defined when the @code{defmethod} form is expanded, so that the
correct method of @code{make-method-specializers-form} is invoked.
The system-provided method on @code{make-method-specializers-form}
generates a call to @code{find-class} for each symbol specializer
name, and a call to @code{intern-eql-specializer} for each @code{(eql
@var{x})} specializer name.

@item
@findex @cl{find-method}
@findex @sbpcl{parse-specializer-using-class}
@findex @sbpcl{unparse-specializer-using-class}
run-time support for converting between specializer names and
specializer metaobjects, mostly for the purposes of
@code{find-method}, is provided by
@code{parse-specializer-using-class} and
@code{unparse-specializer-using-class}, which dispatch on their first
argument, the generic function associated with a method with the given
specializer.  The system-provided methods on those methods convert
between classes and proper names and between lists of the form
@code{(eql @var{x})} and interned eql specializer objects.

@item
@vindex @sbpcl{+slot-unbound+}
@findex @sbmop{standard-instance-access}
@findex @sbmop{funcallable-standard-instance-access}
distinguising unbound instance allocated slots from bound ones when
using @code{standard-instance-access} and
@code{funcallable-standard-instance-access} is possible by comparison
to the constant @code{+slot-unbound+}.

@end itemize

@node  Support For Unix
@comment  node-name,  next,  previous,  up
@section Support For Unix

@menu
* Command-line arguments::
* Querying the process environment::
* Running external programs::
@end menu

@node Command-line arguments
@subsection Command-line arguments
@vindex @sbext{@earmuffs{posix-argv}}

The UNIX command line can be read from the variable
@code{sb-ext:*posix-argv*}.

@node Querying the process environment
@subsection Querying the process environment

The UNIX environment can be queried with the
@code{sb-ext:posix-getenv} function.

@include fun-sb-ext-posix-getenv.texinfo

@node Running external programs
@subsection Running external programs

External programs can be run with @code{sb-ext:run-program}.
@footnote{In SBCL versions prior to 1.0.13, @code{sb-ext:run-program}
searched for executables in a manner somewhat incompatible with other
languages.  As of this version, SBCL uses the system library routine
@code{execvp(3)}, and no longer contains the function,
@code{find-executable-in-search-path}, which implemented the old
search.  Users who need this function may find it
in @file{run-program.lisp} versions 1.67 and earlier in SBCL's CVS
repository here
@url{http://sbcl.cvs.sourceforge.net/sbcl/sbcl/src/code/run-program.lisp?view=log}. However,
we caution such users that this search routine finds executables that
system library routines do not.}

@include fun-sb-ext-run-program.texinfo

When @code{sb-ext:run-program} is called with @code{wait} equal to
NIL, an instance of class @var{sb-ext:process} is returned.  The
following functions are available for use with processes:

@include fun-sb-ext-process-p.texinfo

@include fun-sb-ext-process-input.texinfo

@include fun-sb-ext-process-output.texinfo

@include fun-sb-ext-process-error.texinfo

@include fun-sb-ext-process-alive-p.texinfo

@include fun-sb-ext-process-status.texinfo

@include fun-sb-ext-process-wait.texinfo

@include fun-sb-ext-process-exit-code.texinfo

@include fun-sb-ext-process-core-dumped.texinfo

@include fun-sb-ext-process-close.texinfo

@include fun-sb-ext-process-kill.texinfo

@node  Customization Hooks for Users
@comment  node-name,  next,  previous,  up
@section Customization Hooks for Users

The toplevel repl prompt may be customized, and the function
that reads user input may be replaced completely.
@c <!-- FIXME but I don't currently remember how -->

The behaviour of @code{require} when called with only one argument is
implementation-defined.  In SBCL, @code{require} behaves in the
following way:

@include fun-common-lisp-require.texinfo
@include var-sb-ext-star-module-provider-functions-star.texinfo

Although SBCL does not provide a resident editor, the @code{ed}
function can be customized to hook into user-provided editing
mechanisms as follows:

@include fun-common-lisp-ed.texinfo
@include var-sb-ext-star-ed-functions-star.texinfo

Conditions of type @code{warning} and @code{style-warning} are
sometimes signaled at runtime, especially during execution of Common
Lisp defining forms such as @code{defun}, @code{defmethod}, etc.  To
muffle these warnings at runtime, SBCL provides a variable
@code{sb-ext:*muffled-warnings*}:

@include var-sb-ext-star-muffled-warnings-star.texinfo

@node Tools To Help Developers
@comment  node-name,  next,  previous,  up
@section Tools To Help Developers
@findex @cl{trace}
@findex @cl{inspect}

SBCL provides a profiler and other extensions to the ANSI @code{trace}
facility.  For more information, see @ref{Macro common-lisp:trace}.

The debugger supports a number of options. Its documentation is
accessed by typing @kbd{help} at the debugger prompt. @xref{Debugger}.

Documentation for @code{inspect} is accessed by typing @kbd{help} at
the @code{inspect} prompt.

@node Resolution of Name Conflicts
@section Resolution of Name Conflicts
@tindex @sbext{name-conflict}
@findex @sbext{name-conflict-symbols}

The ANSI standard (section 11.1.1.2.5) requires that name conflicts in
packages be resolvable in favour of any of the conflicting symbols.  In
the interactive debugger, this is achieved by prompting for the symbol
in whose favour the conflict should be resolved; for programmatic use,
the @code{sb-ext:resolve-conflict} restart should be invoked with one
argument, which should be a member of the list returned by the condition
accessor @code{sb-ext:name-conflict-symbols}.

@node    Hash Table Extensions
@comment  node-name,  next,  previous,  up
@section Hash Table Extensions
@cindex Hash tables

Hash table extensions supported by SBCL are all controlled by keyword
arguments to @code{make-hash-table}.

@include fun-common-lisp-make-hash-table.texinfo

@include macro-sb-ext-define-hash-table-test.texinfo

@include macro-sb-ext-with-locked-hash-table.texinfo

@include fun-sb-ext-hash-table-synchronized-p.texinfo

@include fun-sb-ext-hash-table-weakness.texinfo

@node    Miscellaneous Extensions
@comment  node-name,  next,  previous,  up
@section Miscellaneous Extensions

@include fun-sb-ext-array-storage-vector.texinfo
@include fun-sb-ext-delete-directory.texinfo
@include fun-sb-ext-get-time-of-day.texinfo
@include fun-sb-ext-seed-random-state.texinfo
@include macro-sb-ext-wait-for.texinfo

@node Stale Extensions
@comment  node-name,  next,  previous,  up
@section Stale Extensions

SBCL has inherited from CMUCL various hooks to allow the user to
tweak and monitor the garbage collection process. These are somewhat
stale code, and their interface might need to be cleaned up. If you
have urgent need of them, look at the code in @file{src/code/gc.lisp}
and bring it up on the developers' mailing list.

SBCL has various hooks inherited from CMUCL, like
@code{sb-ext:float-denormalized-p}, to allow a program to take
advantage of IEEE floating point arithmetic properties which aren't
conveniently or efficiently expressible using the ANSI standard. These
look good, and their interface looks good, but IEEE support is
slightly broken due to a stupid decision to remove some support for
infinities (because it wasn't in the ANSI spec and it didn't occur to
me that it was in the IEEE spec). If you need this stuff, take a look
at the code and bring it up on the developers' mailing
list.


@node  Efficiency Hacks
@comment  node-name,  next,  previous,  up
@section Efficiency Hacks

The @code{sb-ext:purify} function causes SBCL first to collect all
garbage, then to mark all uncollected objects as permanent, never again
attempting to collect them as garbage. This can cause a large increase
in efficiency when using a primitive garbage collector, or a more
moderate increase in efficiency when using a more sophisticated garbage
collector which is well suited to the program's memory usage pattern. It
also allows permanent code to be frozen at fixed addresses, a
precondition for using copy-on-write to share code between multiple Lisp
processes.  This is less important with modern generational garbage
collectors, but not all SBCL platforms use such a garbage collector.

@include fun-sb-ext-purify.texinfo

The @code{sb-ext:truly-the} special form declares the type of the
result of the operations, producing its argument; the declaration is
not checked. In short: don't use it.

@include special-operator-sb-ext-truly-the.texinfo

The @code{sb-ext:freeze-type} declaration declares that a
type will never change, which can make type testing
(@code{typep}, etc.) more efficient for structure types.