+
+<para>
+The foreign function call interface allows a Lisp program to call
+many functions written in languages that use the C calling convention.
+</para>
+
+<para>
+Lisp sets up various signal handling routines and other environment
+information when it first starts up, and expects these to be in place
+at all times. The C functions called by Lisp should not change the
+environment, especially the signal handlers: the signal handlers
+installed by Lisp typically have interesting flags set (e.g to request
+machine context information, or for signal delivery on an alternate
+stack) which the Lisp runtime relies on for correct operation.
+Precise details of how this works may change without notice between
+versions; the source, or the brain of a friendly &SBCL; developer,
+is the only documentation. Users of a Lisp built with the :sb-thread
+feature should also read the Threading section
+<!-- FIXME I'm sure docbook has some syntax for internal links -->
+of this manual</para>
+
+<sect2><title>The <function>alien-funcall</> Primitive</title>
+
+<synopsis>(sb-alien:alien-funcall alien-function &rest arguments)</>
+
+<para>
+The <function>alien-funcall</> function is the foreign function call
+primitive: <varname>alien-function</> is called with the supplied
+<varname>arguments</> and its C return value is returned as a Lisp value.
+The <varname>alien-function</> is an arbitrary
+run-time expression; to refer to a constant function, use
+<function>extern-alien</> or a value defined by
+<function>define-alien-routine</>.
+</para>
+
+<para>
+The type of <function>alien-function</>
+must be <type>(alien (function ...))</>
+or <type>(alien (* (function ...)))</>.
+The function type is used to
+determine how to call the function (as though it was declared with
+a prototype.) The type need not be known at compile time, but only
+known-type calls are efficiently compiled. Limitations:
+<itemizedlist>
+ <listitem><para>Structure type return values are not implemented.</></>
+ <listitem><para>Passing of structures by value is not implemented.</></>
+</itemizedlist>
+
+<para>
+Here is an example which allocates a <type>(struct foo)</>, calls a foreign
+function to initialize it, then returns a Lisp vector of all the
+<type>(* (struct foo))</> objects filled in by the foreign call:
+<programlisting>
+;; Allocate a foo on the stack.
+(with-alien ((f (struct foo)))
+ ;; Call some C function to fill in foo fields.
+ (alien-funcall (extern-alien "mangle_foo" (function void (* foo)))
+ (addr f))
+ ;; Find how many foos to use by getting the A field.
+ (let* ((num (slot f 'a))
+ (result (make-array num)))
+ ;; Get a pointer to the array so that we don't have to keep extracting it:
+ (with-alien ((a (* (array (* (struct foo)) 100)) (addr (slot f 'b))))
+ ;; Loop over the first N elements and stash them in the result vector.
+ (dotimes (i num)
+ (setf (svref result i) (deref (deref a) i)))
+ ;; Voila.
+ result)))</programlisting>
+</para>
+
+</sect2>
+
+<sect2><title>The <function>define-alien-routine</> Macro</>
+
+<synopsis>(sb-alien:define-alien-routine} name result-type &rest arg-specifiers)</>
+
+<para>
+The <function>define-alien-routine</> macro is a convenience
+for automatically generating Lisp
+interfaces to simple foreign functions. The primary feature is the
+parameter style specification, which translates the C
+pass-by-reference idiom into additional return values.
+</para>
+
+<para>
+<varname>name</> is usually a string external symbol, but may also be a
+symbol Lisp name or a list of the foreign name and the Lisp name.
+If only one name is specified, the other is automatically derived
+as for <function>extern-alien</>.
+<varname>result-type</> is the alien type of the return value.
+</para>
+
+<para>
+Each element of the <varname>arg-specifiers</> list
+specifies an argument to the foreign function, and is
+of the form
+<programlisting>(aname atype &optional style)</programlisting>
+<varname>aname</> is the symbol name of the argument to the constructed
+function (for documentation). <varname>atype</> is the alien type of
+corresponding foreign argument. The semantics of the actual call
+are the same as for <function>alien-funcall</>. <varname>style</>
+specifies how this argument should be handled at call and return time,
+and should be one of the following
+<itemizedlist>
+ <listitem><para><varname>:in</>specifies that the argument is
+ passed by value. This is the default. <varname>:in</> arguments
+ have no corresponding return value from the Lisp function.
+ </para></listitem>
+ <listitem><para><varname>:copy</> is similar to <varname>:in</>,
+ but the argument is copied
+ to a pre-allocated object and a pointer to this object is passed
+ to the foreign routine.</para></listitem>
+ <listitem><para><varname>:out</> specifies a pass-by-reference
+ output value. The type of the argument must be a pointer to
+ a fixed-sized object (such as an integer or pointer).
+ <varname>:out</> and <varname>:in-out</> style cannot
+ be used with pointers to arrays, records or functions. An
+ object of the correct size is allocated on the stack, and
+ its address is passed to the foreign function. When the
+ function returns, the contents
+ of this location are returned as one of the values of the Lisp
+ function (and the location is automatically deallocated).
+ </para></listitem>
+ <listitem><para><varname>:in-out</> is a combination of
+ <varname>:copy</> and <varname>:out</>.
+ The argument is copied to a pre-allocated object and a pointer to
+ this object is passed to the foreign routine. On return, the
+ contents of this location is returned as an additional value.
+ </para></listitem>
+</itemizedlist>
+</para>
+
+<note>
+<para>
+Any efficiency-critical foreign interface function should be inline
+expanded, which can be done by preceding the
+<function>define-alien-routine</> call with:
+<programlisting>(declaim (inline lisp-name))</programlisting>
+In addition to avoiding the Lisp call overhead, this allows
+pointers, word-integers and floats to be passed using non-descriptor
+representations, avoiding consing.)
+</para>
+</note>
+
+</sect2>
+
+<sect2><title><function>define-alien-routine</> Example</title>
+
+<para>
+Consider the C function <function>cfoo</>
+with the following calling convention:
+<programlisting>
+void
+cfoo (str, a, i)
+ char *str;
+ char *a; /* update */
+ int *i; /* out */
+{
+ /* body of cfoo(...) */
+}</programlisting>
+This can be described by the following call to
+<function>define-alien-routine</>:
+<programlisting>
+(define-alien-routine "cfoo" void
+ (str c-string)
+ (a char :in-out)
+ (i int :out))</programlisting>
+The Lisp function <function>cfoo</> will have
+two arguments (<varname>str</> and <varname>a</>)
+and two return values (<varname>a</> and <varname>i</>).
+
+</sect2>
+
+<sect2><title>Calling Lisp From C</>
+
+<para>
+Calling Lisp functions from C is sometimes possible, but is extremely
+hackish and poorly supported as of &SBCL; 0.7.5.
+See <function>funcall0</> ... <function>funcall3</> in
+the runtime system. The
+arguments must be valid &SBCL; object descriptors (so that
+e.g. fixnums must be
+left-shifted by 2.) As of &SBCL; 0.7.5, the format
+of object descriptors is documented only by the source code and, in parts,
+by the old &CMUCL; "INTERNALS" documentation.</para>
+
+<para> Note that the garbage collector moves objects, and won't be
+able to fix up any references in C variables. There are three
+mechanisms for coping with this:
+<orderedlist>
+
+<listitem><para>The <function>sb-ext:purify</> moves all live Lisp
+data into static or read-only areas such that it will never be moved
+(or freed) again in the life of the Lisp session</para></listitem>
+
+<listitem><para><function>sb-sys:with-pinned-objects</function> is a
+macro which arranges for some set of objects to be pinned in memory
+for the dynamic extent of its body forms. On ports which use the
+generational garbage collector (as of &SBCL; 0.8.3, only the x86) this
+has a page granularity - i.e. the entire 4k page or pages containing
+the objects will be locked down. On other ports it is implemented by
+turning off GC for the duration (so could be said to have a
+whole-world granularity). </para></listitem>
+
+<listitem><para>Disable GC, using the <function>without-gcing</function>
+macro or <function>gc-off</function> call.</para></listitem>
+</orderedlist>
+
+<!-- FIXME: This is a "changebar" section from the CMU CL manual.
+ I (WHN 2002-07-14) am not very familiar with this content, so
+ I'm not immediately prepared to try to update it for SBCL, and
+ I'm not feeling masochistic enough to work to encourage this
+ kind of low-level hack anyway. However, I acknowledge that callbacks
+ are sometimes really really necessary, so I include the original
+ text in case someone is hard-core enough to benefit from it. If
+ anyone brings the information up to date for SBCL, it belong
+ either in the main manual or on a CLiki SBCL Internals page.
+LaTeX \subsection{Accessing Lisp Arrays}
+LaTeX
+LaTeX Due to the way \cmucl{} manages memory, the amount of memory that can
+LaTeX be dynamically allocated by \code{malloc} or \funref{make-alien} is
+LaTeX limited\footnote{\cmucl{} mmaps a large piece of memory for it's own
+LaTeX use and this memory is typically about 8 MB above the start of the C
+LaTeX heap. Thus, only about 8 MB of memory can be dynamically
+LaTeX allocated.}.
+
+Empirically determined to be considerably >8Mb on this x86 linux
+machine, but I don't know what the actual values are - dan 2003.09.01
+
+Note that this technique is used in SB-GROVEL in the SBCL contrib
+
+LaTeX
+LaTeX To overcome this limitation, it is possible to access the content of
+LaTeX Lisp arrays which are limited only by the amount of physical memory
+LaTeX and swap space available. However, this technique is only useful if
+LaTeX the foreign function takes pointers to memory instead of allocating
+LaTeX memory for itself. In latter case, you will have to modify the
+LaTeX foreign functions.
+LaTeX
+LaTeX This technique takes advantage of the fact that \cmucl{} has
+LaTeX specialized array types (\pxlref{specialized-array-types}) that match
+LaTeX a typical C array. For example, a \code{(simple-array double-float
+LaTeX (100))} is stored in memory in essentially the same way as the C
+LaTeX array \code{double x[100]} would be. The following function allows us
+LaTeX to get the physical address of such a Lisp array:
+LaTeX \begin{example}
+LaTeX (defun array-data-address (array)
+LaTeX "Return the physical address of where the actual data of an array is
+LaTeX stored.
+LaTeX
+LaTeX ARRAY must be a specialized array type in CMU Lisp. This means ARRAY
+LaTeX must be an array of one of the following types:
+LaTeX
+LaTeX double-float
+LaTeX single-float
+LaTeX (unsigned-byte 32)
+LaTeX (unsigned-byte 16)
+LaTeX (unsigned-byte 8)
+LaTeX (signed-byte 32)
+LaTeX (signed-byte 16)
+LaTeX (signed-byte 8)
+LaTeX "
+LaTeX (declare (type (or #+signed-array (array (signed-byte 8))
+LaTeX #+signed-array (array (signed-byte 16))
+LaTeX #+signed-array (array (signed-byte 32))
+LaTeX (array (unsigned-byte 8))
+LaTeX (array (unsigned-byte 16))
+LaTeX (array (unsigned-byte 32))
+LaTeX (array single-float)
+LaTeX (array double-float))
+LaTeX array)
+LaTeX (optimize (speed 3) (safety 0))
+LaTeX (ext:optimize-interface (safety 3)))
+LaTeX ;; with-array-data will get us to the actual data. However, because
+LaTeX ;; the array could have been displaced, we need to know where the
+LaTeX ;; data starts.
+LaTeX (lisp::with-array-data ((data array)
+LaTeX (start)
+LaTeX (end))
+LaTeX (declare (ignore end))
+LaTeX ;; DATA is a specialized simple-array. Memory is laid out like this:
+LaTeX ;;
+LaTeX ;; byte offset Value
+LaTeX ;; 0 type code (should be 70 for double-float vector)
+LaTeX ;; 4 4 * number of elements in vector
+LaTeX ;; 8 1st element of vector
+LaTeX ;; ... ...
+LaTeX ;;
+LaTeX (let ((addr (+ 8 (logandc1 7 (kernel:get-lisp-obj-address data))))
+LaTeX (type-size (let ((type (array-element-type data)))
+LaTeX (cond ((or (equal type '(signed-byte 8))
+LaTeX (equal type '(unsigned-byte 8)))
+LaTeX 1)
+LaTeX ((or (equal type '(signed-byte 16))
+LaTeX (equal type '(unsigned-byte 16)))
+LaTeX 2)
+LaTeX ((or (equal type '(signed-byte 32))
+LaTeX (equal type '(unsigned-byte 32)))
+LaTeX 4)
+LaTeX ((equal type 'single-float)
+LaTeX 4)
+LaTeX ((equal type 'double-float)
+LaTeX 8)
+LaTeX (t
+LaTeX (error "Unknown specialized array element type"))))))
+LaTeX (declare (type (unsigned-byte 32) addr)
+LaTeX (optimize (speed 3) (safety 0) (ext:inhibit-warnings 3)))
+LaTeX (system:int-sap (the (unsigned-byte 32)
+LaTeX (+ addr (* type-size start)))))))
+LaTeX \end{example}
+LaTeX
+LaTeX Assume we have the C function below that we wish to use:
+LaTeX \begin{example}
+LaTeX double dotprod(double* x, double* y, int n)
+LaTeX \{
+LaTeX int k;
+LaTeX double sum = 0;
+LaTeX
+LaTeX for (k = 0; k < n; ++k) \{
+LaTeX sum += x[k] * y[k];
+LaTeX \}
+LaTeX \}
+LaTeX \end{example}
+LaTeX The following example generates two large arrays in Lisp, and calls the C
+LaTeX function to do the desired computation. This would not have been
+LaTeX possible using \code{malloc} or \code{make-alien} since we need about
+LaTeX 16 MB of memory to hold the two arrays.
+LaTeX \begin{example}
+LaTeX (define-alien-routine "dotprod" double
+LaTeX (x (* double-float) :in)
+LaTeX (y (* double-float) :in)
+LaTeX (n int :in))
+LaTeX
+LaTeX (let ((x (make-array 1000000 :element-type 'double-float))
+LaTeX (y (make-array 1000000 :element-type 'double-float)))
+LaTeX ;; Initialize X and Y somehow
+LaTeX (let ((x-addr (system:int-sap (array-data-address x)))
+LaTeX (y-addr (system:int-sap (array-data-address y))))
+LaTeX (dotprod x-addr y-addr 1000000)))
+LaTeX \end{example}
+LaTeX In this example, it may be useful to wrap the inner \code{let}
+LaTeX expression in an \code{unwind-protect} that first turns off garbage
+LaTeX collection and then turns garbage collection on afterwards. This will
+LaTeX prevent garbage collection from moving \code{x} and \code{y} after we
+LaTeX have obtained the (now erroneous) addresses but before the call to
+LaTeX \code{dotprod} is made.
+LaTeX
+-->
+