--- /dev/null
+<?xml version="1.0" encoding="iso-8859-1"?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
+"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" [
+<!ENTITY % myents SYSTEM "entities.inc">
+%myents;
+]>
+
+<chapter id="ffi"><title>The Foreign Function Interface</title>
+
+<para>This chapter describes &SBCL;'s interface to C programs and
+libraries (and, since C interfaces are a sort of <foreignphrase>lingua
+franca</foreignphrase> of the Unix world, to other programs and libraries in
+general.)</para>
+
+<note><para>In the modern Lisp world, the usual term for this
+functionality is Foreign Function Interface, or <acronym>FFI</acronym>, where
+despite the mention of <quote>function</quote> in this term, <acronym>FFI</acronym> also
+refers to direct manipulation of C data structures as well as
+functions. The traditional &CMUCL; terminology is Alien Interface, and
+while that older terminology is no longer used much in the system
+documentation, it still reflected in names in the
+implementation, notably in the name of the <literal>SB-ALIEN</literal>
+package.</para></note>
+
+<sect1 id="ffi-intro"><title>Introduction to the Foreign Function Interface</title>
+<!-- AKA "Introduction to Aliens" in the CMU CL manual -->
+
+<para>
+Because of Lisp's emphasis on dynamic memory allocation and garbage
+collection, Lisp implementations use non-C-like memory representations
+for objects. This representation mismatch creates friction when a Lisp
+program must share objects with programs which expect C data. There
+are three common approaches to establishing communication:
+<itemizedlist>
+ <listitem><para>The burden can be placed on the foreign program
+ (and programmer) by requiring the knowledge and use of the
+ representations used internally by the Lisp implementation.
+ This can require a considerable amount of <quote>glue</quote> code on the
+ C side, and that code tends to be sensitively dependent on the
+ internal implementation details of the Lisp system.</para></listitem>
+ <listitem><para>The Lisp system can automatically convert objects
+ back and forth between the Lisp and foreign representations.
+ This is convenient, but translation becomes prohibitively slow
+ when large or complex data structures must be shared. This approach
+ is supported by the &SBCL; <acronym>FFI</acronym>, and used automatically
+ by the when passing integers and strings.</para></listitem>
+ <listitem><para>The Lisp program can directly manipulate foreign
+ objects through the use of extensions to the Lisp language.
+ </para></listitem>
+</itemizedlist>
+</para>
+
+<para>&SBCL;, like &CMUCL; before it, relies primarily on the
+automatic conversion and direct manipulation approaches. The SB-ALIEN
+package provices a facility wherein foreign values of simple scalar
+types are automatically converted and complex types are directly
+manipulated in their foreign representation. Additionally the
+lower-level System Area Pointers (or SAPs) can be used where
+necessary to provide untyped access to foreign memory.</para>
+
+<para>Any foreign objects that can't automatically be converted into
+Lisp values are represented by objects of type <type>alien-value</type>.
+Since Lisp is a dynamically typed language, even foreign objects must
+have a run-time type; this type information is provided by
+encapsulating the raw pointer to the foreign data within an
+<type>alien-value</type> object.</para>
+
+<para>The type language and operations on foreign types are
+intentionally similar to those of the C language.</para>
+
+</sect1>
+
+<sect1 id="foreign-types"><title>Foreign Types</title>
+<!-- AKA "Alien Types" in the CMU CL manual -->
+
+<para>Alien types have a description language based on nested list
+structure. For example the C type
+<programlisting>struct foo {
+ int a;
+ struct foo *b[100];
+};</programlisting>
+has the corresponding &SBCL; FFI type
+<programlisting>(struct foo
+ (a int)
+ (b (array (* (struct foo)) 100)))</programlisting>
+</para>
+
+<sect2><title>Defining Foreign Types</title>
+
+<para>
+Types may be either named or anonymous. With structure and union
+types, the name is part of the type specifier, allowing recursively
+defined types such as:
+<programlisting>(struct foo (a (* (struct foo))))</programlisting>
+An anonymous structure or union type is specified by using the name
+<literal>nil</literal>. The <function>with-alien</function> macro defines a local
+scope which <quote>captures</quote> any named type definitions. Other types
+are not inherently named, but can be given named abbreviations using
+the <function>define-alien-type</function> macro.
+</para>
+
+</sect2>
+
+<sect2><title>Foreign Types and Lisp Types</title>
+
+<para>
+The foreign types form a subsystem of the &SBCL; type system. An
+<type>alien</type> type specifier provides a way to use any foreign type as a
+Lisp type specifier. For example,
+<programlisting>(typep foo '(alien (* int)))</programlisting>
+can be used to determine whether <varname>foo</varname> is a pointer to a foreign
+<type>int</type>. <type>alien</type> type specifiers can be used in the same ways
+as ordinary Lisp type specifiers (like <type>string</type>.) Alien type
+declarations are subject to the same
+precise type checking <!-- FIXME: should be linked to id="precisetypechecking" -->
+as any other declaration.
+</para>
+
+<para>
+Note that the type identifiers used in the
+foreign type system overlap with native Lisp type
+specifiers in some cases. For example, the type specifier
+<type>(alien single-float)</type> is identical to <type>single-float</type>, since
+foreign floats are automatically converted to Lisp floats. When
+<function>type-of</function> is called on an alien value that is not automatically
+converted to a Lisp value, then it will return an <type>alien</type> type
+specifier.
+</para>
+
+</sect2>
+
+<sect2><title>Foreign Type Specifiers</title>
+
+<note><para>
+All foreign type names are exported from the <literal>sb-alien</literal>
+package. Some foreign type names are also symbols in
+the <literal>common-lisp</literal> package, in which case they are
+reexported from the <literal>sb-alien</literal> package, so that
+e.g. it is legal to refer to <type>sb-alien:single-float</type>.
+</para></note>
+
+<para>
+These are the basic foreign type specifiers:
+<!-- FIXME: There must be some better way of formatting definitions
+ in DocBook than this. I haven't found it yet, but suggestions
+ or patches would be welcome. -->
+<itemizedlist>
+ <listitem>
+ <para>
+ The foreign type specifier <type>(* foo)</type> describes a
+ pointer to an object of type <type>foo</type>. A pointed-to type
+ <type>foo</type> of <type>t</type> indicates a pointer to anything,
+ similar to <type>void *</type> in ANSI C. A null alien pointer can
+ be detected with the <function>sb-alien:null-alien</function>
+ function.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>(array foo &optional dimensions)</type>
+ describes array of the specified <literal>dimensions</literal>, holding
+ elements of type <type>foo</type>. Note that (unlike in C) <type>(* foo)</type>
+ <type>(array foo)}</type> are considered to be different types when
+ type checking is done. If equivalence of pointer and array types
+ is desired, it may be explicitly coerced using
+ <function>sb-alien:cast</function>.
+ </para>
+ <para>
+ Arrays are accessed using <function>sb-alien:deref</function>, passing
+ the indices as additional arguments. Elements are stored in
+ column-major order (as in C), so the first dimension determines
+ only the size of the memory block, and not the layout of the
+ higher dimensions. An array whose first dimension is variable
+ may be specified by using <literal>nil</literal> as the first dimension.
+ Fixed-size arrays can be allocated as array elements, structure
+ slots or <function>sb-alien:with-alien</function> variables. Dynamic
+ arrays can only be allocated using <function>sb-alien:make-alien</function>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier
+ <type>(sb-alien:struct name &rest fields)</type>
+ describes a structure type with the specified <varname>name</varname> and
+ <varname>fields</varname>. Fields are allocated at the same offsets
+ used by the implementation's C compiler. If <varname>name</varname>
+ is <literal>nil</literal> then the structure is anonymous.
+ </para>
+ <para>
+ If a named foreign <type>struct</type> specifier is passed to
+ <function>define-alien-type</function> or <function>with-alien</function>,
+ then this defines, respectively, a new global or local foreign
+ structure type. If no <varname>fields</varname> are specified, then
+ the fields are taken from the current (local or global) alien
+ structure type definition of <varname>name</varname>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier
+ <type>(sb-alien:union name &rest fields)</type>
+ is similar to <type>sb-alien:struct</type>, but describes a union type.
+ All fields are allocated at the same offset, and the size of the
+ union is the size of the largest field. The programmer must
+ determine which field is active from context.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>(sb-alien:enum name &rest specs)</type>
+ describes an enumeration type that maps between integer values
+ and keywords. If <varname>name</varname> is <literal>nil</literal>, then the
+ type is anonymous. Each element of the <varname>specs</varname>
+ list is either a Lisp keyword, or a list <literal>(keyword value)</literal>.
+ <varname>value</varname> is an integer. If <varname>value</varname> is not
+ supplied, then it defaults to one greater than the value for
+ the preceding spec (or to zero if it is the first spec.)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>(sb-alien:signed &optional bits)</type>
+ specifies a signed integer with the specified number of
+ <varname>bits</varname> precision. The upper limit on integer
+ precision is determined by the machine's word
+ size. If <varname>bits</varname> is not specified, the maximum
+ size will be used.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>(integer &optional bits)</type> is
+ equivalent to the corresponding type specifier using
+ <type>sb-alien:signed</type> instead of <type>integer</type>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier
+ <type>(sb-alien:unsigned &optional bits)</type>
+ is like corresponding type specifier using <type>sb-alien:signed</type>
+ except that the variable is treated as an unsigned integer.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>(boolean &optional bits)</type> is
+ similar to an enumeration type, but maps from Lisp <literal>nil</literal>
+ and <literal>t</literal> to C <literal>0</literal> and <literal>1</literal>
+ respectively. <varname>bits</varname> determines the amount of
+ storage allocated to hold the truth value.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>single-float</type> describes a
+ floating-point number in IEEE single-precision format.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>double-float</type> describes a
+ floating-point number in IEEE double-precision format.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier
+ <type>(function result-type &rest arg-types)</type>
+ describes a foreign function that takes arguments of the specified
+ <varname>arg-types</varname> and returns a result of type <type>result-type</type>.
+ Note that the only context where a foreign <type>function</type> type
+ is directly specified is in the argument to
+ <function>sb-alien:alien-funcall</function>.
+ In all other contexts, foreign functions are represented by
+ foreign function pointer types: <type>(* (function ...))</type>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>sb-alien:system-area-pointer</type>
+ describes a pointer which is represented in Lisp as a
+ <type>system-area-pointer</type> object. &SBCL; exports this type from
+ <literal>sb-alien</literal> because &CMUCL; did, but tentatively (as of
+ the first draft of this section of the manual, &SBCL; 0.7.6) it is
+ deprecated, since it doesn't seem to be required by user code.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>sb-alien:void</type> is
+ used in function types to declare that no useful value
+ is returned. Using <function>alien-funcall</function>
+ to call a <type>void</type> foreign function will return
+ zero values.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>sb-alien:c-string</type>
+ is similar to <type>(* char)</type>, but is interpreted as a
+ null-terminated string, and is automatically converted into a
+ Lisp string when accessed; or if the pointer is C <literal>NULL</literal>
+ or <literal>0</literal>, then accessing it gives Lisp <literal>nil</literal>.
+ Lisp strings are stored with a trailing NUL termination, so no
+ copying (either by the user or the implementation) is necessary
+ when passing them to foreign code.
+ </para>
+ <para>
+ Assigning a Lisp string to a <type>c-string</type> structure field or
+ variable stores the contents of the string to the memory already
+ pointed to by that variable. When a foreign object of type
+ <type>(* char)</type> is assigned to a <type>c-string</type>, then the
+ <type>c-string</type> pointer is assigned to. This allows
+ <type>c-string</type> pointers to be initialized. For example:
+ <programlisting>(cl:in-package "CL-USER") ; which USEs package "SB-ALIEN"
+ (define-alien-type nil (struct foo (str c-string)))
+ (defun make-foo (str) (let ((my-foo (make-alien (struct foo))))
+ (setf (slot my-foo 'str) (make-alien char (length str))
+ (slot my-foo 'str) str) my-foo))</programlisting>
+ Storing Lisp <literal>NIL</literal> in a <type>c-string</type> writes C
+ <literal>NULL</literal> to the variable.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>sb-alien</literal> also exports translations of these C type
+ specifiers as foreign type specifiers:
+ <type>sb-alien:char</type>,
+ <type>sb-alien:short</type>,
+ <type>sb-alien:int</type>,
+ <type>sb-alien:long</type>,
+ <type>sb-alien:unsigned-char</type>,
+ <type>sb-alien:unsigned-short</type>,
+ <type>sb-alien:unsigned-int</type>,
+ <type>sb-alien:unsigned-long</type>,
+ <type>sb-alien:float</type>, and
+ <type>sb-alien:double</type>.
+ </para>
+ </listitem>
+
+</itemizedlist>
+
+</para>
+
+</sect2>
+
+</sect1>
+
+<sect1 id="foreign-values"><title>Operations On Foreign Values</title>
+<!-- AKA "Alien Operations" in the CMU CL manual -->
+
+<para>This section describes how to read foreign values as Lisp
+values, how to coerce foreign values to different kinds of foreign values, and
+how to dynamically allocate and free foreign variables.</para>
+
+<sect2><title>Accessing Foreign Values</title>
+
+<synopsis>(sb-alien:deref pointer-or-array &rest indices)</synopsis>
+
+<para>The <function>sb-alien:deref</function> function returns the value pointed to by
+a foreign pointer, or the value of a foreign array element. When
+dereferencing a pointer, an optional single index can be specified to
+give the equivalent of C pointer arithmetic; this index is scaled by
+the size of the type pointed to. When dereferencing an array, the
+number of indices must be the same as the number of dimensions in the
+array type. <function>deref</function> can be set with <function>setf</function> to
+assign a new value.</para>
+
+<synopsis>(sb-alien:slot struct-or-union &rest slot-names)</synopsis>
+
+<para>The <function>sb-alien:slot</function> function extracts the value of
+the slot named <varname>slot-name</varname> from a foreign <type>struct</type> or
+<type>union</type>. If <varname>struct-or-union</varname> is a pointer to a
+structure or union, then it is automatically dereferenced.
+<function>sb-alien:slot</function> can be set with <function>setf</function> to assign
+a new value. Note that <varname>slot-name</varname> is evaluated, and need
+not be a compile-time constant (but only constant slot accesses are
+efficiently compiled.)</para>
+
+<sect3><title>Untyped memory</title>
+
+<para>As noted at the beginning of the chapter, the System Area
+Pointer facilities allow untyped access to foreign memory. SAPs can
+be converted to and from the usual typed foreign values using
+<function>sap-alien</function> and <function>alien-sap</function>
+(described elsewhere), and also to and from integers - raw machine
+addresses. They should thus be used with caution; corrupting the Lisp
+heap or other memory with SAPs is trivial.</para>
+
+<synopsis>(sb-sys:int-sap machine-address)</synopsis>
+
+<para>Creates a SAP pointing at the virtual address
+<varname>machine-address</varname>. </para>
+
+<synopsis>(sb-sys:sap-ref-32 sap offset)</synopsis>
+
+<para>Access the value of the memory location at
+<varname>offset</varname> bytes from <varname>sap</varname>. This form
+may also be used with <function>setf</function> to alter the memory at
+that location.</para>
+
+<synopsis>(sb-sys:sap= sap1 sap2)</synopsis>
+
+<para>Compare <varname>sap1</varname> and <varname>sap2</varname> for
+equality.</para>
+
+<para>Similarly named functions exist for accessing other sizes of
+word, other comparisons, and other conversions. The reader is invited
+to use <function>apropos</function> and <function>describe</function>
+for more details</para>
+<programlisting>
+(apropos "sap" :sb-sys)
+</programlisting>
+</sect3></sect2>
+
+<sect2><title>Coercing Foreign Values</title>
+
+<synopsis>(sb-alien:addr alien-expr)</synopsis>
+
+<para>
+The <function>sb-alien:addr</function> macro
+returns a pointer to the location specified by
+<varname>alien-expr</varname>, which must be either a foreign variable, a use of
+<function>sb-alien:deref</function>, a use of <function>sb-alien:slot</function>, or a use of
+<function>sb-alien:extern-alien</function>.
+</para>
+
+<synopsis>(sb-alien:cast foreign-value new-type)</synopsis>
+
+<para>The <function>sb-alien:cast</function>
+converts <varname>foreign-value</varname> to a new foreign value with the specified
+<varname>new-type</varname>. Both types, old and new, must be foreign pointer,
+array or function types. Note that the resulting Lisp
+foreign variable object
+is not <function>eq</function> to the
+argument, but it does refer to the same foreign data bits.</para>
+
+<synopsis>(sb-alien:sap-alien sap type)</synopsis>
+
+<para>The <function>sb-alien:sap-alien</function> function converts <varname>sap</varname>
+(a system area pointer) to a foreign value with the specified
+<varname>type</varname>. <varname>type</varname> is not evaluated.
+</para>
+
+<para>The <varname>type</varname> must be some foreign pointer, array, or
+record type.</para>
+
+<synopsis>(sb-alien:alien-sap foreign-value type)</synopsis>
+
+<para>The <function>sb-alien:alien-sap</function> function
+returns the SAP which points to <varname>alien-value</varname>'s data.
+</para>
+
+<para>The <varname>foreign-value</varname> must be of some foreign pointer,
+array, or record type.</para>
+
+</sect2>
+
+<sect2><title>Foreign Dynamic Allocation</title>
+
+<para>Lisp code can call the C standard library functions
+<function>malloc</function> and <function>free</function> to dynamically allocate and
+deallocate foreign variables. The Lisp code shares the same allocator
+with foreign C code, so it's OK for foreign code to call
+<function>free</function> on the result of Lisp
+<function>sb-alien:make-alien</function>, or for Lisp code to call
+<function>sb-alien:free-alien</function> on foreign objects allocated by C
+code.</para>
+
+<synopsis>(sb-alien:make-alien type size)</synopsis>
+
+<para>The <function>sb-alien:make-alien</function> macro
+returns a dynamically allocated foreign value of the specified
+<varname>type</varname> (which is not evaluated.) The allocated memory is not
+initialized, and may contain arbitrary junk. If supplied,
+<varname>size</varname> is an expression to evaluate to compute the size of the
+allocated object. There are two major cases:
+<itemizedlist>
+ <listitem>
+ <para>When <varname>type</varname> is a foreign array type, an array of
+ that type is allocated and a pointer to it is returned. Note that you
+ must use <function>deref</function> to change the result to an array before you
+ can use <function>deref</function> to read or write elements:
+ <programlisting>
+ (cl:in-package "CL-USER") ; which USEs package "SB-ALIEN"
+ (defvar *foo* (make-alien (array char 10)))
+ (type-of *foo*) => (alien (* (array (signed 8) 10)))
+ (setf (deref (deref foo) 0) 10) => 10</programlisting>
+ If supplied, <varname>size</varname> is used as the first dimension for the
+ array.</para>
+ </listitem>
+ <listitem>
+ <para>When <varname>type</varname> is any other foreign type, then an
+ object for that type is allocated, and a pointer to it is
+ returned. So <function>(make-alien int)</function> returns a <type>(* int)</type>.
+ If <varname>size</varname> is specified, then a block of that many
+ objects is allocated, with the result pointing to the first one.</para>
+ </listitem>
+</itemizedlist>
+</para>
+
+<synopsis>(sb-alien:free-alien foreign-value)</synopsis>
+
+<para>The <function>sb-alien:free-alien</function> function
+frees the storage for <varname>foreign-value</varname>,
+which must have been allocated with Lisp <function>make-alien</function>
+or C <function>malloc</function>.</para>
+
+<para>See also the <function>sb-alien:with-alien</function> macro, which
+allocates foreign values on the stack.</para>
+
+</sect2>
+
+</sect1>
+
+<sect1 id="foreign-variables"><title>Foreign Variables</title>
+<!-- AKA "Alien Variables" in the CMU CL manual -->
+
+<para>
+Both local (stack allocated) and external (C global) foreign variables are
+supported.
+</para>
+
+<sect2><title>Local Foreign Variables</title>
+
+<synopsis>(sb-alien:with-alien var-definitions &body body)</synopsis>
+
+<para>The <function>with-alien</function>
+macro establishes local
+foreign variables
+with the specified
+alien types and names.
+This form is analogous to defining a local variable in C: additional
+storage is allocated, and the initial value is copied.
+This form is less
+analogous to LET-allocated Lisp variables, since the variables
+can't be captured in closures: they live only for the dynamic extent
+of the body, and referring to them outside is a gruesome error.
+</para>
+
+<para>The <varname>var-definitions</varname> argument is a list of
+variable definitions, each of the form
+<programlisting>(name type &optional initial-value)</programlisting>
+The names of the variables are established as symbol-macros; the bindings have
+lexical scope, and may be assigned with <function>setq</function>
+or <function>setf</function>.
+</para>
+
+<para>The <function>with-alien</function> macro also establishes
+a new scope for named structures
+and unions. Any <varname>type</varname> specified for a variable may contain
+named structure or union types with the slots specified. Within the
+lexical scope of the binding specifiers and body, a locally defined
+foreign structure type <type>foo</type> can be referenced by its name using
+<type>(struct foo)</type>.
+</para>
+
+</sect2>
+
+<sect2><title>External Foreign Variables</title>
+
+<para>
+External foreign names are strings, and Lisp names are symbols. When
+an external foreign value is represented using a Lisp variable, there
+must be a way to convert from one name syntax into the other. The
+macros <function>extern-alien</function>, <function>define-alien-variable</function> and
+<function>define-alien-routine</function> use this conversion heuristic:
+<itemizedlist>
+ <listitem><para>Alien names are converted to Lisp names by uppercasing and
+ replacing underscores with hyphens.</para></listitem>
+ <listitem><para>Conversely, Lisp names are converted to alien names by
+ lowercasing and replacing hyphens with underscores.</para></listitem>
+ <listitem><para>Both the Lisp symbol and alien string names may be
+ separately specified by using a list of the form
+ <programlisting>(alien-string lisp-symbol)</programlisting></para></listitem>
+</itemizedlist>
+</para>
+
+<synopsis>(sb-alien:define-alien-variable name type)</synopsis>
+
+<para>
+The <function>define-alien-variable</function> macro
+defines <varname>name</varname> as an external foreign variable of the
+specified foreign <type>type</type>. <varname>name</varname> and <type>type</type> are not
+evaluated. The Lisp name of the variable (see above) becomes a
+global alien variable. Global alien variables
+are effectively ``global symbol macros''; a reference to the
+variable fetches the contents of the external variable. Similarly,
+setting the variable stores new contents---the new contents must be
+of the declared <type>type</type>. Someday, they may well be implemented
+using the &ANSI; <function>define-symbol-macro</function> mechanism, but
+as of &SBCL; 0.7.5, they are still implemented using an older
+more-or-less parallel mechanism inherited from &CMUCL;.
+</para>
+
+<para>
+For example, to access a C-level counter <varname>foo</varname>, one could
+write
+<programlisting>
+(define-alien-variable "foo" int)
+;; Now it is possible to get the value of the C variable foo simply by
+;; referencing that Lisp variable:
+(print foo)
+(setf foo 14)
+(incf foo)</programlisting>
+</para>
+
+<synopsis>(sb-alien:get-errno)</synopsis>
+
+<para>
+Since in modern C libraries, the <varname>errno</varname> "variable" is typically
+no longer a variable, but some bizarre artificial construct
+which behaves superficially like a variable within a given thread,
+it can no longer reliably be accessed through the ordinary
+<varname>define-alien-variable</varname> mechanism. Instead, &SBCL; provides
+the operator <function>sb-alien:get-errno</function> to allow Lisp code to read it.
+</para>
+
+<synopsis>(sb-alien:extern-alien name type)</synopsis>
+
+<para>
+The <function>extern-alien</function> macro
+returns an alien with the specified <type>type</type> which
+points to an externally defined value. <varname>name</varname> is not evaluated,
+and may be either a string or a symbol. <type>type</type> is
+an unevaluated alien type specifier.
+</para>
+
+</sect2>
+
+</sect1>
+
+<sect1 id="foreign-data-structure"><title>Foreign Data Structure Examples</title>
+<!-- AKA "Alien Data Structure Example" in the CMU CL manual -->
+
+<para>
+Now that we have alien types, operations and variables, we can manipulate
+foreign data structures. This C declaration
+<programlisting>
+struct foo {
+ int a;
+ struct foo *b[100];
+};</programlisting>
+can be translated into the following alien type:
+<programlisting>(define-alien-type nil
+ (struct foo
+ (a int)
+ (b (array (* (struct foo)) 100))))</programlisting>
+</para>
+
+<para>
+Once the <type>foo</type> alien type has been defined as above,
+the C expression
+<programlisting>
+struct foo f;
+f.b[7].a</programlisting>
+can be translated in this way:
+<programlisting>
+(with-alien ((f (struct foo)))
+ (slot (deref (slot f 'b) 7) 'a)
+ ;;
+ ;; Do something with f...
+ )</programlisting>
+</para>
+
+<para>
+Or consider this example of an external C variable and some accesses:
+<programlisting>
+struct c_struct {
+ short x, y;
+ char a, b;
+ int z;
+ c_struct *n;
+};
+extern struct c_struct *my_struct;
+my_struct->x++;
+my_struct->a = 5;
+my_struct = my_struct->n;</programlisting>
+which can be manipulated in Lisp like this:
+<programlisting>
+(define-alien-type nil
+ (struct c-struct
+ (x short)
+ (y short)
+ (a char)
+ (b char)
+ (z int)
+ (n (* c-struct))))
+(define-alien-variable "my_struct" (* c-struct))
+(incf (slot my-struct 'x))
+(setf (slot my-struct 'a) 5)
+(setq my-struct (slot my-struct 'n))</programlisting>
+</para>
+
+</sect1>
+
+<sect1 id="load-object"><title>Loading Unix Object Files</title>
+
+<para>
+Foreign object files can be loaded into the running Lisp process by
+calling the functions <function>load-foreign</function> or
+<function>load-1-foreign</function>.
+</para>
+
+<para> The <function>sb-alien:load-1-foreign</function> function is the more
+primitive of the two operations. It loads a single object file. into
+the currently running Lisp. The external symbols defining routines and
+variables are made available for future external references (e.g. by
+<function>extern-alien</function>). Forward references to foreign symbols
+aren't supported: <function>load-1-foreign</function> must be run before any
+of the defined symbols are referenced.
+</para>
+
+<para><function>sb-alien:load-foreign</function> is built in terms of
+<function>load-1-foreign</function> and some other machinery
+like <function>sb-ext:run-program</function>.
+It accepts a list of files and libraries,
+and runs the linker on the files and
+libraries, creating an absolute Unix object file which is then
+processed by <function>load-1-foreign</function>.</para>
+
+<note><para>As of &SBCL; 0.7.5, all foreign code (code loaded
+with <function>load-1-function</function> or <function>load-function</function>) is
+lost when a Lisp core is saved with
+<function>sb-ext:save-lisp-and-die</function>, and no attempt is made to
+restore it when the core is loaded. Historically this has been an
+annoyance both for &SBCL; users and for &CMUCL; users.
+It's hard to solve this problem completely cleanly, but some
+generally-reliable partial solution might be useful. Once someone in
+either camp gets sufficiently annoyed to create it, &SBCL; is
+likely to adopt some mechanism for automatically restoring foreign
+code when a saved core is loaded.</para></note>
+
+</sect1>
+
+<sect1 id="foreign-function-calls"><title>Foreign Function Calls</title>
+
+<para>
+The foreign function call interface allows a Lisp program to call
+many functions written in languages that use the C calling convention.
+</para>
+
+<para>
+Lisp sets up various signal handling routines and other environment
+information when it first starts up, and expects these to be in place
+at all times. The C functions called by Lisp should not change the
+environment, especially the signal handlers: the signal handlers
+installed by Lisp typically have interesting flags set (e.g to request
+machine context information, or for signal delivery on an alternate
+stack) which the Lisp runtime relies on for correct operation.
+Precise details of how this works may change without notice between
+versions; the source, or the brain of a friendly &SBCL; developer,
+is the only documentation. Users of a Lisp built with the :sb-thread
+feature should also read the Threading section
+<!-- FIXME I'm sure docbook has some syntax for internal links -->
+of this manual</para>
+
+<sect2><title>The <function>alien-funcall</function> Primitive</title>
+
+<synopsis>(sb-alien:alien-funcall alien-function &rest arguments)</synopsis>
+
+<para>
+The <function>alien-funcall</function> function is the foreign function call
+primitive: <varname>alien-function</varname> is called with the supplied
+<varname>arguments</varname> and its C return value is returned as a Lisp value.
+The <varname>alien-function</varname> is an arbitrary
+run-time expression; to refer to a constant function, use
+<function>extern-alien</function> or a value defined by
+<function>define-alien-routine</function>.
+</para>
+
+<para>
+The type of <function>alien-function</function>
+must be <type>(alien (function ...))</type>
+or <type>(alien (* (function ...)))</type>.
+The function type is used to
+determine how to call the function (as though it was declared with
+a prototype.) The type need not be known at compile time, but only
+known-type calls are efficiently compiled. Limitations:
+<itemizedlist>
+ <listitem><para>Structure type return values are not implemented.</para></listitem>
+ <listitem><para>Passing of structures by value is not implemented.</para></listitem>
+</itemizedlist>
+</para>
+
+<para>
+Here is an example which allocates a <type>(struct foo)</type>, calls a foreign
+function to initialize it, then returns a Lisp vector of all the
+<type>(* (struct foo))</type> objects filled in by the foreign call:
+<programlisting>
+;; Allocate a foo on the stack.
+(with-alien ((f (struct foo)))
+ ;; Call some C function to fill in foo fields.
+ (alien-funcall (extern-alien "mangle_foo" (function void (* foo)))
+ (addr f))
+ ;; Find how many foos to use by getting the A field.
+ (let* ((num (slot f 'a))
+ (result (make-array num)))
+ ;; Get a pointer to the array so that we don't have to keep extracting it:
+ (with-alien ((a (* (array (* (struct foo)) 100)) (addr (slot f 'b))))
+ ;; Loop over the first N elements and stash them in the result vector.
+ (dotimes (i num)
+ (setf (svref result i) (deref (deref a) i)))
+ ;; Voila.
+ result)))</programlisting>
+</para>
+
+</sect2>
+
+<sect2><title>The <function>define-alien-routine</function> Macro</title>
+
+<synopsis>(sb-alien:define-alien-routine} name result-type &rest arg-specifiers)</synopsis>
+
+<para>
+The <function>define-alien-routine</function> macro is a convenience
+for automatically generating Lisp
+interfaces to simple foreign functions. The primary feature is the
+parameter style specification, which translates the C
+pass-by-reference idiom into additional return values.
+</para>
+
+<para>
+<varname>name</varname> is usually a string external symbol, but may also be a
+symbol Lisp name or a list of the foreign name and the Lisp name.
+If only one name is specified, the other is automatically derived
+as for <function>extern-alien</function>.
+<varname>result-type</varname> is the alien type of the return value.
+</para>
+
+<para>
+Each element of the <varname>arg-specifiers</varname> list
+specifies an argument to the foreign function, and is
+of the form
+<programlisting>(aname atype &optional style)</programlisting>
+<varname>aname</varname> is the symbol name of the argument to the constructed
+function (for documentation). <varname>atype</varname> is the alien type of
+corresponding foreign argument. The semantics of the actual call
+are the same as for <function>alien-funcall</function>. <varname>style</varname>
+specifies how this argument should be handled at call and return time,
+and should be one of the following
+<itemizedlist>
+ <listitem><para><varname>:in</varname>specifies that the argument is
+ passed by value. This is the default. <varname>:in</varname> arguments
+ have no corresponding return value from the Lisp function.
+ </para></listitem>
+ <listitem><para><varname>:copy</varname> is similar to <varname>:in</varname>,
+ but the argument is copied
+ to a pre-allocated object and a pointer to this object is passed
+ to the foreign routine.</para></listitem>
+ <listitem><para><varname>:out</varname> specifies a pass-by-reference
+ output value. The type of the argument must be a pointer to
+ a fixed-sized object (such as an integer or pointer).
+ <varname>:out</varname> and <varname>:in-out</varname> style cannot
+ be used with pointers to arrays, records or functions. An
+ object of the correct size is allocated on the stack, and
+ its address is passed to the foreign function. When the
+ function returns, the contents
+ of this location are returned as one of the values of the Lisp
+ function (and the location is automatically deallocated).
+ </para></listitem>
+ <listitem><para><varname>:in-out</varname> is a combination of
+ <varname>:copy</varname> and <varname>:out</varname>.
+ The argument is copied to a pre-allocated object and a pointer to
+ this object is passed to the foreign routine. On return, the
+ contents of this location is returned as an additional value.
+ </para></listitem>
+</itemizedlist>
+</para>
+
+<note>
+<para>
+Any efficiency-critical foreign interface function should be inline
+expanded, which can be done by preceding the
+<function>define-alien-routine</function> call with:
+<programlisting>(declaim (inline lisp-name))</programlisting>
+In addition to avoiding the Lisp call overhead, this allows
+pointers, word-integers and floats to be passed using non-descriptor
+representations, avoiding consing.)
+</para>
+</note>
+
+</sect2>
+
+<sect2><title><function>define-alien-routine</function> Example</title>
+
+<para>
+Consider the C function <function>cfoo</function>
+with the following calling convention:
+<programlisting>
+void
+cfoo (str, a, i)
+ char *str;
+ char *a; /* update */
+ int *i; /* out */
+{
+ /* body of cfoo(...) */
+}</programlisting>
+This can be described by the following call to
+<function>define-alien-routine</function>:
+<programlisting>
+(define-alien-routine "cfoo" void
+ (str c-string)
+ (a char :in-out)
+ (i int :out))</programlisting>
+The Lisp function <function>cfoo</function> will have
+two arguments (<varname>str</varname> and <varname>a</varname>)
+and two return values (<varname>a</varname> and <varname>i</varname>).
+</para>
+
+</sect2>
+
+<sect2><title>Calling Lisp From C</title>
+
+<para>
+Calling Lisp functions from C is sometimes possible, but is extremely
+hackish and poorly supported as of &SBCL; 0.7.5.
+See <function>funcall0</function> ... <function>funcall3</function> in
+the runtime system. The
+arguments must be valid &SBCL; object descriptors (so that
+e.g. fixnums must be
+left-shifted by 2.) As of &SBCL; 0.7.5, the format
+of object descriptors is documented only by the source code and, in parts,
+by the old &CMUCL; "INTERNALS" documentation.</para>
+
+<para> Note that the garbage collector moves objects, and won't be
+able to fix up any references in C variables. There are three
+mechanisms for coping with this:
+<orderedlist>
+
+<listitem><para>The <function>sb-ext:purify</function> moves all live Lisp
+data into static or read-only areas such that it will never be moved
+(or freed) again in the life of the Lisp session</para></listitem>
+
+<listitem><para><function>sb-sys:with-pinned-objects</function> is a
+macro which arranges for some set of objects to be pinned in memory
+for the dynamic extent of its body forms. On ports which use the
+generational garbage collector (as of &SBCL; 0.8.3, only the x86) this
+has a page granularity - i.e. the entire 4k page or pages containing
+the objects will be locked down. On other ports it is implemented by
+turning off GC for the duration (so could be said to have a
+whole-world granularity). </para></listitem>
+
+<listitem><para>Disable GC, using the <function>without-gcing</function>
+macro or <function>gc-off</function> call.</para></listitem>
+</orderedlist>
+
+</para>
+
+<!-- FIXME: This is a "changebar" section from the CMU CL manual.
+ I (WHN 2002-07-14) am not very familiar with this content, so
+ I'm not immediately prepared to try to update it for SBCL, and
+ I'm not feeling masochistic enough to work to encourage this
+ kind of low-level hack anyway. However, I acknowledge that callbacks
+ are sometimes really really necessary, so I include the original
+ text in case someone is hard-core enough to benefit from it. If
+ anyone brings the information up to date for SBCL, it belong
+ either in the main manual or on a CLiki SBCL Internals page.
+LaTeX \subsection{Accessing Lisp Arrays}
+LaTeX
+LaTeX Due to the way \cmucl{} manages memory, the amount of memory that can
+LaTeX be dynamically allocated by \code{malloc} or \funref{make-alien} is
+LaTeX limited\footnote{\cmucl{} mmaps a large piece of memory for it's own
+LaTeX use and this memory is typically about 8 MB above the start of the C
+LaTeX heap. Thus, only about 8 MB of memory can be dynamically
+LaTeX allocated.}.
+
+Empirically determined to be considerably >8Mb on this x86 linux
+machine, but I don't know what the actual values are - dan 2003.09.01
+
+Note that this technique is used in SB-GROVEL in the SBCL contrib
+
+LaTeX
+LaTeX To overcome this limitation, it is possible to access the content of
+LaTeX Lisp arrays which are limited only by the amount of physical memory
+LaTeX and swap space available. However, this technique is only useful if
+LaTeX the foreign function takes pointers to memory instead of allocating
+LaTeX memory for itself. In latter case, you will have to modify the
+LaTeX foreign functions.
+LaTeX
+LaTeX This technique takes advantage of the fact that \cmucl{} has
+LaTeX specialized array types (\pxlref{specialized-array-types}) that match
+LaTeX a typical C array. For example, a \code{(simple-array double-float
+LaTeX (100))} is stored in memory in essentially the same way as the C
+LaTeX array \code{double x[100]} would be. The following function allows us
+LaTeX to get the physical address of such a Lisp array:
+LaTeX \begin{example}
+LaTeX (defun array-data-address (array)
+LaTeX "Return the physical address of where the actual data of an array is
+LaTeX stored.
+LaTeX
+LaTeX ARRAY must be a specialized array type in CMU Lisp. This means ARRAY
+LaTeX must be an array of one of the following types:
+LaTeX
+LaTeX double-float
+LaTeX single-float
+LaTeX (unsigned-byte 32)
+LaTeX (unsigned-byte 16)
+LaTeX (unsigned-byte 8)
+LaTeX (signed-byte 32)
+LaTeX (signed-byte 16)
+LaTeX (signed-byte 8)
+LaTeX "
+LaTeX (declare (type (or #+signed-array (array (signed-byte 8))
+LaTeX #+signed-array (array (signed-byte 16))
+LaTeX #+signed-array (array (signed-byte 32))
+LaTeX (array (unsigned-byte 8))
+LaTeX (array (unsigned-byte 16))
+LaTeX (array (unsigned-byte 32))
+LaTeX (array single-float)
+LaTeX (array double-float))
+LaTeX array)
+LaTeX (optimize (speed 3) (safety 0))
+LaTeX (ext:optimize-interface (safety 3)))
+LaTeX ;; with-array-data will get us to the actual data. However, because
+LaTeX ;; the array could have been displaced, we need to know where the
+LaTeX ;; data starts.
+LaTeX (lisp::with-array-data ((data array)
+LaTeX (start)
+LaTeX (end))
+LaTeX (declare (ignore end))
+LaTeX ;; DATA is a specialized simple-array. Memory is laid out like this:
+LaTeX ;;
+LaTeX ;; byte offset Value
+LaTeX ;; 0 type code (should be 70 for double-float vector)
+LaTeX ;; 4 4 * number of elements in vector
+LaTeX ;; 8 1st element of vector
+LaTeX ;; ... ...
+LaTeX ;;
+LaTeX (let ((addr (+ 8 (logandc1 7 (kernel:get-lisp-obj-address data))))
+LaTeX (type-size (let ((type (array-element-type data)))
+LaTeX (cond ((or (equal type '(signed-byte 8))
+LaTeX (equal type '(unsigned-byte 8)))
+LaTeX 1)
+LaTeX ((or (equal type '(signed-byte 16))
+LaTeX (equal type '(unsigned-byte 16)))
+LaTeX 2)
+LaTeX ((or (equal type '(signed-byte 32))
+LaTeX (equal type '(unsigned-byte 32)))
+LaTeX 4)
+LaTeX ((equal type 'single-float)
+LaTeX 4)
+LaTeX ((equal type 'double-float)
+LaTeX 8)
+LaTeX (t
+LaTeX (error "Unknown specialized array element type"))))))
+LaTeX (declare (type (unsigned-byte 32) addr)
+LaTeX (optimize (speed 3) (safety 0) (ext:inhibit-warnings 3)))
+LaTeX (system:int-sap (the (unsigned-byte 32)
+LaTeX (+ addr (* type-size start)))))))
+LaTeX \end{example}
+LaTeX
+LaTeX Assume we have the C function below that we wish to use:
+LaTeX \begin{example}
+LaTeX double dotprod(double* x, double* y, int n)
+LaTeX \{
+LaTeX int k;
+LaTeX double sum = 0;
+LaTeX
+LaTeX for (k = 0; k < n; ++k) \{
+LaTeX sum += x[k] * y[k];
+LaTeX \}
+LaTeX \}
+LaTeX \end{example}
+LaTeX The following example generates two large arrays in Lisp, and calls the C
+LaTeX function to do the desired computation. This would not have been
+LaTeX possible using \code{malloc} or \code{make-alien} since we need about
+LaTeX 16 MB of memory to hold the two arrays.
+LaTeX \begin{example}
+LaTeX (define-alien-routine "dotprod" double
+LaTeX (x (* double-float) :in)
+LaTeX (y (* double-float) :in)
+LaTeX (n int :in))
+LaTeX
+LaTeX (let ((x (make-array 1000000 :element-type 'double-float))
+LaTeX (y (make-array 1000000 :element-type 'double-float)))
+LaTeX ;; Initialize X and Y somehow
+LaTeX (let ((x-addr (system:int-sap (array-data-address x)))
+LaTeX (y-addr (system:int-sap (array-data-address y))))
+LaTeX (dotprod x-addr y-addr 1000000)))
+LaTeX \end{example}
+LaTeX In this example, it may be useful to wrap the inner \code{let}
+LaTeX expression in an \code{unwind-protect} that first turns off garbage
+LaTeX collection and then turns garbage collection on afterwards. This will
+LaTeX prevent garbage collection from moving \code{x} and \code{y} after we
+LaTeX have obtained the (now erroneous) addresses but before the call to
+LaTeX \code{dotprod} is made.
+LaTeX
+-->
+
+</sect2>
+
+</sect1>
+
+<sect1 id="ffi-example"><title>Step-By-Step Example of the Foreign Function Interface</title>
+
+<para>
+This section presents a complete example of an interface to a somewhat
+complicated C function.
+</para>
+
+<para>
+Suppose you have the following C function which you want to be able to
+call from Lisp in the file <filename>test.c</filename>
+<programlisting>
+struct c_struct
+{
+ int x;
+ char *s;
+};
+
+struct c_struct *c_function (i, s, r, a)
+ int i;
+ char *s;
+ struct c_struct *r;
+ int a[10];
+{
+ int j;
+ struct c_struct *r2;
+
+ printf("i = %d\n", i);
+ printf("s = %s\n", s);
+ printf("r->x = %d\n", r->x);
+ printf("r->s = %s\n", r->s);
+ for (j = 0; j < 10; j++) printf("a[%d] = %d.\n", j, a[j]);
+ r2 = (struct c_struct *) malloc (sizeof(struct c_struct));
+ r2->x = i + 5;
+ r2->s = "a C string";
+ return(r2);
+};</programlisting>
+</para>
+
+<para>
+It is possible to call this C function from Lisp using the file
+<filename>test.lisp</filename> containing
+<programlisting>
+(cl:defpackage "TEST-C-CALL" (:use "CL" "SB-ALIEN" "SB-C-CALL"))
+(cl:in-package "TEST-C-CALL")
+
+;;; Define the record C-STRUCT in Lisp.
+(define-alien-type nil
+ (struct c-struct
+ (x int)
+ (s c-string)))
+
+;;; Define the Lisp function interface to the C routine. It returns a
+;;; pointer to a record of type C-STRUCT. It accepts four parameters:
+;;; I, an int; S, a pointer to a string; R, a pointer to a C-STRUCT
+;;; record; and A, a pointer to the array of 10 ints.
+;;;
+;;; The INLINE declaration eliminates some efficiency notes about heap
+;;; allocation of alien values.
+(declaim (inline c-function))
+(define-alien-routine c-function
+ (* (struct c-struct))
+ (i int)
+ (s c-string)
+ (r (* (struct c-struct)))
+ (a (array int 10)))
+
+;;; a function which sets up the parameters to the C function and
+;;; actually calls it
+(defun call-cfun ()
+ (with-alien ((ar (array int 10))
+ (c-struct (struct c-struct)))
+ (dotimes (i 10) ; Fill array.
+ (setf (deref ar i) i))
+ (setf (slot c-struct 'x) 20)
+ (setf (slot c-struct 's) "a Lisp string")
+
+ (with-alien ((res (* (struct c-struct))
+ (c-function 5 "another Lisp string" (addr c-struct) ar)))
+ (format t "~&back from C function~%")
+ (multiple-value-prog1
+ (values (slot res 'x)
+ (slot res 's))
+
+ ;; Deallocate result. (after we are done referring to it:
+ ;; "Pillage, *then* burn.")
+ (free-alien res)))))</programlisting>
+</para>
+
+<para>
+To execute the above example, it is necessary to compile the C routine,
+e.g.:
+<userinput>cc -c test.c</userinput>
+(In order to enable incremental loading with some linkers, you may need
+to say
+<userinput>cc -G 0 -c test.c</userinput>)
+</para>
+
+<para>
+Once the C code has been compiled, you can start up Lisp and load it in:
+<userinput>sbcl</userinput>.
+Lisp should start up with its normal prompt.</para>
+
+<para>
+Within Lisp,
+compile the Lisp file. (This step can be done separately. You don't
+have to recompile every time.)
+<userinput>(compile-file "test.lisp")</userinput>
+</para>
+
+<para>
+Within Lisp, load the foreign object file to define the necessary
+symbols:
+<userinput>(load-foreign "test.o")</userinput>.
+This must be done before loading any code that refers
+to these symbols.
+</para>
+
+<para>
+Now you can load the compiled Lisp ("fasl") file into Lisp:
+<userinput>(load "test.fasl")</userinput>
+And once the Lisp file is loaded, you can call the
+Lisp routine that sets up the parameters and calls the C
+function:
+<userinput>(test-c-call::call-cfun)</userinput>
+</para>
+
+<para>
+The C routine should print the following information to standard output:
+<!-- FIXME: What should be here is a verbatim environment for computer
+ output, but since I don't know one in DocBook, I made do with
+ PROGRAMLISTING for now... -->
+<programlisting>i = 5
+s = another Lisp string
+r->x = 20
+r->s = a Lisp string
+a[0] = 0.
+a[1] = 1.
+a[2] = 2.
+a[3] = 3.
+a[4] = 4.
+a[5] = 5.
+a[6] = 6.
+a[7] = 7.
+a[8] = 8.
+a[9] = 9.</programlisting>
+After return from the C function,
+the Lisp wrapper function should print the following output:
+<programlisting>back from C function</programlisting>
+And upon return from the Lisp wrapper function,
+before the next prompt is printed, the
+Lisp read-eval-print loop should print the following return values:
+<!-- FIXME: As above, it's not a program listing, but computer output... -->
+<programlisting>
+10
+"a C string"
+</programlisting>
+</para>
+
+</sect1>
+
+</chapter>
projects / sbcl.git / blobdiff