<chapter id="ffi"><title>The Foreign Function Interface</>
-<para>FIXME: The material in the &CMUCL; manual about the foreign
-function interface should be reviewed, reformatted in DocBook,
-lightly edited for &SBCL;, and substituted into this manual. But in
-the meantime, the original &CMUCL; manual is still 95+% correct for
-the &SBCL; version of the foreign function interface. (The main
-difference is that the package names have changed. CMU CL's FFI
-is in two packages named <quote><literal>ALIEN</></> and
-<quote><literal>C-CALL</></>. The corresponding SBCL FFI is
-in a single package named
-<quote><literal>SB-ALIEN</></>.)
- <!-- FIXME: Oh, and I seem to remember that the CMUCL manual
- was out of date about how to test for a null pointer,
- there's a builtin operator to do it, you don't need to
- do the nasty idiom the manual says you need to do. -->
- <!-- FIXME: Also, the CMU CL alien documentation claims you
- can just do (DEFINE-ALIEN-VARIABLE "errno" INT), but it's
- wrong. That fails with the multithreading hacks hacks
- now built into the C library. -->
- <!-- FIXME: Also, we support not only LOAD-FOREIGN but
- also LOAD-1-FOREIGN. -->
-See the sections
+<para>This chapter describes &SBCL;'s interface to C programs and
+libraries (and, since C interfaces are a sort of <foreignphrase>lingua
+franca</> of the Unix world, to other programs and libraries in
+general.)</para>
+
+<note><para>In the modern Lisp world, the usual term for this
+functionality is Foreign Function Interface, or <acronym>FFI</>, where
+despite the mention of <quote>function</> in this term, <acronym>FFI</> also
+refers to direct manipulation of C data structures as well as
+functions. The traditional &CMUCL; terminology is Alien Interface, and
+while that older terminology is no longer used much in the system
+documentation, it still reflected in names in the
+implementation, notably in the name of the <literal>SB-ALIEN</>
+package.</para></note>
+
+<sect1><title>Introduction to the Foreign Function Interface</>
+<!-- AKA "Introduction to Aliens" in the CMU CL manual -->
+
+<para>
+Because of Lisp's emphasis on dynamic memory allocation and garbage
+collection, Lisp implementations use non-C-like memory representations
+for objects. This representation mismatch creates friction when a Lisp
+program must share objects with programs which expect C data. There
+are three common approaches to establishing communication:
+<itemizedlist>
+ <listitem><para>The burden can be placed on the foreign program
+ (and programmer) by requiring the knowledge and use of the
+ representations used internally by the Lisp implementation.
+ This can require a considerable amount of <quote>glue</> code on the
+ C side, and that code tends to be sensitively dependent on the
+ internal implementation details of the Lisp system.</para></listitem>
+ <listitem><para>The Lisp system can automatically convert objects
+ back and forth between the Lisp and foreign representations.
+ This is convenient, but translation becomes prohibitively slow
+ when large or complex data structures must be shared. This approach
+ is supported by the &SBCL; <acronym>FFI</>, and used automatically
+ by the when passing integers and strings.</para></listitem>
+ <listitem><para>The Lisp program can directly manipulate foreign
+ objects through the use of extensions to the Lisp language.
+ </para></listitem>
+</itemizedlist>
+
+<para>&SBCL;, like &CMUCL; before it,
+relies primarily on the automatic conversion and direct manipulation
+approaches. Foreign values of simple scalar types are automatically
+converted, complex types are directly manipulated in their foreign
+representation. Furthermore, Lisp strings are represented internally
+with null termination bytes so that they can be passed directly to
+C interfaces without allocating new zero-terminated copies.</para>
+
+<para>Any foreign objects that can't automatically be converted into
+Lisp values are represented by objects of type <type>alien-value</>.
+Since Lisp is a dynamically typed language, even foreign objects must
+have a run-time type; this type information is provided by
+encapsulating the raw pointer to the foreign data within an
+<type>alien-value</> object.</para>
+
+<para>The type language and operations on foreign types are
+intentionally similar to those of the C language.</para>
+
+</sect1>
+
+<sect1><title>Foreign Types</>
+<!-- AKA "Alien Types" in the CMU CL manual -->
+
+<para>Alien types have a description language based on nested list
+structure. For example the C type
+<programlisting>struct foo {
+ int a;
+ struct foo *b[100];
+};</programlisting>
+has the corresponding &SBCL; FFI type
+<programlisting>(struct foo
+ (a int)
+ (b (array (* (struct foo)) 100)))</programlisting>
+</para>
+
+<sect2><title>Defining Foreign Types</>
+
+<para>
+Types may be either named or anonymous. With structure and union
+types, the name is part of the type specifier, allowing recursively
+defined types such as:
+<programlisting>(struct foo (a (* (struct foo))))</programlisting>
+An anonymous structure or union type is specified by using the name
+<literal>nil</>. The <function>with-alien</> macro defines a local
+scope which <quote>captures</> any named type definitions. Other types
+are not inherently named, but can be given named abbreviations using
+the <function>define-alien-type</> macro.
+</para>
+
+</sect2>
+
+<sect2><title>Foreign Types and Lisp Types</>
+
+<para>
+The foreign types form a subsystem of the &SBCL; type system. An
+<type>alien</> type specifier provides a way to use any foreign type as a
+Lisp type specifier. For example,
+<programlisting>(typep foo '(alien (* int)))</programlisting>
+can be used to determine whether <varname>foo</> is a pointer to a foreign
+<type>int</>. <type>alien</> type specifiers can be used in the same ways
+as ordinary Lisp type specifiers (like <type>string</>.) Alien type
+declarations are subject to the same
+precise type checking <!-- FIXME: should be linked to id="precisetypechecking" -->
+as any other declaration.
+</para>
+
+<para>
+Note that the type identifiers used in the
+foreign type system overlap with native Lisp type
+specifiers in some cases. For example, the type specifier
+<type>(alien single-float)</type> is identical to <type>single-float</>, since
+foreign floats are automatically converted to Lisp floats. When
+<function>type-of</> is called on an alien value that is not automatically
+converted to a Lisp value, then it will return an <type>alien</> type
+specifier.
+</para>
+
+</sect2>
+
+<sect2><title>Foreign Type Specifiers</>
+
+<note><para>
+All foreign type names are exported from the <literal>sb-alien</>
+package. Some foreign type names are also symbols in
+the <literal>common-lisp</> package, in which case they are
+reexported from the <literal>sb-alien</> package, so that
+e.g. it is legal to refer to <type>sb-alien:single-float</>.
+</para></note>
+
+<para>
+These are the basic foreign type specifiers:
+<!-- FIXME: There must be some better way of formatting definitions
+ in DocBook than this. I haven't found it yet, but suggestions
+ or patches would be welcome. -->
+<itemizedlist>
+ <listitem>
+ <para>
+ The foreign type specifier <type>(* foo)</> describes a
+ pointer to an object of type <type>foo</>. A pointed-to type
+ <type>foo</> of <type>t</> indicates a pointer to anything,
+ similar to <type>void *</> in ANSI C. A null alien pointer can
+ be detected with the <function>sb-alien:null-alien</>
+ function.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>(array foo &optional dimensions)</>
+ describes array of the specified <literal>dimensions</>, holding
+ elements of type <type>foo</>. Note that (unlike in C) <type>(* foo)</>
+ <type>(array foo)}</> are considered to be different types when
+ type checking is done. If equivalence of pointer and array types
+ is desired, it may be explicitly coerced using
+ <function>sb-alien:cast</>.
+ </para>
+ <para>
+ Arrays are accessed using <function>sb-alien:deref</>, passing
+ the indices as additional arguments. Elements are stored in
+ column-major order (as in C), so the first dimension determines
+ only the size of the memory block, and not the layout of the
+ higher dimensions. An array whose first dimension is variable
+ may be specified by using <literal>nil</> as the first dimension.
+ Fixed-size arrays can be allocated as array elements, structure
+ slots or <function>sb-alien:with-alien</> variables. Dynamic
+ arrays can only be allocated using <function>sb-alien:make-alien</>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier
+ <type>(sb-alien:struct name &rest fields)</>
+ describes a structure type with the specified <varname>name</> and
+ <varname>fields</>. Fields are allocated at the same offsets
+ used by the implementation's C compiler. If <varname>name</>
+ is <literal>nil</> then the structure is anonymous.
+ </para>
+ <para>
+ If a named foreign <type>struct</> specifier is passed to
+ <function>define-alien-type</> or <function>with-alien</>,
+ then this defines, respectively, a new global or local foreign
+ structure type. If no <varname>fields</> are specified, then
+ the fields are taken from the current (local or global) alien
+ structure type definition of <varname>name</>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier
+ <type>(sb-alien:union name &rest fields)</>
+ is similar to <type>sb-alien:struct</>, but describes a union type.
+ All fields are allocated at the same offset, and the size of the
+ union is the size of the largest field. The programmer must
+ determine which field is active from context.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>(sb-alien:enum name &rest specs)</>
+ describes an enumeration type that maps between integer values
+ and keywords. If <varname>name</> is <literal>nil</>, then the
+ type is anonymous. Each element of the <varname>specs</>
+ list is either a Lisp keyword, or a list <literal>(keyword value)</>.
+ <varname>value</> is an integer. If <varname>value</> is not
+ supplied, then it defaults to one greater than the value for
+ the preceding spec (or to zero if it is the first spec.)
+ <para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>(sb-alien:signed &optional bits)</>
+ specifies a signed integer with the specified number of
+ <varname>bits</> precision. The upper limit on integer
+ precision is determined by the machine's word
+ size. If <varname>bits</> is not specified, the maximum
+ size will be used.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>(integer &optional bits)</> is
+ equivalent to the corresponding type specifier using
+ <type>sb-alien:signed</> instead of <type>integer</>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier
+ <type>(sb-alien:unsigned &optional bits)</>
+ is like corresponding type specifier using <type>sb-alien:signed</>
+ except that the variable is treated as an unsigned integer.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>(boolean &optional bits)</> is
+ similar to an enumeration type, but maps from Lisp <literal>nil</>
+ and <literal>t</> to C <literal>0</> and <literal>1</>
+ respectively. <varname>bits</> determines the amount of
+ storage allocated to hold the truth value.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>single-float</> describes a
+ floating-point number in IEEE single-precision format.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>double-float</> describes a
+ floating-point number in IEEE double-precision format.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier
+ <type>(function result-type &rest arg-types)</>
+ describes a foreign function that takes arguments of the specified
+ <varname>arg-types</> and returns a result of type <type>result-type</>.
+ Note that the only context where a foreign <type>function</> type
+ is directly specified is in the argument to
+ <function>sb-alien:alien-funcall</>.
+ In all other contexts, foreign functions are represented by
+ foreign function pointer types: <type>(* (function ...))</>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>sb-alien:system-area-pointer</>
+ describes a pointer which is represented in Lisp as a
+ <type>system-area-pointer</> object. &SBCL; exports this type from
+ <literal>sb-alien</> because &CMUCL; did, but tentatively (as of
+ the first draft of this section of the manual, &SBCL; 0.7.6) it is
+ deprecated, since it doesn't seem to be required by user code.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>sb-alien:void</> is
+ used in function types to declare that no useful value
+ is returned. Using <function>alien-funcall</>
+ to call a <type>void</> foreign function will return
+ zero values.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The foreign type specifier <type>sb-alien:c-string</>
+ is similar to <type>(* char)</>, but is interpreted as a
+ null-terminated string, and is automatically converted into a
+ Lisp string when accessed; or if the pointer is C <literal>NULL</>
+ or <literal>0</>, then accessing it gives Lisp <literal>nil</>.
+ </para>
+ <para>
+ Assigning a Lisp string to a <type>c-string</> structure field or
+ variable stores the contents of the string to the memory already
+ pointed to by that variable. When a foreign object of type
+ <type>(* char)</> is assigned to a <type>c-string</>, then the
+ <type>c-string</> pointer is assigned to. This allows
+ <type>c-string</> pointers to be initialized. For example:
+ <programlisting>(cl:in-package "CL-USER") ; which USEs package "SB-ALIEN"
+ (define-alien-type nil (struct foo (str c-string)))
+ (defun make-foo (str) (let ((my-foo (make-alien (struct foo))))
+ (setf (slot my-foo 'str) (make-alien char (length str))
+ (slot my-foo 'str) str) my-foo))</programlisting>
+ Storing Lisp <literal>NIL</> in a <type>c-string</> writes C
+ <literal>NULL</> to the variable.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>sb-alien</> also exports translations of these C type
+ specifiers as foreign type specifiers:
+ <type>sb-alien:char</>,
+ <type>sb-alien:short</>,
+ <type>sb-alien:int</>,
+ <type>sb-alien:long</>,
+ <type>sb-alien:unsigned-char</>,
+ <type>sb-alien:unsigned-short</>,
+ <type>sb-alien:unsigned-int</>,
+ <type>sb-alien:unsigned-long</>,
+ <type>sb-alien:float</>, and
+ <type>sb-alien:double</>.
+ </para>
+ </listitem>
+
+</itemizedlist>
+
+</sect1>
+
+<sect1><title>Operations On Foreign Values</>
+<!-- AKA "Alien Operations" in the CMU CL manual -->
+
+<para>This section describes how to read foreign values as Lisp
+values, how to coerce foreign values to different kinds of foreign values, and
+how to dynamically allocate and free foreign variables.</para>
+
+<sect2><title>Accessing Foreign Values</>
+
+<synopsis>(sb-alien:deref pointer-or-array &rest indices)</>
+
+<para>The <function>sb-alien:deref</> function returns the value pointed to by
+a foreign pointer, or the value of a foreign array element. When
+dereferencing a pointer, an optional single index can be specified to
+give the equivalent of C pointer arithmetic; this index is scaled by
+the size of the type pointed to. When dereferencing an array, the
+number of indices must be the same as the number of dimensions in the
+array type. <function>deref</> can be set with <function>setf</> to
+assign a new value.</para>
+
+<synopsis>(sb-alien:slot struct-or-union &rest slot-names)</>
+
+<para>The <function>sb-alien:slot</> function extracts the value of
+the slot named <varname>slot-name</> from a foreign <type>struct</> or
+<type>union</>. If <varname>struct-or-union</> is a pointer to a
+structure or union, then it is automatically dereferenced.
+<function>sb-alien:slot</> can be set with <function>setf</> to assign
+a new value. Note that <varname>slot-name</> is evaluated, and need
+not be a compile-time constant (but only constant slot accesses are
+efficiently compiled.)</para>
+
+</sect2>
+
+<sect2><title>Coercing Foreign Values</>
+
+<synopsis>(sb-alien:addr alien-expr)</>
+
+<para>
+The <function>sb-alien:addr</> macro
+returns a pointer to the location specified by
+<varname>alien-expr</>, which must be either a foreign variable, a use of
+<function>sb-alien:deref</>, a use of <function>sb-alien:slot</>, or a use of
+<function>sb-alien:extern-alien</>.
+</para>
+
+<synopsis>(sb-alien:cast foreign-value new-type)</>
+
+<para>The <function>sb-alien:cast</>
+converts <varname>foreign-value</> to a new foreign value with the specified
+<varname>new-type</>. Both types, old and new, must be foreign pointer,
+array or function types. Note that the resulting Lisp
+foreign variable object
+is not <function>eq</> to the
+argument, but it does refer to the same foreign data bits.</para>
+
+<synopsis>(sb-alien:sap-alien sap type)</>
+
+<para>The <function>sb-alien:sap-alien</> function converts <varname>sap</>
+(a system area pointer) to a foreign value with the specified
+<varname>type</>. <varname>type</> is not evaluated.
+As of &SBCL; 0.7.6, it looks as though this and other SAP functionality
+may become deprecated, since it shouldn't be needed by user code.
+</para>
+
+<para>The <varname>type</> must be some foreign pointer, array, or
+record type.</para>
+
+<synopsis>(sb-alien:alien-sap foreign-value type)</>
+
+<para>The <function>sb-alien:alien-sap</> function
+returns the SAP which points to <varname>alien-value</>'s data.
+As of &SBCL; 0.7.6, it looks as though this and other SAP functionality
+may become deprecated, since it shouldn't be needed by user code.
+</para>
+
+<para>The <varname>foreign-value</> must be of some foreign pointer,
+array, or record type.</para>
+
+</sect2>
+
+<sect2><title>Foreign Dynamic Allocation</>
+
+<para>Lisp code can call the C standard library functions
+<function>malloc</> and <function>free</> to dynamically allocate and
+deallocate foreign variables. The Lisp code shares the same allocator
+with foreign C code, so it's OK for foreign code to call
+<function>free</> on the result of Lisp
+<function>sb-alien:make-alien</>, or for Lisp code to call
+<function>sb-alien:free-alien</> on foreign objects allocated by C
+code.</para>
+
+<synopsis>(sb-alien:make-alien type size)</>
+
+<para>The <function>sb-alien:make-alien</> macro
+returns a dynamically allocated foreign value of the specified
+<varname>type</> (which is not evaluated.) The allocated memory is not
+initialized, and may contain arbitrary junk. If supplied,
+<varname>size</> is an expression to evaluate to compute the size of the
+allocated object. There are two major cases:
+<itemizedlist>
+ <listitem>
+ <para>When <varname>type</> is a foreign array type, an array of
+ that type is allocated and a pointer to it is returned. Note that you
+ must use <function>deref</> to change the result to an array before you
+ can use <function>deref</> to read or write elements:
+ <programlisting>
+ (cl:in-package "CL-USER") ; which USEs package "SB-ALIEN"
+ (defvar *foo* (make-alien (array char 10)))
+ (type-of *foo*) => (alien (* (array (signed 8) 10)))
+ (setf (deref (deref foo) 0) 10) => 10</programlisting>
+ If supplied, <varname>size</> is used as the first dimension for the
+ array.</para>
+ </listitem>
+ <listitem>
+ <para>When <varname>type</> is any other foreign type, then an
+ object for that type is allocated, and a pointer to it is
+ returned. So <function>(make-alien int)</> returns a <type>(* int)</>.
+ If <varname>size</> is specified, then a block of that many
+ objects is allocated, with the result pointing to the first one.</para>
+ </listitem>
+</itemizedlist>
+</para>
+
+<synopsis>(sb-alien:free-alien foreign-value)</>
+
+<para>The <function>sb-alien:free-alien</> function
+frees the storage for <varname>foreign-value</>,
+which must have been allocated with Lisp <function>make-alien</>
+or C <function>malloc</>.</para>
+
+<para>See also the <function>sb-alien:with-alien</> macro, which
+allocates foreign values on the stack.</para>
+
+</sect1>
+
+<sect1><title>Foreign Variables</>
+<!-- AKA "Alien Variables" in the CMU CL manual -->
+
+<para>
+Both local (stack allocated) and external (C global) foreign variables are
+supported.
+</para>
+
+<sect2><title>Local Foreign Variables</>
+
+<synopsis>(sb-alien:with-alien var-definitions &body body)</>
+
+<para>The <function>with-alien</>
+macro establishes local
+foreign variables
+with the specified
+alien types and names.
+This form is analogous to defining a local variable in C: additional
+storage is allocated, and the initial value is copied.
+This form is less
+analogous to LET-allocated Lisp variables, since the variables
+can't be captured in closures: they live only for the dynamic extent
+of the body, and referring to them outside is a gruesome error.
+</para>
+
+<para>The <varname>var-definitions</> argument is a list of
+variable definitions, each of the form
+<programlisting>(name type &optional initial-value)</programlisting>
+The names of the variables are established as symbol-macros; the bindings have
+lexical scope, and may be assigned with <function>setq</>
+or <function>setf</>.
+</para>
+
+<para>The <function>with-alien</> macro also establishes
+a new scope for named structures
+and unions. Any <varname>type</> specified for a variable may contain
+named structure or union types with the slots specified. Within the
+lexical scope of the binding specifiers and body, a locally defined
+foreign structure type <type>foo</> can be referenced by its name using
+<type>(struct foo)</>.
+</para>
+
+</sect2>
+
+<sect2><title>External Foreign Variables</>
+
+<para>
+External foreign names are strings, and Lisp names are symbols. When
+an external foreign value is represented using a Lisp variable, there
+must be a way to convert from one name syntax into the other. The
+macros <function>extern-alien</>, <function>define-alien-variable</> and
+<function>define-alien-routine</> use this conversion heuristic:
<itemizedlist>
- <listitem><para>Type Translations</></>
- <listitem><para>System Area Pointers</></>
- <listitem><para>Alien Objects</></>
- <listitem><para>Alien Types</></>
- <listitem><para>Alien Operations</></>
- <listitem><para>Alien Variables</></>
- <listitem><para>Alien Function Calls</></>
+ <listitem><para>Alien names are converted to Lisp names by uppercasing and
+ replacing underscores with hyphens.</para></listitem>
+ <listitem><para>Conversely, Lisp names are converted to alien names by
+ lowercasing and replacing hyphens with underscores.</para></listitem>
+ <listitem><para>Both the Lisp symbol and alien string names may be
+ separately specified by using a list of the form
+ <programlisting>(alien-string lisp-symbol)</></para></listitem>
</itemizedlist>
+
+<synopsis>(sb-alien:define-alien-variable name type)</>
+
+<para>
+The <function>define-alien-variable</> macro
+defines <varname>name</> as an external foreign variable of the
+specified foreign <type>type</>. <varname>name</> and <type>type</> are not
+evaluated. The Lisp name of the variable (see above) becomes a
+global alien variable. Global alien variables
+are effectively ``global symbol macros''; a reference to the
+variable fetches the contents of the external variable. Similarly,
+setting the variable stores new contents---the new contents must be
+of the declared <type>type</>. Someday, they may well be implemented
+using the &ANSI; <function>define-symbol-macro</> mechanism, but
+as of &SBCL; 0.7.5, they are still implemented using an older
+more-or-less parallel mechanism inherited from &CMUCL;.
+</para>
+
+<para>
+For example, to access a C-level counter <varname>foo</>, one could
+write
+<programlisting>
+(define-alien-variable "foo" int)
+;; Now it is possible to get the value of the C variable foo simply by
+;; referencing that Lisp variable:
+(print foo)
+(setf foo 14)
+(incf foo)</programlisting>
</para>
-</chapter>
\ No newline at end of file
+<synopsis>(sb-alien:get-errno)</>
+
+<para>
+Since in modern C libraries, the <varname>errno</> "variable" is typically
+no longer a variable, but some bizarre artificial construct
+which behaves superficially like a variable within a given thread,
+it can no longer reliably be accessed through the ordinary
+<varname>define-alien-variable</> mechanism. Instead, &SBCL; provides
+the operator <function>sb-alien:get-errno</> to allow Lisp code to read it.
+</para>
+
+<synopsis>(sb-alien:extern-alien name type)</>
+
+<para>
+The <function>extern-alien</> macro
+returns an alien with the specified <type>type</> which
+points to an externally defined value. <varname>name</> is not evaluated,
+and may be either a string or a symbol. <type>type</> is
+an unevaluated alien type specifier.
+</para>
+
+</sect1>
+
+<sect1><title>Foreign Data Structure Examples</>
+<!-- AKA "Alien Data Structure Example" in the CMU CL manual -->
+
+<para>
+Now that we have alien types, operations and variables, we can manipulate
+foreign data structures. This C declaration
+<programlisting>
+struct foo {
+ int a;
+ struct foo *b[100];
+};</programlisting>
+can be translated into the following alien type:
+<programlisting>(define-alien-type nil
+ (struct foo
+ (a int)
+ (b (array (* (struct foo)) 100))))</programlisting>
+</para>
+
+<para>
+Once the <type>foo</> alien type has been defined as above,
+the C expression
+<programlisting>
+struct foo f;
+f.b[7].a</programlisting>
+can be translated in this way:
+<programlisting>
+(with-alien ((f (struct foo)))
+ (slot (deref (slot f 'b) 7) 'a)
+ ;;
+ ;; Do something with f...
+ )</programlisting>
+</para>
+
+<para>
+Or consider this example of an external C variable and some accesses:
+<programlisting>
+struct c_struct {
+ short x, y;
+ char a, b;
+ int z;
+ c_struct *n;
+};
+extern struct c_struct *my_struct;
+my_struct->x++;
+my_struct->a = 5;
+my_struct = my_struct->n;</programlisting>
+which can be manipulated in Lisp like this:
+<programlisting>
+(define-alien-type nil
+ (struct c-struct
+ (x short)
+ (y short)
+ (a char)
+ (b char)
+ (z int)
+ (n (* c-struct))))
+(define-alien-variable "my_struct" (* c-struct))
+(incf (slot my-struct 'x))
+(setf (slot my-struct 'a) 5)
+(setq my-struct (slot my-struct 'n))</programlisting>
+</para>
+
+</sect1>
+
+<sect1><title>Loading Unix Object Files</>
+
+<para>
+Foreign object files can be loaded into the running Lisp process by
+calling the functions <function>load-foreign</> or
+<function>load-1-foreign</>.
+</para>
+
+<para> The <function>sb-alien:load-1-foreign</> function is the more
+primitive of the two operations. It loads a single object file. into
+the currently running Lisp. The external symbols defining routines and
+variables are made available for future external references (e.g. by
+<function>extern-alien</>). Forward references to foreign symbols
+aren't supported: <function>load-1-foreign</> must be run before any
+of the defined symbols are referenced.
+</para>
+
+<para><function>sb-alien:load-foreign</> is built in terms of
+<function>load-1-foreign</> and some other machinery
+like <function>sb-ext:run-program</>.
+It accepts a list of files and libraries,
+and runs the linker on the files and
+libraries, creating an absolute Unix object file which is then
+processed by <function>load-1-foreign</>.</para>
+
+<note><para>As of &SBCL; 0.7.5, all foreign code (code loaded
+with <function>load-1-function</> or <function>load-function</>) is
+lost when a Lisp core is saved with
+<function>sb-ext:save-lisp-and-die</>, and no attempt is made to
+restore it when the core is loaded. Historically this has been an
+annoyance both for &SBCL; users and for &CMUCL; users.
+It's hard to solve this problem completely cleanly, but some
+generally-reliable partial solution might be useful. Once someone in
+either camp gets sufficiently annoyed to create it, &SBCL; is
+likely to adopt some mechanism for automatically restoring foreign
+code when a saved core is loaded.</para></note>
+
+</sect1>
+
+<sect1><title>Foreign Function Calls</>
+
+<para>
+The foreign function call interface allows a Lisp program to call
+functions written in other languages using the C calling convention.
+</para>
+
+<para>
+Lisp sets up various interrupt handling routines and other environment
+information when it first starts up, and expects these to be in place
+at all times. The C functions called by Lisp should either not change
+the environment, especially the interrupt entry points, or should make
+sure that these entry points are restored when the C function returns
+to Lisp. If a C function makes changes without restoring things to the
+way they were when the C function was entered, there is no telling
+what will happen.
+</para>
+
+<sect2><title>The <function>alien-funcall</> Primitive</title>
+
+<synopsis>(sb-alien:alien-funcall alien-function &rest arguments)</>
+
+<para>
+The <function>alien-funcall</> function is the foreign function call
+primitive: <varname>alien-function</> is called with the supplied
+<varname>arguments</> and its C return value is returned as a Lisp value.
+The <varname>alien-function</> is an arbitrary
+run-time expression; to refer to a constant function, use
+<function>extern-alien</> or a value defined by
+<function>define-alien-routine</>.
+</para>
+
+<para>
+The type of <function>alien-function</>
+must be <type>(alien (function ...))</>
+or <type>(alien (* (function ...)))</>.
+The function type is used to
+determine how to call the function (as though it was declared with
+a prototype.) The type need not be known at compile time, but only
+known-type calls are efficiently compiled. Limitations:
+<itemizedlist>
+ <listitem><para>Structure type return values are not implemented.</></>
+ <listitem><para>Passing of structures by value is not implemented.</></>
+</itemizedlist>
+
+<para>
+Here is an example which allocates a <type>(struct foo)</>, calls a foreign
+function to initialize it, then returns a Lisp vector of all the
+<type>(* (struct foo))</> objects filled in by the foreign call:
+<programlisting>
+;; Allocate a foo on the stack.
+(with-alien ((f (struct foo)))
+ ;; Call some C function to fill in foo fields.
+ (alien-funcall (extern-alien "mangle_foo" (function void (* foo)))
+ (addr f))
+ ;; Find how many foos to use by getting the A field.
+ (let* ((num (slot f 'a))
+ (result (make-array num)))
+ ;; Get a pointer to the array so that we don't have to keep extracting it:
+ (with-alien ((a (* (array (* (struct foo)) 100)) (addr (slot f 'b))))
+ ;; Loop over the first N elements and stash them in the result vector.
+ (dotimes (i num)
+ (setf (svref result i) (deref (deref a) i)))
+ ;; Voila.
+ result)))</programlisting>
+</para>
+
+</sect2>
+
+<sect2><title>The <function>define-alien-routine</> Macro</>
+
+<synopsis>(sb-alien:define-alien-routine} name result-type &rest arg-specifiers)</>
+
+<para>
+The <function>define-alien-routine</> macro is a convenience
+for automatically generating Lisp
+interfaces to simple foreign functions. The primary feature is the
+parameter style specification, which translates the C
+pass-by-reference idiom into additional return values.
+</para>
+
+<para>
+<varname>name</> is usually a string external symbol, but may also be a
+symbol Lisp name or a list of the foreign name and the Lisp name.
+If only one name is specified, the other is automatically derived
+as for <function>extern-alien</>.
+<varname>result-type</> is the alien type of the return value.
+</para>
+
+<para>
+Each element of the <varname>arg-specifiers</> list
+specifies an argument to the foreign function, and is
+of the form
+<programlisting>(aname atype &optional style)</programlisting>
+<varname>aname</> is the symbol name of the argument to the constructed
+function (for documentation). <varname>atype</> is the alien type of
+corresponding foreign argument. The semantics of the actual call
+are the same as for <function>alien-funcall</>. <varname>style</>
+specifies how this argument should be handled at call and return time,
+and should be one of the following
+<itemizedlist>
+ <listitem><para><varname>:in</>specifies that the argument is
+ passed by value. This is the default. <varname>:in</> arguments
+ have no corresponding return value from the Lisp function.
+ </para></listitem>
+ <listitem><para><varname>:copy</> is similar to <varname>:in</>,
+ but the argument is copied
+ to a pre-allocated object and a pointer to this object is passed
+ to the foreign routine.</para></listitem>
+ <listitem><para><varname>:out</> specifies a pass-by-reference
+ output value. The type of the argument must be a pointer to
+ a fixed-sized object (such as an integer or pointer).
+ <varname>:out</> and <varname>:in-out</> style cannot
+ be used with pointers to arrays, records or functions. An
+ object of the correct size is allocated on the stack, and
+ its address is passed to the foreign function. When the
+ function returns, the contents
+ of this location are returned as one of the values of the Lisp
+ function (and the location is automatically deallocated).
+ </para></listitem>
+ <listitem><para><varname>:in-out</> is a combination of
+ <varname>:copy</> and <varname>:out</>.
+ The argument is copied to a pre-allocated object and a pointer to
+ this object is passed to the foreign routine. On return, the
+ contents of this location is returned as an additional value.
+ </para></listitem>
+</itemizedlist>
+</para>
+
+<note>
+<para>
+Any efficiency-critical foreign interface function should be inline
+expanded, which can be done by preceding the
+<function>define-alien-routine</> call with:
+<programlisting>(declaim (inline lisp-name))</programlisting>
+In addition to avoiding the Lisp call overhead, this allows
+pointers, word-integers and floats to be passed using non-descriptor
+representations, avoiding consing.)
+</para>
+</note>
+
+</sect2>
+
+<sect2><title><function>define-alien-routine</> Example</title>
+
+<para>
+Consider the C function <function>cfoo</>
+with the following calling convention:
+<programlisting>
+void
+cfoo (str, a, i)
+ char *str;
+ char *a; /* update */
+ int *i; /* out */
+{
+ /* body of cfoo(...) */
+}</programlisting>
+This can be described by the following call to
+<function>define-alien-routine</>:
+<programlisting>
+(define-alien-routine "cfoo" void
+ (str c-string)
+ (a char :in-out)
+ (i int :out))</programlisting>
+The Lisp function <function>cfoo</> will have
+two arguments (<varname>str</> and <varname>a</>)
+and two return values (<varname>a</> and <varname>i</>).
+
+</sect2>
+
+<sect2><title>Calling Lisp From C</>
+
+<para>
+Calling Lisp functions from C is sometimes possible, but is extremely
+hackish and poorly supported as of &SBCL; 0.7.5.
+See <function>funcall0</> ... <function>funcall3</> in
+the runtime system. The
+arguments must be valid &SBCL; object descriptors (so that
+e.g. fixnums must be
+left-shifted by 2.) As of &SBCL; 0.7.5, the format
+of object descriptors is documented only by the source code and
+by the old &CMUCL; "INTERNALS" documentation.</para>
+
+<para> Note that the garbage collector moves objects, and won't be
+able to fix up any references in C variables, so either turn GC off or
+don't keep Lisp pointers in C data unless they are to statically
+allocated objects. It is possible to use the
+<function>sb-ext:purify</> function to place live data structures in
+static space so that they won't move during GC. </para>
+
+<!-- FIXME: This is a "changebar" section from the CMU CL manual.
+ I (WHN 2002-07-14) am not very familiar with this content, so
+ I'm not immediately prepared to try to update it for SBCL, and
+ I'm not feeling masochistic enough to work to encourage this
+ kind of low-level hack anyway. However, I acknowledge that callbacks
+ are sometimes really really necessary, so I include the original
+ text in case someone is hard-core enough to benefit from it. If
+ anyone brings the information up to date for SBCL, it belong
+ either in the main manual or on a CLiki SBCL Internals page.
+LaTeX \subsection{Accessing Lisp Arrays}
+LaTeX
+LaTeX Due to the way \cmucl{} manages memory, the amount of memory that can
+LaTeX be dynamically allocated by \code{malloc} or \funref{make-alien} is
+LaTeX limited\footnote{\cmucl{} mmaps a large piece of memory for it's own
+LaTeX use and this memory is typically about 8 MB above the start of the C
+LaTeX heap. Thus, only about 8 MB of memory can be dynamically
+LaTeX allocated.}.
+LaTeX
+LaTeX To overcome this limitation, it is possible to access the content of
+LaTeX Lisp arrays which are limited only by the amount of physical memory
+LaTeX and swap space available. However, this technique is only useful if
+LaTeX the foreign function takes pointers to memory instead of allocating
+LaTeX memory for itself. In latter case, you will have to modify the
+LaTeX foreign functions.
+LaTeX
+LaTeX This technique takes advantage of the fact that \cmucl{} has
+LaTeX specialized array types (\pxlref{specialized-array-types}) that match
+LaTeX a typical C array. For example, a \code{(simple-array double-float
+LaTeX (100))} is stored in memory in essentially the same way as the C
+LaTeX array \code{double x[100]} would be. The following function allows us
+LaTeX to get the physical address of such a Lisp array:
+LaTeX \begin{example}
+LaTeX (defun array-data-address (array)
+LaTeX "Return the physical address of where the actual data of an array is
+LaTeX stored.
+LaTeX
+LaTeX ARRAY must be a specialized array type in CMU Lisp. This means ARRAY
+LaTeX must be an array of one of the following types:
+LaTeX
+LaTeX double-float
+LaTeX single-float
+LaTeX (unsigned-byte 32)
+LaTeX (unsigned-byte 16)
+LaTeX (unsigned-byte 8)
+LaTeX (signed-byte 32)
+LaTeX (signed-byte 16)
+LaTeX (signed-byte 8)
+LaTeX "
+LaTeX (declare (type (or #+signed-array (array (signed-byte 8))
+LaTeX #+signed-array (array (signed-byte 16))
+LaTeX #+signed-array (array (signed-byte 32))
+LaTeX (array (unsigned-byte 8))
+LaTeX (array (unsigned-byte 16))
+LaTeX (array (unsigned-byte 32))
+LaTeX (array single-float)
+LaTeX (array double-float))
+LaTeX array)
+LaTeX (optimize (speed 3) (safety 0))
+LaTeX (ext:optimize-interface (safety 3)))
+LaTeX ;; with-array-data will get us to the actual data. However, because
+LaTeX ;; the array could have been displaced, we need to know where the
+LaTeX ;; data starts.
+LaTeX (lisp::with-array-data ((data array)
+LaTeX (start)
+LaTeX (end))
+LaTeX (declare (ignore end))
+LaTeX ;; DATA is a specialized simple-array. Memory is laid out like this:
+LaTeX ;;
+LaTeX ;; byte offset Value
+LaTeX ;; 0 type code (should be 70 for double-float vector)
+LaTeX ;; 4 4 * number of elements in vector
+LaTeX ;; 8 1st element of vector
+LaTeX ;; ... ...
+LaTeX ;;
+LaTeX (let ((addr (+ 8 (logandc1 7 (kernel:get-lisp-obj-address data))))
+LaTeX (type-size (let ((type (array-element-type data)))
+LaTeX (cond ((or (equal type '(signed-byte 8))
+LaTeX (equal type '(unsigned-byte 8)))
+LaTeX 1)
+LaTeX ((or (equal type '(signed-byte 16))
+LaTeX (equal type '(unsigned-byte 16)))
+LaTeX 2)
+LaTeX ((or (equal type '(signed-byte 32))
+LaTeX (equal type '(unsigned-byte 32)))
+LaTeX 4)
+LaTeX ((equal type 'single-float)
+LaTeX 4)
+LaTeX ((equal type 'double-float)
+LaTeX 8)
+LaTeX (t
+LaTeX (error "Unknown specialized array element type"))))))
+LaTeX (declare (type (unsigned-byte 32) addr)
+LaTeX (optimize (speed 3) (safety 0) (ext:inhibit-warnings 3)))
+LaTeX (system:int-sap (the (unsigned-byte 32)
+LaTeX (+ addr (* type-size start)))))))
+LaTeX \end{example}
+LaTeX
+LaTeX Assume we have the C function below that we wish to use:
+LaTeX \begin{example}
+LaTeX double dotprod(double* x, double* y, int n)
+LaTeX \{
+LaTeX int k;
+LaTeX double sum = 0;
+LaTeX
+LaTeX for (k = 0; k < n; ++k) \{
+LaTeX sum += x[k] * y[k];
+LaTeX \}
+LaTeX \}
+LaTeX \end{example}
+LaTeX The following example generates two large arrays in Lisp, and calls the C
+LaTeX function to do the desired computation. This would not have been
+LaTeX possible using \code{malloc} or \code{make-alien} since we need about
+LaTeX 16 MB of memory to hold the two arrays.
+LaTeX \begin{example}
+LaTeX (define-alien-routine "dotprod" double
+LaTeX (x (* double-float) :in)
+LaTeX (y (* double-float) :in)
+LaTeX (n int :in))
+LaTeX
+LaTeX (let ((x (make-array 1000000 :element-type 'double-float))
+LaTeX (y (make-array 1000000 :element-type 'double-float)))
+LaTeX ;; Initialize X and Y somehow
+LaTeX (let ((x-addr (system:int-sap (array-data-address x)))
+LaTeX (y-addr (system:int-sap (array-data-address y))))
+LaTeX (dotprod x-addr y-addr 1000000)))
+LaTeX \end{example}
+LaTeX In this example, it may be useful to wrap the inner \code{let}
+LaTeX expression in an \code{unwind-protect} that first turns off garbage
+LaTeX collection and then turns garbage collection on afterwards. This will
+LaTeX prevent garbage collection from moving \code{x} and \code{y} after we
+LaTeX have obtained the (now erroneous) addresses but before the call to
+LaTeX \code{dotprod} is made.
+LaTeX
+-->
+
+</sect1>
+
+<sect1><title>Step-By-Step Example of the Foreign Function Interface</>
+
+<para>
+This section presents a complete example of an interface to a somewhat
+complicated C function.
+</para>
+
+<para>
+Suppose you have the following C function which you want to be able to
+call from Lisp in the file <filename>test.c</>
+<programlisting>
+struct c_struct
+{
+ int x;
+ char *s;
+};
+
+struct c_struct *c_function (i, s, r, a)
+ int i;
+ char *s;
+ struct c_struct *r;
+ int a[10];
+{
+ int j;
+ struct c_struct *r2;
+
+ printf("i = %d\n", i);
+ printf("s = %s\n", s);
+ printf("r->x = %d\n", r->x);
+ printf("r->s = %s\n", r->s);
+ for (j = 0; j < 10; j++) printf("a[%d] = %d.\n", j, a[j]);
+ r2 = (struct c_struct *) malloc (sizeof(struct c_struct));
+ r2->x = i + 5;
+ r2->s = "a C string";
+ return(r2);
+};</programlisting>
+</para>
+
+<para>
+It is possible to call this C function from Lisp using the file
+<filename>test.lisp</> containing
+<programlisting>
+(cl:defpackage "TEST-C-CALL" (:use "CL" "SB-ALIEN" "SB-C-CALL"))
+(cl:in-package "TEST-C-CALL")
+
+;;; Define the record C-STRUCT in Lisp.
+(define-alien-type nil
+ (struct c-struct
+ (x int)
+ (s c-string)))
+
+;;; Define the Lisp function interface to the C routine. It returns a
+;;; pointer to a record of type C-STRUCT. It accepts four parameters:
+;;; I, an int; S, a pointer to a string; R, a pointer to a C-STRUCT
+;;; record; and A, a pointer to the array of 10 ints.
+;;;
+;;; The INLINE declaration eliminates some efficiency notes about heap
+;;; allocation of alien values.
+(declaim (inline c-function))
+(define-alien-routine c-function
+ (* (struct c-struct))
+ (i int)
+ (s c-string)
+ (r (* (struct c-struct)))
+ (a (array int 10)))
+
+;;; a function which sets up the parameters to the C function and
+;;; actually calls it
+(defun call-cfun ()
+ (with-alien ((ar (array int 10))
+ (c-struct (struct c-struct)))
+ (dotimes (i 10) ; Fill array.
+ (setf (deref ar i) i))
+ (setf (slot c-struct 'x) 20)
+ (setf (slot c-struct 's) "a Lisp string")
+
+ (with-alien ((res (* (struct c-struct))
+ (c-function 5 "another Lisp string" (addr c-struct) ar)))
+ (format t "~&back from C function~%")
+ (multiple-value-prog1
+ (values (slot res 'x)
+ (slot res 's))
+
+ ;; Deallocate result. (after we are done referring to it:
+ ;; "Pillage, *then* burn.")
+ (free-alien res)))))</programlisting>
+</para>
+
+<para>
+To execute the above example, it is necessary to compile the C routine,
+e.g.:
+<userinput>cc -c test.c</>
+(In order to enable incremental loading with some linkers, you may need
+to say
+<userinput>cc -G 0 -c test.c</>)
+</para>
+
+<para>
+Once the C code has been compiled, you can start up Lisp and load it in:
+<userinput>sbcl</>.
+Lisp should start up with its normal prompt.</para>
+
+<para>
+Within Lisp,
+compile the Lisp file. (This step can be done separately. You don't
+have to recompile every time.)
+<userinput>(compile-file "test.lisp")</>
+
+<para>
+Within Lisp, load the foreign object file to define the necessary
+symbols:
+<userinput>(load-foreign "test.o")</>.
+This must be done before loading any code that refers
+to these symbols.
+<para>
+
+<para>
+Now you can load the compiled Lisp ("fasl") file into Lisp:
+<userinput>(load "test.fasl")</>
+And once the Lisp file is loaded, you can call the
+Lisp routine that sets up the parameters and calls the C
+function:
+<userinput>(test-c-call::call-cfun)</>
+</para>
+
+<para>
+The C routine should print the following information to standard output:
+<!-- FIXME: What should be here is a verbatim environment for computer
+ output, but since I don't know one in DocBook, I made do with
+ PROGRAMLISTING for now... -->
+<programlisting>i = 5
+s = another Lisp string
+r->x = 20
+r->s = a Lisp string
+a[0] = 0.
+a[1] = 1.
+a[2] = 2.
+a[3] = 3.
+a[4] = 4.
+a[5] = 5.
+a[6] = 6.
+a[7] = 7.
+a[8] = 8.
+a[9] = 9.</programlisting>
+After return from the C function,
+the Lisp wrapper function should print the following output:
+<programlisting>back from C function</programlisting>
+And upon return from the Lisp wrapper function,
+before the next prompt is printed, the
+Lisp read-eval-print loop should print the following return values:
+<!-- FIXME: As above, it's not a program listing, but computer output... -->
+<programlisting>
+10
+"a C string"
+</programlisting>
+</para>
+
+</sect1>
+
+</chapter>
projects / sbcl.git / blobdiff