1 <chapter id="ffi"><title>The Foreign Function Interface</>
3 <para>This chapter describes &SBCL;'s interface to C programs and
4 libraries (and, since C interfaces are a sort of <foreignphrase>lingua
5 franca</> of the Unix world, to other programs and libraries in
8 <note><para>In the modern Lisp world, the usual term for this
9 functionality is Foreign Function Interface, or <acronym>FFI</>, where
10 despite the mention of <quote>function</> in this term, <acronym>FFI</> also
11 refers to direct manipulation of C data structures as well as
12 functions. The traditional &CMUCL; terminology is Alien Interface, and
13 while that older terminology is no longer used much in the system
14 documentation, it still reflected in internal names in the
15 implementation, notably in the name of the <literal>SB-ALIEN</>
16 package.</para></note>
18 <sect1><title>Introduction to the Foreign Function Interface</>
19 <!-- AKA "Introduction to Aliens" in the CMU CL manual -->
22 Because of Lisp's emphasis on dynamic memory allocation and garbage
23 collection, Lisp implementations use unconventional memory representations
24 for objects. This representation mismatch creates problems when a Lisp
25 program must share objects with programs written in another language. There
26 are three different approaches to establishing communication:
28 <listitem><para>The burden can be placed on the foreign program
29 (and programmer) by requiring the knowledge and use of the
30 representations used internally by the Lisp implementation.
31 This can require a considerable amount of <quote>glue</> code on the
32 C side, and that code tends to be sensitively dependent on the
33 internal implementation details of the Lisp system.</para></listitem>
34 <listitem><para>The Lisp system can automatically convert objects
35 back and forth between the Lisp and foreign representations.
36 This is convenient, but translation becomes prohibitively slow
37 when large or complex data structures must be shared. This approach
38 is supported by the &SBCL; <acronym>FFI</>, and used automatically
39 by the when passing integers and strings.</para></listitem>
40 <listitem><para>The Lisp program can directly manipulate foreign
41 objects through the use of extensions to the Lisp language.
45 <para>&SBCL;, like &CMUCL; before it,
46 relies primarily on the automatic conversion and direct manipulation
47 approaches. Aliens of simple scalar types are automatically converted,
48 complex types are directly manipulated in their foreign
49 representation. Furthermore, Lisp strings are represented internally
50 with null termination bytes so that they can be passed directly to
51 C interfaces without allocating new zero-terminated copies.</para>
53 <para>Any foreign objects that can't automatically be converted into
54 Lisp values are represented by objects of type <type>alien-value</>.
55 Since Lisp is a dynamically typed language, even foreign objects must
56 have a run-time type; this type information is provided by
57 encapsulating the raw pointer to the foreign data within an
58 <type>alien-value</> object.</para>
60 <para>The type language and operations on foreign types are
61 intentionally similar to those of the C language. And as discussed
62 above, they are applicable not only to communication with native C
63 programs, but also to programs in other languages which provide
64 C-level interfaces. </para>
68 <sect1><title>Foreign Types</>
69 <!-- AKA "Alien Types" in the CMU CL manual -->
71 <para>Alien types have a description language based on nested list
72 structure. For example the C type
78 has the corresponding &SBCL; FFI type
82 (b (array (* (struct foo)) 100)))</programlisting>
85 <sect2><title>Defining Foreign Types</>
88 Types may be either named or anonymous. With structure and union
89 types, the name is part of the type specifier, allowing recursively
90 defined types such as:
92 (struct foo (a (* (struct foo))))</programlisting>
93 An anonymous structure or union type is specified by using the name
94 <literal>nil</>. The <function>with-alien</> macro defines a local
95 scope which <quote>captures</> any named type definitions. Other types
96 are not inherently named, but can be given named abbreviations using
97 the <function>define-alien-type</> macro.
102 <sect2><title>Foreign Types and Lisp Types</>
105 The foreign types form a subsystem of the &SBCL; type system. An
106 <type>alien</> type specifier provides a way to use any foreign type as a
107 Lisp type specifier. For example
109 (typep foo '(alien (* int)))</programlisting>
110 can be used to determine whether <varname>foo</> is a pointer to a foreign
111 <type>int</>. <type>alien</> type specifiers can be used in the same ways
112 as ordinary Lisp type specifiers (like <type>string</>.) Alien type
113 declarations are subject to the same
114 precise type checking <!-- FIXME: should be linked to id="precisetypechecking" -->
115 as any other declaration.
119 Note that the type identifiers used in the
120 foreign type system overlap with native Lisp type
121 specifiers in some cases. For example, the type specifier
122 <type>(alien single-float)</type> is identical to <type>single-float</>, since
123 foreign floats are automatically converted to Lisp floats. When
124 <function>type-of</> is called on an Alien value that is not automatically
125 converted to a Lisp value, then it will return an <type>alien</> type
131 <sect2><title>Foreign Type Specifiers</>
134 All foreign type names are exported from the <literal>sb-alien</>
135 package. Some foreign type names are also symbols in
136 the <literal>common-lisp</> package, in which case they are
137 reexported from the <literal>sb-alien</> package, so that
138 e.g. it is legal to refer to <type>sb-alien:single-float</>.
142 These are the basic foreign type specifiers:
143 <!-- FIXME: There must be some better way of formatting definitions
144 in DocBook than this. I haven't found it yet, but suggestions
145 or patches would be welcome. -->
149 The foreign type specifier <type>(* foo)</> describes a
150 pointer to an object of type <type>foo</>. A pointed-to type
151 <type>foo</> of <type>t</> indicates a pointer to anything,
152 similar to <type>void *</> in ANSI C. A null alien pointer can
153 be detected with the <function>sb-alien:null-alien</>
159 The foreign type specifier <type>(array foo &optional dimensions)</>
160 describes array of the specified <literal>dimensions</>, holding
161 elements of type <type>foo</>. Note that (unlike in C) <type>(* foo)</>
162 <type>(array foo)}</> are considered to be different types when
163 type checking is done. If equivalence of pointer and array types
164 is desired, it may be explicitly coerced using
165 <function>sb-alien:cast</>.
168 Arrays are accessed using <function>sb-alien:deref</>, passing
169 the indices as additional arguments. Elements are stored in
170 column-major order (as in C), so the first dimension determines
171 only the size of the memory block, and not the layout of the
172 higher dimensions. An array whose first dimension is variable
173 may be specified by using <literal>nil</> as the first dimension.
174 Fixed-size arrays can be allocated as array elements, structure
175 slots or <function>sb-alien:with-alien</> variables. Dynamic
176 arrays can only be allocated using <function>sb-alien:make-alien</>.
181 The foreign type specifier
182 <type>(sb-alien:struct name &rest fields)</>
183 describes a structure type with the specified <varname>name</> and
184 <varname>fields</>. Fields are allocated at the same offsets
185 used by the implementation's C compiler. If <varname>name</>
186 is <literal>nil</> then the structure is anonymous.
189 If a named foreign <type>struct</> specifier is passed to
190 <function>define-alien-type</> or <function>with-alien</>,
191 then this defines, respectively, a new global or local foreign
192 structure type. If no <varname>fields</> are specified, then
193 the fields are taken from the current (local or global) Alien
194 structure type definition of <varname>name</>.
199 The foreign type specifier
200 <type>(sb-alien:union name &rest fields)</>
201 is similar to <type>sb-alien:struct</>, but describes a union type.
202 All fields are allocated at the same offset, and the size of the
203 union is the size of the largest field. The programmer must
204 determine which field is active from context.
209 The foreign type specifier <type>(sb-alien:enum name &rest specs)</>
210 describes an enumeration type that maps between integer values
211 and keywords. If <varname>name</> is <literal>nil</>, then the
212 type is anonymous. Each element of the <varname>specs</>
213 list is either a Lisp keyword, or a list <literal>(keyword value)</>.
214 <varname>value</> is an integer. If <varname>value</> is not
215 supplied, then it defaults to one greater than the value for
216 the preceding spec (or to zero if it is the first spec.)
221 The foreign type specifier <type>(sb-alien:signed &optional bits)</>
222 specifies a signed integer with the specified number of
223 <varname>bits</> precision. The upper limit on integer
224 precision is determined by the machine's word
225 size. If <varname>bits</> is not specified, the maximum
231 The foreign type specifier <type>(integer &optional bits)</> is
232 equivalent to the corresponding type specifier using
233 <type>sb-alien:signed</> instead of <type>integer</>.
238 The foreign type specifier
239 <type>(sb-alien:unsigned &optional bits)</>
240 is like corresponding type specifier using <type>sb-alien:signed</>
241 except that the variable is treated as an unsigned integer.
246 The foreign type specifier <type>(boolean &optional bits)</> is
247 similar to an enumeration type, but maps from Lisp <literal>nil</>
248 and <literal>t</> to C <literal>0</> and <literal>1</>
249 respectively. <varname>bits</> determines the amount of
250 storage allocated to hold the truth value.
255 The foreign type specifier <type>single-float</> describes a
256 floating-point number in IEEE single-precision format.
261 The foreign type specifier <type>double-float</> describes a
262 floating-point number in IEEE double-precision format.
267 The foreign type specifier
268 <type>(function result-type &rest arg-types)</>
269 describes a foreign function that takes arguments of the specified
270 <varname>arg-types</> and returns a result of type <type>result-type</>.
271 Note that the only context where a foreign <type>function</> type
272 is directly specified is in the argument to
273 <function>sb-alien:alien-funcall</>.
274 In all other contexts, foreign functions are represented by
275 foreign function pointer types: <type>(* (function ...))</>.
280 The foreign type specifier <type>sb-alien:system-area-pointer</>
281 describes a pointer which is represented in Lisp as a
282 <type>system-area-pointer</> object. &SBCL; exports this type from
283 <literal>sb-alien</> because &CMUCL; did, but tentatively (as of
284 the first draft of this section of the manual, 2002-07-04) it is
285 deprecated, since it doesn't seem to be required by user code.
290 The foreign type specifier <type>sb-alien:void</> is
291 used in function types to declare that no useful value
292 is returned. Using <function>alien-funcall</>
293 to call a <type>void</> foreign function will return
299 The foreign type specifier <type>sb-alien:c-string</>
300 is similar to \code{(* char)}, but is interpreted as a
301 null-terminated string, and is automatically converted into a
302 Lisp string when accessed; or if the pointer is C <literal>NULL</>
303 or <literal>0</>, then accessing it gives Lisp <literal>nil</>.
306 Assigning a Lisp string to a \code{c-string} structure field or
307 variable stores the contents of the string to the memory already
308 pointed to by that variable. When a foreign object of type
309 <type>(* char)</> is assigned to a <type>c-string</>, then the
310 <type>c-string</> pointer is assigned to. This allows
311 <type>c-string</> pointers to be initialized. For example:
313 (cl:in-package "CL-USER") ; which USEs package "SB-ALIEN"
314 (define-alien-type nil (struct foo (str c-string)))
315 (defun make-foo (str) (let ((my-foo (make-alien (struct foo))))
316 (setf (slot my-foo 'str) (make-alien char (length str))
317 (slot my-foo 'str) str) my-foo))</programlisting>
318 Storing Lisp <literal>NIL</> in a <type>c-string</> writes C
319 <literal>NULL</> to the variable.
324 <literal>sb-alien</> also exports translations of these C type
325 specifiers as foreign type specifiers:
326 <type>sb-alien:char</>,
327 <type>sb-alien:short</>,
328 <type>sb-alien:int</>,
329 <type>sb-alien:long</>,
330 <type>sb-alien:unsigned-char</>,
331 <type>sb-alien:unsigned-short</>,
332 <type>sb-alien:unsigned-int</>,
333 <type>sb-alien:unsigned-long</>,
334 <type>sb-alien:float</>, and
335 <type>sb-alien:double</>.
343 <sect1><title>Operations On Foreign Values</>
344 <!-- AKA "Alien Operations" in the CMU CL manual -->
346 <para>This section describes how to read foreign values as Lisp
347 values, how to coerce foreign values to different kinds of foreign values, and
348 how to dynamically allocate and free foreign variables.</para>
350 <sect2><title>Accessing Foreign Values</>
352 <synopsis>(sb-alien:deref pointer-or-array &rest indices)</>
354 <para>The <function>sb-alien:deref</> function returns the value pointed to by
355 a foreign pointer, or the value of a foreign array element. When
356 dereferencing a pointer, an optional single index can be specified to
357 give the equivalent of C pointer arithmetic; this index is scaled by
358 the size of the type pointed to. When dereferencing an array, the
359 number of indices must be the same as the number of dimensions in the
360 array type. <function>deref</> can be set with <function>setf</> to
361 assign a new value.</para>
363 <synopsis>(sb-alien:slot struct-or-union &rest slot-names)</>
365 <para>The <function>sb-alien:slot</> function extracts the value of
366 the slot named <varname>slot-name</> from a foreign <type>struct</> or
367 <type>union</>. If <varname>struct-or-union</> is a pointer to a
368 structure or union, then it is automatically dereferenced.
369 <function>sb-alien:slot</> can be set with <function>setf</> to assign
370 a new value. Note that <varname>slot-name</> is evaluated, and need
371 not be a compile-time constant (but only constant slot accesses are
372 efficiently compiled.)</para>
376 <sect2><title>Coercing Foreign Values</>
378 <synopsis>(sb-alien:addr alien-expr)</>
381 The <function>sb-alien:addr</> macro
382 returns a pointer to the location specified by
383 <varname>alien-expr</>, which must be either a foreign variable, a use of
384 <function>sb-alien:deref</>, a use of <function>sb-alien:slot</>, or a use of
385 <function>sb-alien:extern-alien</>.
388 <synopsis>(sb-alien:cast foreign-value new-type)</>
390 <para>The <function>sb-alien:cast</>
391 converts <varname>foreign-value</> to a new foreign value with the specified
392 <varname>new-type</>. Both types, old and new, must be foreign pointer,
393 array or function types. Note that the resulting Lisp
394 foreign variable object
395 is not <function>eq</> to the
396 argument, but it does refer to the same foreign data bits.</para>
398 <synopsis>(sb-alien:sap-alien sap type)</>
400 <para>The <function>sb-alien:sap-alien</> function converts <varname>sap</>
401 (a system area pointer) to a foreign value with the specified
402 <varname>type</>. <varname>type</> is not evaluated.
403 As of 2002-07-04, it looks as though this and other SAP functionality
404 may become deprecated, since it shouldn't be needed by user code.
407 <para>The <varname>type</> must be some foreign pointer, array, or
410 <synopsis>(sb-alien:alien-sap foreign-value type)</>
412 <para>The <function>sb-alien:alien-sap</> function
413 returns the SAP which points to \var{alien-value}'s data.
414 As of 2002-07-04, it looks as though this and other SAP functionality
415 may become deprecated, since it shouldn't be needed by user code.
418 <para>The <varname>foreign-value</> must be of some foreign pointer,
419 array, or record type.</para>
423 <sect2><title>Foreign Dynamic Allocation</>
425 <para>Lisp code can call the C standard library functions
426 <function>malloc</> and <function>free</> to dynamically allocate and
427 deallocate foreign variables. The Lisp code shares the same allocator
428 with foreign C code, so it's OK for foreign code to call
429 <function>free</> on the result of Lisp
430 <function>sb-alien:make-alien</>, or for Lisp code to call
431 <function>sb-alien:free-alien</> on foreign objects allocated by C
434 <synopsis>(sb-alien:make-alien type size)</>
436 <para>The <function>sb-alien:make-alien</> macro
437 returns a dynamically allocated foreign value of the specified
438 <varname>type</> (which is not evaluated.) The allocated memory is not
439 initialized, and may contain arbitrary junk. If supplied,
440 <varname>size</> is an expression to evaluate to compute the size of the
441 allocated object. There are two major cases:
444 <para>When <varname>type</> is a foreign array type, an array of
445 that type is allocated and a pointer to it is returned. Note that you
446 must use <function>deref</> to change the result to an array before you
447 can use <function>deref</> to read or write elements:
449 (cl:in-package "CL-USER") ; which USEs package "SB-ALIEN"
450 (defvar *foo* (make-alien (array char 10)))
451 (type-of *foo*) => (alien (* (array (signed 8) 10)))
452 (setf (deref (deref foo) 0) 10) => 10</programlisting>
453 If supplied, <varname>size</> is used as the first dimension for the
457 <para>When <varname>type</> is any other foreign type, then an
458 object for that type is allocated, and a pointer to it is
459 returned. So <function>(make-alien int)</> returns a <type>(* int)</>.
460 If <varname>size</> is specified, then a block of that many
461 objects is allocated, with the result pointing to the first one.</para>
466 <synopsis>(sb-alien:free-alien foreign-value)</>
468 <para>The <function>sb-alien:free-alien</> function
469 frees the storage for <varname>foreign-value</>,
470 which must have been allocated with Lisp <function>make-alien</>
471 or C <function>malloc</>.</para>
473 <para>See also the <function>sb-alien:with-alien</> macro, which
474 allocates foreign values on the stack.</para>
478 <sect1><title>Foreign Variables</>
479 <!-- AKA "Alien Variables" in the CMU CL manual -->
480 <para>(TO DO: Update corresponding section of &CMUCL; manual.)</para>
483 <sect1><title>Foreign Data Structure Example</>
484 <!-- AKA "Alien Data Structure Example" in the CMU CL manual -->
485 <para>(TO DO: Update corresponding section of &CMUCL; manual.)</para>
488 <sect1><title>Loading Unix Object Files</>
489 <para>(TO DO: Update corresponding section of &CMUCL; manual.)</para>
492 <sect1><title>Foreign Function Calls</>
493 <!-- AKA "Alien Function Calls" in the CMU CL manual -->
494 <para>(TO DO: Update corresponding section of &CMUCL; manual.)</para>
497 <sect1><title>Step-by-Step Example of the Foreign Function Interface</>
498 <!-- AKA "Step-by-Step Alien Example" in the CMU CL manual -->
499 <para>(TO DO: Update corresponding section of &CMUCL; manual.)</para>