1 <chapter id="ffi"><title>The Foreign Function Interface</>
3 <para>This chapter describes &SBCL;'s interface to C programs and
4 libraries (and, since C interfaces are a sort of <foreignphrase>lingua
5 franca</> of the Unix world, to other programs and libraries in
8 <note><para>In the modern Lisp world, the usual term for this
9 functionality is Foreign Function Interface, or <acronym>FFI</>, where
10 despite the mention of <quote>function</> in this term, <acronym>FFI</> also
11 refers to direct manipulation of C data structures as well as
12 functions. The traditional &CMUCL; terminology is Alien Interface, and
13 while that older terminology is no longer used much in the system
14 documentation, it still reflected in names in the
15 implementation, notably in the name of the <literal>SB-ALIEN</>
16 package.</para></note>
18 <sect1><title>Introduction to the Foreign Function Interface</>
19 <!-- AKA "Introduction to Aliens" in the CMU CL manual -->
22 Because of Lisp's emphasis on dynamic memory allocation and garbage
23 collection, Lisp implementations use non-C-like memory representations
24 for objects. This representation mismatch creates friction when a Lisp
25 program must share objects with programs which expect C data. There
26 are three common approaches to establishing communication:
28 <listitem><para>The burden can be placed on the foreign program
29 (and programmer) by requiring the knowledge and use of the
30 representations used internally by the Lisp implementation.
31 This can require a considerable amount of <quote>glue</> code on the
32 C side, and that code tends to be sensitively dependent on the
33 internal implementation details of the Lisp system.</para></listitem>
34 <listitem><para>The Lisp system can automatically convert objects
35 back and forth between the Lisp and foreign representations.
36 This is convenient, but translation becomes prohibitively slow
37 when large or complex data structures must be shared. This approach
38 is supported by the &SBCL; <acronym>FFI</>, and used automatically
39 by the when passing integers and strings.</para></listitem>
40 <listitem><para>The Lisp program can directly manipulate foreign
41 objects through the use of extensions to the Lisp language.
46 <para>&SBCL;, like &CMUCL; before it, relies primarily on the
47 automatic conversion and direct manipulation approaches. The SB-ALIEN
48 package provices a facility wherein foreign values of simple scalar
49 types are automatically converted and complex types are directly
50 manipulated in their foreign representation. Additionally the
51 lower-level System Area Pointers (or SAPs) can be used where
52 necessary to provide untyped access to foreign memory.</para>
54 <para>Any foreign objects that can't automatically be converted into
55 Lisp values are represented by objects of type <type>alien-value</>.
56 Since Lisp is a dynamically typed language, even foreign objects must
57 have a run-time type; this type information is provided by
58 encapsulating the raw pointer to the foreign data within an
59 <type>alien-value</> object.</para>
61 <para>The type language and operations on foreign types are
62 intentionally similar to those of the C language.</para>
66 <sect1><title>Foreign Types</>
67 <!-- AKA "Alien Types" in the CMU CL manual -->
69 <para>Alien types have a description language based on nested list
70 structure. For example the C type
71 <programlisting>struct foo {
75 has the corresponding &SBCL; FFI type
76 <programlisting>(struct foo
78 (b (array (* (struct foo)) 100)))</programlisting>
81 <sect2><title>Defining Foreign Types</>
84 Types may be either named or anonymous. With structure and union
85 types, the name is part of the type specifier, allowing recursively
86 defined types such as:
87 <programlisting>(struct foo (a (* (struct foo))))</programlisting>
88 An anonymous structure or union type is specified by using the name
89 <literal>nil</>. The <function>with-alien</> macro defines a local
90 scope which <quote>captures</> any named type definitions. Other types
91 are not inherently named, but can be given named abbreviations using
92 the <function>define-alien-type</> macro.
97 <sect2><title>Foreign Types and Lisp Types</>
100 The foreign types form a subsystem of the &SBCL; type system. An
101 <type>alien</> type specifier provides a way to use any foreign type as a
102 Lisp type specifier. For example,
103 <programlisting>(typep foo '(alien (* int)))</programlisting>
104 can be used to determine whether <varname>foo</> is a pointer to a foreign
105 <type>int</>. <type>alien</> type specifiers can be used in the same ways
106 as ordinary Lisp type specifiers (like <type>string</>.) Alien type
107 declarations are subject to the same
108 precise type checking <!-- FIXME: should be linked to id="precisetypechecking" -->
109 as any other declaration.
113 Note that the type identifiers used in the
114 foreign type system overlap with native Lisp type
115 specifiers in some cases. For example, the type specifier
116 <type>(alien single-float)</type> is identical to <type>single-float</>, since
117 foreign floats are automatically converted to Lisp floats. When
118 <function>type-of</> is called on an alien value that is not automatically
119 converted to a Lisp value, then it will return an <type>alien</> type
125 <sect2><title>Foreign Type Specifiers</>
128 All foreign type names are exported from the <literal>sb-alien</>
129 package. Some foreign type names are also symbols in
130 the <literal>common-lisp</> package, in which case they are
131 reexported from the <literal>sb-alien</> package, so that
132 e.g. it is legal to refer to <type>sb-alien:single-float</>.
136 These are the basic foreign type specifiers:
137 <!-- FIXME: There must be some better way of formatting definitions
138 in DocBook than this. I haven't found it yet, but suggestions
139 or patches would be welcome. -->
143 The foreign type specifier <type>(* foo)</> describes a
144 pointer to an object of type <type>foo</>. A pointed-to type
145 <type>foo</> of <type>t</> indicates a pointer to anything,
146 similar to <type>void *</> in ANSI C. A null alien pointer can
147 be detected with the <function>sb-alien:null-alien</>
153 The foreign type specifier <type>(array foo &optional dimensions)</>
154 describes array of the specified <literal>dimensions</>, holding
155 elements of type <type>foo</>. Note that (unlike in C) <type>(* foo)</>
156 <type>(array foo)}</> are considered to be different types when
157 type checking is done. If equivalence of pointer and array types
158 is desired, it may be explicitly coerced using
159 <function>sb-alien:cast</>.
162 Arrays are accessed using <function>sb-alien:deref</>, passing
163 the indices as additional arguments. Elements are stored in
164 column-major order (as in C), so the first dimension determines
165 only the size of the memory block, and not the layout of the
166 higher dimensions. An array whose first dimension is variable
167 may be specified by using <literal>nil</> as the first dimension.
168 Fixed-size arrays can be allocated as array elements, structure
169 slots or <function>sb-alien:with-alien</> variables. Dynamic
170 arrays can only be allocated using <function>sb-alien:make-alien</>.
175 The foreign type specifier
176 <type>(sb-alien:struct name &rest fields)</>
177 describes a structure type with the specified <varname>name</> and
178 <varname>fields</>. Fields are allocated at the same offsets
179 used by the implementation's C compiler. If <varname>name</>
180 is <literal>nil</> then the structure is anonymous.
183 If a named foreign <type>struct</> specifier is passed to
184 <function>define-alien-type</> or <function>with-alien</>,
185 then this defines, respectively, a new global or local foreign
186 structure type. If no <varname>fields</> are specified, then
187 the fields are taken from the current (local or global) alien
188 structure type definition of <varname>name</>.
193 The foreign type specifier
194 <type>(sb-alien:union name &rest fields)</>
195 is similar to <type>sb-alien:struct</>, but describes a union type.
196 All fields are allocated at the same offset, and the size of the
197 union is the size of the largest field. The programmer must
198 determine which field is active from context.
203 The foreign type specifier <type>(sb-alien:enum name &rest specs)</>
204 describes an enumeration type that maps between integer values
205 and keywords. If <varname>name</> is <literal>nil</>, then the
206 type is anonymous. Each element of the <varname>specs</>
207 list is either a Lisp keyword, or a list <literal>(keyword value)</>.
208 <varname>value</> is an integer. If <varname>value</> is not
209 supplied, then it defaults to one greater than the value for
210 the preceding spec (or to zero if it is the first spec.)
215 The foreign type specifier <type>(sb-alien:signed &optional bits)</>
216 specifies a signed integer with the specified number of
217 <varname>bits</> precision. The upper limit on integer
218 precision is determined by the machine's word
219 size. If <varname>bits</> is not specified, the maximum
225 The foreign type specifier <type>(integer &optional bits)</> is
226 equivalent to the corresponding type specifier using
227 <type>sb-alien:signed</> instead of <type>integer</>.
232 The foreign type specifier
233 <type>(sb-alien:unsigned &optional bits)</>
234 is like corresponding type specifier using <type>sb-alien:signed</>
235 except that the variable is treated as an unsigned integer.
240 The foreign type specifier <type>(boolean &optional bits)</> is
241 similar to an enumeration type, but maps from Lisp <literal>nil</>
242 and <literal>t</> to C <literal>0</> and <literal>1</>
243 respectively. <varname>bits</> determines the amount of
244 storage allocated to hold the truth value.
249 The foreign type specifier <type>single-float</> describes a
250 floating-point number in IEEE single-precision format.
255 The foreign type specifier <type>double-float</> describes a
256 floating-point number in IEEE double-precision format.
261 The foreign type specifier
262 <type>(function result-type &rest arg-types)</>
263 describes a foreign function that takes arguments of the specified
264 <varname>arg-types</> and returns a result of type <type>result-type</>.
265 Note that the only context where a foreign <type>function</> type
266 is directly specified is in the argument to
267 <function>sb-alien:alien-funcall</>.
268 In all other contexts, foreign functions are represented by
269 foreign function pointer types: <type>(* (function ...))</>.
274 The foreign type specifier <type>sb-alien:system-area-pointer</>
275 describes a pointer which is represented in Lisp as a
276 <type>system-area-pointer</> object. &SBCL; exports this type from
277 <literal>sb-alien</> because &CMUCL; did, but tentatively (as of
278 the first draft of this section of the manual, &SBCL; 0.7.6) it is
279 deprecated, since it doesn't seem to be required by user code.
284 The foreign type specifier <type>sb-alien:void</> is
285 used in function types to declare that no useful value
286 is returned. Using <function>alien-funcall</>
287 to call a <type>void</> foreign function will return
293 The foreign type specifier <type>sb-alien:c-string</>
294 is similar to <type>(* char)</>, but is interpreted as a
295 null-terminated string, and is automatically converted into a
296 Lisp string when accessed; or if the pointer is C <literal>NULL</>
297 or <literal>0</>, then accessing it gives Lisp <literal>nil</>.
298 Lisp strings are stored with a trailing NUL termination, so no
299 copying (either by the user or the implementation) is necessary
300 when passing them to foreign code.
303 Assigning a Lisp string to a <type>c-string</> structure field or
304 variable stores the contents of the string to the memory already
305 pointed to by that variable. When a foreign object of type
306 <type>(* char)</> is assigned to a <type>c-string</>, then the
307 <type>c-string</> pointer is assigned to. This allows
308 <type>c-string</> pointers to be initialized. For example:
309 <programlisting>(cl:in-package "CL-USER") ; which USEs package "SB-ALIEN"
310 (define-alien-type nil (struct foo (str c-string)))
311 (defun make-foo (str) (let ((my-foo (make-alien (struct foo))))
312 (setf (slot my-foo 'str) (make-alien char (length str))
313 (slot my-foo 'str) str) my-foo))</programlisting>
314 Storing Lisp <literal>NIL</> in a <type>c-string</> writes C
315 <literal>NULL</> to the variable.
320 <literal>sb-alien</> also exports translations of these C type
321 specifiers as foreign type specifiers:
322 <type>sb-alien:char</>,
323 <type>sb-alien:short</>,
324 <type>sb-alien:int</>,
325 <type>sb-alien:long</>,
326 <type>sb-alien:unsigned-char</>,
327 <type>sb-alien:unsigned-short</>,
328 <type>sb-alien:unsigned-int</>,
329 <type>sb-alien:unsigned-long</>,
330 <type>sb-alien:float</>, and
331 <type>sb-alien:double</>.
343 <sect1><title>Operations On Foreign Values</>
344 <!-- AKA "Alien Operations" in the CMU CL manual -->
346 <para>This section describes how to read foreign values as Lisp
347 values, how to coerce foreign values to different kinds of foreign values, and
348 how to dynamically allocate and free foreign variables.</para>
350 <sect2><title>Accessing Foreign Values</>
352 <synopsis>(sb-alien:deref pointer-or-array &rest indices)</>
354 <para>The <function>sb-alien:deref</> function returns the value pointed to by
355 a foreign pointer, or the value of a foreign array element. When
356 dereferencing a pointer, an optional single index can be specified to
357 give the equivalent of C pointer arithmetic; this index is scaled by
358 the size of the type pointed to. When dereferencing an array, the
359 number of indices must be the same as the number of dimensions in the
360 array type. <function>deref</> can be set with <function>setf</> to
361 assign a new value.</para>
363 <synopsis>(sb-alien:slot struct-or-union &rest slot-names)</>
365 <para>The <function>sb-alien:slot</> function extracts the value of
366 the slot named <varname>slot-name</> from a foreign <type>struct</> or
367 <type>union</>. If <varname>struct-or-union</> is a pointer to a
368 structure or union, then it is automatically dereferenced.
369 <function>sb-alien:slot</> can be set with <function>setf</> to assign
370 a new value. Note that <varname>slot-name</> is evaluated, and need
371 not be a compile-time constant (but only constant slot accesses are
372 efficiently compiled.)</para>
374 <sect3><title>Untyped memory</>
376 <para>As noted at the beginning of the chapter, the System Area
377 Pointer facilities allow untyped access to foreign memory. SAPs can
378 be converted to and from the usual typed foreign values using
379 <function>sap-alien</function> and <function>alien-sap</function>
380 (described elsewhere), and also to and from integers - raw machine
381 addresses. They should thus be used with caution; corrupting the Lisp
382 heap or other memory with SAPs is trivial.</para>
384 <synopsis>(sb-sys:int-sap machine-address)</>
386 <para>Creates a SAP pointing at the virtual address
387 <varname>machine-address</varname>. </para>
389 <synopsis>(sb-sys:sap-ref-32 sap offset)</>
391 <para>Access the value of the memory location at
392 <varname>offset</varname> bytes from <varname>sap</varname>. This form
393 may also be used with <function>setf</function> to alter the memory at
394 that location.</para>
396 <synopsis>(sb-sys:sap= sap1 sap2)</>
398 <para>Compare <varname>sap1</varname> and <varname>sap2</varname> for
401 <para>Similarly named functions exist for accessing other sizes of
402 word, other comparisons, and other conversions. The reader is invited
403 to use <function>apropos</function> and <function>describe</function>
404 for more details</para>
406 (apropos "sap" :sb-sys)
410 <sect2><title>Coercing Foreign Values</>
412 <synopsis>(sb-alien:addr alien-expr)</>
415 The <function>sb-alien:addr</> macro
416 returns a pointer to the location specified by
417 <varname>alien-expr</>, which must be either a foreign variable, a use of
418 <function>sb-alien:deref</>, a use of <function>sb-alien:slot</>, or a use of
419 <function>sb-alien:extern-alien</>.
422 <synopsis>(sb-alien:cast foreign-value new-type)</>
424 <para>The <function>sb-alien:cast</>
425 converts <varname>foreign-value</> to a new foreign value with the specified
426 <varname>new-type</>. Both types, old and new, must be foreign pointer,
427 array or function types. Note that the resulting Lisp
428 foreign variable object
429 is not <function>eq</> to the
430 argument, but it does refer to the same foreign data bits.</para>
432 <synopsis>(sb-alien:sap-alien sap type)</>
434 <para>The <function>sb-alien:sap-alien</> function converts <varname>sap</>
435 (a system area pointer) to a foreign value with the specified
436 <varname>type</>. <varname>type</> is not evaluated.
439 <para>The <varname>type</> must be some foreign pointer, array, or
442 <synopsis>(sb-alien:alien-sap foreign-value type)</>
444 <para>The <function>sb-alien:alien-sap</> function
445 returns the SAP which points to <varname>alien-value</>'s data.
448 <para>The <varname>foreign-value</> must be of some foreign pointer,
449 array, or record type.</para>
453 <sect2><title>Foreign Dynamic Allocation</>
455 <para>Lisp code can call the C standard library functions
456 <function>malloc</> and <function>free</> to dynamically allocate and
457 deallocate foreign variables. The Lisp code shares the same allocator
458 with foreign C code, so it's OK for foreign code to call
459 <function>free</> on the result of Lisp
460 <function>sb-alien:make-alien</>, or for Lisp code to call
461 <function>sb-alien:free-alien</> on foreign objects allocated by C
464 <synopsis>(sb-alien:make-alien type size)</>
466 <para>The <function>sb-alien:make-alien</> macro
467 returns a dynamically allocated foreign value of the specified
468 <varname>type</> (which is not evaluated.) The allocated memory is not
469 initialized, and may contain arbitrary junk. If supplied,
470 <varname>size</> is an expression to evaluate to compute the size of the
471 allocated object. There are two major cases:
474 <para>When <varname>type</> is a foreign array type, an array of
475 that type is allocated and a pointer to it is returned. Note that you
476 must use <function>deref</> to change the result to an array before you
477 can use <function>deref</> to read or write elements:
479 (cl:in-package "CL-USER") ; which USEs package "SB-ALIEN"
480 (defvar *foo* (make-alien (array char 10)))
481 (type-of *foo*) => (alien (* (array (signed 8) 10)))
482 (setf (deref (deref foo) 0) 10) => 10</programlisting>
483 If supplied, <varname>size</> is used as the first dimension for the
487 <para>When <varname>type</> is any other foreign type, then an
488 object for that type is allocated, and a pointer to it is
489 returned. So <function>(make-alien int)</> returns a <type>(* int)</>.
490 If <varname>size</> is specified, then a block of that many
491 objects is allocated, with the result pointing to the first one.</para>
496 <synopsis>(sb-alien:free-alien foreign-value)</>
498 <para>The <function>sb-alien:free-alien</> function
499 frees the storage for <varname>foreign-value</>,
500 which must have been allocated with Lisp <function>make-alien</>
501 or C <function>malloc</>.</para>
503 <para>See also the <function>sb-alien:with-alien</> macro, which
504 allocates foreign values on the stack.</para>
510 <sect1><title>Foreign Variables</>
511 <!-- AKA "Alien Variables" in the CMU CL manual -->
514 Both local (stack allocated) and external (C global) foreign variables are
518 <sect2><title>Local Foreign Variables</>
520 <synopsis>(sb-alien:with-alien var-definitions &body body)</>
522 <para>The <function>with-alien</>
523 macro establishes local
526 alien types and names.
527 This form is analogous to defining a local variable in C: additional
528 storage is allocated, and the initial value is copied.
530 analogous to LET-allocated Lisp variables, since the variables
531 can't be captured in closures: they live only for the dynamic extent
532 of the body, and referring to them outside is a gruesome error.
535 <para>The <varname>var-definitions</> argument is a list of
536 variable definitions, each of the form
537 <programlisting>(name type &optional initial-value)</programlisting>
538 The names of the variables are established as symbol-macros; the bindings have
539 lexical scope, and may be assigned with <function>setq</>
540 or <function>setf</>.
543 <para>The <function>with-alien</> macro also establishes
544 a new scope for named structures
545 and unions. Any <varname>type</> specified for a variable may contain
546 named structure or union types with the slots specified. Within the
547 lexical scope of the binding specifiers and body, a locally defined
548 foreign structure type <type>foo</> can be referenced by its name using
549 <type>(struct foo)</>.
554 <sect2><title>External Foreign Variables</>
557 External foreign names are strings, and Lisp names are symbols. When
558 an external foreign value is represented using a Lisp variable, there
559 must be a way to convert from one name syntax into the other. The
560 macros <function>extern-alien</>, <function>define-alien-variable</> and
561 <function>define-alien-routine</> use this conversion heuristic:
563 <listitem><para>Alien names are converted to Lisp names by uppercasing and
564 replacing underscores with hyphens.</para></listitem>
565 <listitem><para>Conversely, Lisp names are converted to alien names by
566 lowercasing and replacing hyphens with underscores.</para></listitem>
567 <listitem><para>Both the Lisp symbol and alien string names may be
568 separately specified by using a list of the form
569 <programlisting>(alien-string lisp-symbol)</></para></listitem>
573 <synopsis>(sb-alien:define-alien-variable name type)</>
576 The <function>define-alien-variable</> macro
577 defines <varname>name</> as an external foreign variable of the
578 specified foreign <type>type</>. <varname>name</> and <type>type</> are not
579 evaluated. The Lisp name of the variable (see above) becomes a
580 global alien variable. Global alien variables
581 are effectively ``global symbol macros''; a reference to the
582 variable fetches the contents of the external variable. Similarly,
583 setting the variable stores new contents---the new contents must be
584 of the declared <type>type</>. Someday, they may well be implemented
585 using the &ANSI; <function>define-symbol-macro</> mechanism, but
586 as of &SBCL; 0.7.5, they are still implemented using an older
587 more-or-less parallel mechanism inherited from &CMUCL;.
591 For example, to access a C-level counter <varname>foo</>, one could
594 (define-alien-variable "foo" int)
595 ;; Now it is possible to get the value of the C variable foo simply by
596 ;; referencing that Lisp variable:
599 (incf foo)</programlisting>
602 <synopsis>(sb-alien:get-errno)</>
605 Since in modern C libraries, the <varname>errno</> "variable" is typically
606 no longer a variable, but some bizarre artificial construct
607 which behaves superficially like a variable within a given thread,
608 it can no longer reliably be accessed through the ordinary
609 <varname>define-alien-variable</> mechanism. Instead, &SBCL; provides
610 the operator <function>sb-alien:get-errno</> to allow Lisp code to read it.
613 <synopsis>(sb-alien:extern-alien name type)</>
616 The <function>extern-alien</> macro
617 returns an alien with the specified <type>type</> which
618 points to an externally defined value. <varname>name</> is not evaluated,
619 and may be either a string or a symbol. <type>type</> is
620 an unevaluated alien type specifier.
627 <sect1><title>Foreign Data Structure Examples</>
628 <!-- AKA "Alien Data Structure Example" in the CMU CL manual -->
631 Now that we have alien types, operations and variables, we can manipulate
632 foreign data structures. This C declaration
638 can be translated into the following alien type:
639 <programlisting>(define-alien-type nil
642 (b (array (* (struct foo)) 100))))</programlisting>
646 Once the <type>foo</> alien type has been defined as above,
650 f.b[7].a</programlisting>
651 can be translated in this way:
653 (with-alien ((f (struct foo)))
654 (slot (deref (slot f 'b) 7) 'a)
656 ;; Do something with f...
661 Or consider this example of an external C variable and some accesses:
669 extern struct c_struct *my_struct;
672 my_struct = my_struct->n;</programlisting>
673 which can be manipulated in Lisp like this:
675 (define-alien-type nil
683 (define-alien-variable "my_struct" (* c-struct))
684 (incf (slot my-struct 'x))
685 (setf (slot my-struct 'a) 5)
686 (setq my-struct (slot my-struct 'n))</programlisting>
691 <sect1><title>Loading Unix Object Files</>
694 Foreign object files can be loaded into the running Lisp process by
695 calling the functions <function>load-foreign</> or
696 <function>load-1-foreign</>.
699 <para> The <function>sb-alien:load-1-foreign</> function is the more
700 primitive of the two operations. It loads a single object file. into
701 the currently running Lisp. The external symbols defining routines and
702 variables are made available for future external references (e.g. by
703 <function>extern-alien</>). Forward references to foreign symbols
704 aren't supported: <function>load-1-foreign</> must be run before any
705 of the defined symbols are referenced.
708 <para><function>sb-alien:load-foreign</> is built in terms of
709 <function>load-1-foreign</> and some other machinery
710 like <function>sb-ext:run-program</>.
711 It accepts a list of files and libraries,
712 and runs the linker on the files and
713 libraries, creating an absolute Unix object file which is then
714 processed by <function>load-1-foreign</>.</para>
716 <note><para>As of &SBCL; 0.7.5, all foreign code (code loaded
717 with <function>load-1-function</> or <function>load-function</>) is
718 lost when a Lisp core is saved with
719 <function>sb-ext:save-lisp-and-die</>, and no attempt is made to
720 restore it when the core is loaded. Historically this has been an
721 annoyance both for &SBCL; users and for &CMUCL; users.
722 It's hard to solve this problem completely cleanly, but some
723 generally-reliable partial solution might be useful. Once someone in
724 either camp gets sufficiently annoyed to create it, &SBCL; is
725 likely to adopt some mechanism for automatically restoring foreign
726 code when a saved core is loaded.</para></note>
730 <sect1><title>Foreign Function Calls</>
733 The foreign function call interface allows a Lisp program to call
734 many functions written in languages that use the C calling convention.
738 Lisp sets up various signal handling routines and other environment
739 information when it first starts up, and expects these to be in place
740 at all times. The C functions called by Lisp should not change the
741 environment, especially the signal handlers: the signal handlers
742 installed by Lisp typically have interesting flags set (e.g to request
743 machine context information, or for signal delivery on an alternate
744 stack) which the Lisp runtime relies on for correct operation.
745 Precise details of how this works may change without notice between
746 versions; the source, or the brain of a friendly &SBCL; developer,
747 is the only documentation. Users of a Lisp built with the :sb-thread
748 feature should also read the Threading section
749 <!-- FIXME I'm sure docbook has some syntax for internal links -->
750 of this manual</para>
752 <sect2><title>The <function>alien-funcall</> Primitive</title>
754 <synopsis>(sb-alien:alien-funcall alien-function &rest arguments)</>
757 The <function>alien-funcall</> function is the foreign function call
758 primitive: <varname>alien-function</> is called with the supplied
759 <varname>arguments</> and its C return value is returned as a Lisp value.
760 The <varname>alien-function</> is an arbitrary
761 run-time expression; to refer to a constant function, use
762 <function>extern-alien</> or a value defined by
763 <function>define-alien-routine</>.
767 The type of <function>alien-function</>
768 must be <type>(alien (function ...))</>
769 or <type>(alien (* (function ...)))</>.
770 The function type is used to
771 determine how to call the function (as though it was declared with
772 a prototype.) The type need not be known at compile time, but only
773 known-type calls are efficiently compiled. Limitations:
775 <listitem><para>Structure type return values are not implemented.</></>
776 <listitem><para>Passing of structures by value is not implemented.</></>
781 Here is an example which allocates a <type>(struct foo)</>, calls a foreign
782 function to initialize it, then returns a Lisp vector of all the
783 <type>(* (struct foo))</> objects filled in by the foreign call:
785 ;; Allocate a foo on the stack.
786 (with-alien ((f (struct foo)))
787 ;; Call some C function to fill in foo fields.
788 (alien-funcall (extern-alien "mangle_foo" (function void (* foo)))
790 ;; Find how many foos to use by getting the A field.
791 (let* ((num (slot f 'a))
792 (result (make-array num)))
793 ;; Get a pointer to the array so that we don't have to keep extracting it:
794 (with-alien ((a (* (array (* (struct foo)) 100)) (addr (slot f 'b))))
795 ;; Loop over the first N elements and stash them in the result vector.
797 (setf (svref result i) (deref (deref a) i)))
799 result)))</programlisting>
804 <sect2><title>The <function>define-alien-routine</> Macro</>
806 <synopsis>(sb-alien:define-alien-routine} name result-type &rest arg-specifiers)</>
809 The <function>define-alien-routine</> macro is a convenience
810 for automatically generating Lisp
811 interfaces to simple foreign functions. The primary feature is the
812 parameter style specification, which translates the C
813 pass-by-reference idiom into additional return values.
817 <varname>name</> is usually a string external symbol, but may also be a
818 symbol Lisp name or a list of the foreign name and the Lisp name.
819 If only one name is specified, the other is automatically derived
820 as for <function>extern-alien</>.
821 <varname>result-type</> is the alien type of the return value.
825 Each element of the <varname>arg-specifiers</> list
826 specifies an argument to the foreign function, and is
828 <programlisting>(aname atype &optional style)</programlisting>
829 <varname>aname</> is the symbol name of the argument to the constructed
830 function (for documentation). <varname>atype</> is the alien type of
831 corresponding foreign argument. The semantics of the actual call
832 are the same as for <function>alien-funcall</>. <varname>style</>
833 specifies how this argument should be handled at call and return time,
834 and should be one of the following
836 <listitem><para><varname>:in</>specifies that the argument is
837 passed by value. This is the default. <varname>:in</> arguments
838 have no corresponding return value from the Lisp function.
840 <listitem><para><varname>:copy</> is similar to <varname>:in</>,
841 but the argument is copied
842 to a pre-allocated object and a pointer to this object is passed
843 to the foreign routine.</para></listitem>
844 <listitem><para><varname>:out</> specifies a pass-by-reference
845 output value. The type of the argument must be a pointer to
846 a fixed-sized object (such as an integer or pointer).
847 <varname>:out</> and <varname>:in-out</> style cannot
848 be used with pointers to arrays, records or functions. An
849 object of the correct size is allocated on the stack, and
850 its address is passed to the foreign function. When the
851 function returns, the contents
852 of this location are returned as one of the values of the Lisp
853 function (and the location is automatically deallocated).
855 <listitem><para><varname>:in-out</> is a combination of
856 <varname>:copy</> and <varname>:out</>.
857 The argument is copied to a pre-allocated object and a pointer to
858 this object is passed to the foreign routine. On return, the
859 contents of this location is returned as an additional value.
866 Any efficiency-critical foreign interface function should be inline
867 expanded, which can be done by preceding the
868 <function>define-alien-routine</> call with:
869 <programlisting>(declaim (inline lisp-name))</programlisting>
870 In addition to avoiding the Lisp call overhead, this allows
871 pointers, word-integers and floats to be passed using non-descriptor
872 representations, avoiding consing.)
878 <sect2><title><function>define-alien-routine</> Example</title>
881 Consider the C function <function>cfoo</>
882 with the following calling convention:
887 char *a; /* update */
890 /* body of cfoo(...) */
892 This can be described by the following call to
893 <function>define-alien-routine</>:
895 (define-alien-routine "cfoo" void
898 (i int :out))</programlisting>
899 The Lisp function <function>cfoo</> will have
900 two arguments (<varname>str</> and <varname>a</>)
901 and two return values (<varname>a</> and <varname>i</>).
906 <sect2><title>Calling Lisp From C</>
909 Calling Lisp functions from C is sometimes possible, but is extremely
910 hackish and poorly supported as of &SBCL; 0.7.5.
911 See <function>funcall0</> ... <function>funcall3</> in
912 the runtime system. The
913 arguments must be valid &SBCL; object descriptors (so that
915 left-shifted by 2.) As of &SBCL; 0.7.5, the format
916 of object descriptors is documented only by the source code and, in parts,
917 by the old &CMUCL; "INTERNALS" documentation.</para>
919 <para> Note that the garbage collector moves objects, and won't be
920 able to fix up any references in C variables. There are three
921 mechanisms for coping with this:
924 <listitem><para>The <function>sb-ext:purify</> moves all live Lisp
925 data into static or read-only areas such that it will never be moved
926 (or freed) again in the life of the Lisp session</para></listitem>
928 <listitem><para><function>sb-sys:with-pinned-objects</function> is a
929 macro which arranges for some set of objects to be pinned in memory
930 for the dynamic extent of its body forms. On ports which use the
931 generational garbage collector (as of &SBCL; 0.8.3, only the x86) this
932 has a page granularity - i.e. the entire 4k page or pages containing
933 the objects will be locked down. On other ports it is implemented by
934 turning off GC for the duration (so could be said to have a
935 whole-world granularity). </para></listitem>
937 <listitem><para>Disable GC, using the <function>without-gcing</function>
938 macro or <function>gc-off</function> call.</para></listitem>
943 <!-- FIXME: This is a "changebar" section from the CMU CL manual.
944 I (WHN 2002-07-14) am not very familiar with this content, so
945 I'm not immediately prepared to try to update it for SBCL, and
946 I'm not feeling masochistic enough to work to encourage this
947 kind of low-level hack anyway. However, I acknowledge that callbacks
948 are sometimes really really necessary, so I include the original
949 text in case someone is hard-core enough to benefit from it. If
950 anyone brings the information up to date for SBCL, it belong
951 either in the main manual or on a CLiki SBCL Internals page.
952 LaTeX \subsection{Accessing Lisp Arrays}
954 LaTeX Due to the way \cmucl{} manages memory, the amount of memory that can
955 LaTeX be dynamically allocated by \code{malloc} or \funref{make-alien} is
956 LaTeX limited\footnote{\cmucl{} mmaps a large piece of memory for it's own
957 LaTeX use and this memory is typically about 8 MB above the start of the C
958 LaTeX heap. Thus, only about 8 MB of memory can be dynamically
961 Empirically determined to be considerably >8Mb on this x86 linux
962 machine, but I don't know what the actual values are - dan 2003.09.01
964 Note that this technique is used in SB-GROVEL in the SBCL contrib
967 LaTeX To overcome this limitation, it is possible to access the content of
968 LaTeX Lisp arrays which are limited only by the amount of physical memory
969 LaTeX and swap space available. However, this technique is only useful if
970 LaTeX the foreign function takes pointers to memory instead of allocating
971 LaTeX memory for itself. In latter case, you will have to modify the
972 LaTeX foreign functions.
974 LaTeX This technique takes advantage of the fact that \cmucl{} has
975 LaTeX specialized array types (\pxlref{specialized-array-types}) that match
976 LaTeX a typical C array. For example, a \code{(simple-array double-float
977 LaTeX (100))} is stored in memory in essentially the same way as the C
978 LaTeX array \code{double x[100]} would be. The following function allows us
979 LaTeX to get the physical address of such a Lisp array:
980 LaTeX \begin{example}
981 LaTeX (defun array-data-address (array)
982 LaTeX "Return the physical address of where the actual data of an array is
985 LaTeX ARRAY must be a specialized array type in CMU Lisp. This means ARRAY
986 LaTeX must be an array of one of the following types:
990 LaTeX (unsigned-byte 32)
991 LaTeX (unsigned-byte 16)
992 LaTeX (unsigned-byte 8)
993 LaTeX (signed-byte 32)
994 LaTeX (signed-byte 16)
995 LaTeX (signed-byte 8)
997 LaTeX (declare (type (or #+signed-array (array (signed-byte 8))
998 LaTeX #+signed-array (array (signed-byte 16))
999 LaTeX #+signed-array (array (signed-byte 32))
1000 LaTeX (array (unsigned-byte 8))
1001 LaTeX (array (unsigned-byte 16))
1002 LaTeX (array (unsigned-byte 32))
1003 LaTeX (array single-float)
1004 LaTeX (array double-float))
1006 LaTeX (optimize (speed 3) (safety 0))
1007 LaTeX (ext:optimize-interface (safety 3)))
1008 LaTeX ;; with-array-data will get us to the actual data. However, because
1009 LaTeX ;; the array could have been displaced, we need to know where the
1010 LaTeX ;; data starts.
1011 LaTeX (lisp::with-array-data ((data array)
1014 LaTeX (declare (ignore end))
1015 LaTeX ;; DATA is a specialized simple-array. Memory is laid out like this:
1017 LaTeX ;; byte offset Value
1018 LaTeX ;; 0 type code (should be 70 for double-float vector)
1019 LaTeX ;; 4 4 * number of elements in vector
1020 LaTeX ;; 8 1st element of vector
1023 LaTeX (let ((addr (+ 8 (logandc1 7 (kernel:get-lisp-obj-address data))))
1024 LaTeX (type-size (let ((type (array-element-type data)))
1025 LaTeX (cond ((or (equal type '(signed-byte 8))
1026 LaTeX (equal type '(unsigned-byte 8)))
1028 LaTeX ((or (equal type '(signed-byte 16))
1029 LaTeX (equal type '(unsigned-byte 16)))
1031 LaTeX ((or (equal type '(signed-byte 32))
1032 LaTeX (equal type '(unsigned-byte 32)))
1034 LaTeX ((equal type 'single-float)
1036 LaTeX ((equal type 'double-float)
1039 LaTeX (error "Unknown specialized array element type"))))))
1040 LaTeX (declare (type (unsigned-byte 32) addr)
1041 LaTeX (optimize (speed 3) (safety 0) (ext:inhibit-warnings 3)))
1042 LaTeX (system:int-sap (the (unsigned-byte 32)
1043 LaTeX (+ addr (* type-size start)))))))
1046 LaTeX Assume we have the C function below that we wish to use:
1047 LaTeX \begin{example}
1048 LaTeX double dotprod(double* x, double* y, int n)
1051 LaTeX double sum = 0;
1053 LaTeX for (k = 0; k < n; ++k) \{
1054 LaTeX sum += x[k] * y[k];
1058 LaTeX The following example generates two large arrays in Lisp, and calls the C
1059 LaTeX function to do the desired computation. This would not have been
1060 LaTeX possible using \code{malloc} or \code{make-alien} since we need about
1061 LaTeX 16 MB of memory to hold the two arrays.
1062 LaTeX \begin{example}
1063 LaTeX (define-alien-routine "dotprod" double
1064 LaTeX (x (* double-float) :in)
1065 LaTeX (y (* double-float) :in)
1068 LaTeX (let ((x (make-array 1000000 :element-type 'double-float))
1069 LaTeX (y (make-array 1000000 :element-type 'double-float)))
1070 LaTeX ;; Initialize X and Y somehow
1071 LaTeX (let ((x-addr (system:int-sap (array-data-address x)))
1072 LaTeX (y-addr (system:int-sap (array-data-address y))))
1073 LaTeX (dotprod x-addr y-addr 1000000)))
1075 LaTeX In this example, it may be useful to wrap the inner \code{let}
1076 LaTeX expression in an \code{unwind-protect} that first turns off garbage
1077 LaTeX collection and then turns garbage collection on afterwards. This will
1078 LaTeX prevent garbage collection from moving \code{x} and \code{y} after we
1079 LaTeX have obtained the (now erroneous) addresses but before the call to
1080 LaTeX \code{dotprod} is made.
1088 <sect1><title>Step-By-Step Example of the Foreign Function Interface</>
1091 This section presents a complete example of an interface to a somewhat
1092 complicated C function.
1096 Suppose you have the following C function which you want to be able to
1097 call from Lisp in the file <filename>test.c</>
1105 struct c_struct *c_function (i, s, r, a)
1112 struct c_struct *r2;
1114 printf("i = %d\n", i);
1115 printf("s = %s\n", s);
1116 printf("r->x = %d\n", r->x);
1117 printf("r->s = %s\n", r->s);
1118 for (j = 0; j < 10; j++) printf("a[%d] = %d.\n", j, a[j]);
1119 r2 = (struct c_struct *) malloc (sizeof(struct c_struct));
1121 r2->s = "a C string";
1127 It is possible to call this C function from Lisp using the file
1128 <filename>test.lisp</> containing
1130 (cl:defpackage "TEST-C-CALL" (:use "CL" "SB-ALIEN" "SB-C-CALL"))
1131 (cl:in-package "TEST-C-CALL")
1133 ;;; Define the record C-STRUCT in Lisp.
1134 (define-alien-type nil
1139 ;;; Define the Lisp function interface to the C routine. It returns a
1140 ;;; pointer to a record of type C-STRUCT. It accepts four parameters:
1141 ;;; I, an int; S, a pointer to a string; R, a pointer to a C-STRUCT
1142 ;;; record; and A, a pointer to the array of 10 ints.
1144 ;;; The INLINE declaration eliminates some efficiency notes about heap
1145 ;;; allocation of alien values.
1146 (declaim (inline c-function))
1147 (define-alien-routine c-function
1148 (* (struct c-struct))
1151 (r (* (struct c-struct)))
1154 ;;; a function which sets up the parameters to the C function and
1155 ;;; actually calls it
1157 (with-alien ((ar (array int 10))
1158 (c-struct (struct c-struct)))
1159 (dotimes (i 10) ; Fill array.
1160 (setf (deref ar i) i))
1161 (setf (slot c-struct 'x) 20)
1162 (setf (slot c-struct 's) "a Lisp string")
1164 (with-alien ((res (* (struct c-struct))
1165 (c-function 5 "another Lisp string" (addr c-struct) ar)))
1166 (format t "~&back from C function~%")
1167 (multiple-value-prog1
1168 (values (slot res 'x)
1171 ;; Deallocate result. (after we are done referring to it:
1172 ;; "Pillage, *then* burn.")
1173 (free-alien res)))))</programlisting>
1177 To execute the above example, it is necessary to compile the C routine,
1179 <userinput>cc -c test.c</>
1180 (In order to enable incremental loading with some linkers, you may need
1182 <userinput>cc -G 0 -c test.c</>)
1186 Once the C code has been compiled, you can start up Lisp and load it in:
1188 Lisp should start up with its normal prompt.</para>
1192 compile the Lisp file. (This step can be done separately. You don't
1193 have to recompile every time.)
1194 <userinput>(compile-file "test.lisp")</>
1198 Within Lisp, load the foreign object file to define the necessary
1200 <userinput>(load-foreign "test.o")</>.
1201 This must be done before loading any code that refers
1206 Now you can load the compiled Lisp ("fasl") file into Lisp:
1207 <userinput>(load "test.fasl")</>
1208 And once the Lisp file is loaded, you can call the
1209 Lisp routine that sets up the parameters and calls the C
1211 <userinput>(test-c-call::call-cfun)</>
1215 The C routine should print the following information to standard output:
1216 <!-- FIXME: What should be here is a verbatim environment for computer
1217 output, but since I don't know one in DocBook, I made do with
1218 PROGRAMLISTING for now... -->
1219 <programlisting>i = 5
1220 s = another Lisp string
1222 r->s = a Lisp string
1232 a[9] = 9.</programlisting>
1233 After return from the C function,
1234 the Lisp wrapper function should print the following output:
1235 <programlisting>back from C function</programlisting>
1236 And upon return from the Lisp wrapper function,
1237 before the next prompt is printed, the
1238 Lisp read-eval-print loop should print the following return values:
1239 <!-- FIXME: As above, it's not a program listing, but computer output... -->