1 @node The Foreign Function Interface, Function Index, Beyond The ANSI Standard, Top
2 @comment node-name, next, previous, up
3 @chapter The Foreign Function Interface
5 This chapter describes SBCL's interface to C programs and
6 libraries (and, since C interfaces are a sort of @emph{ingua
7 franca} of the Unix world, to other programs and libraries in
11 Note: In the modern Lisp world, the usual term for this functionality
12 is Foreign Function Interface, or @acronym{FFI}, where despite the
13 mention of ``function'' in this term, @acronym{FFI} also
14 refers to direct manipulation of C data structures as well as
15 functions. The traditional CMUCL terminology is Alien Interface, and
16 while that older terminology is no longer used much in the system
17 documentation, it still reflected in names in the implementation,
18 notably in the name of the @code{SB-ALIEN} package.
22 * Introduction to the Foreign Function Interface::
24 * Operations On Foreign Values::
26 * Foreign Data Structure Examples::
27 * Loading Unix Object Files::
28 * Foreign Function Calls::
29 * Step-By-Step Example of the Foreign Function Interface::
32 @node Introduction to the Foreign Function Interface, Foreign Types, The Foreign Function Interface, The Foreign Function Interface
33 @comment node-name, next, previous, up
34 @section Introduction to the Foreign Function Interface
35 @c AKA "Introduction to Aliens" in the CMU CL manual
37 Because of Lisp's emphasis on dynamic memory allocation and garbage
38 collection, Lisp implementations use non-C-like memory representations
39 for objects. This representation mismatch creates friction when a Lisp
40 program must share objects with programs which expect C data. There
41 are three common approaches to establishing communication:
45 The burden can be placed on the foreign program (and programmer) by
46 requiring the knowledge and use of the representations used internally
47 by the Lisp implementation. This can require a considerable amount of
48 ``glue'' code on the C side, and that code tends to be sensitively
49 dependent on the internal implementation details of the Lisp system.
52 The Lisp system can automatically convert objects back and forth
53 between the Lisp and foreign representations. This is convenient, but
54 translation becomes prohibitively slow when large or complex data
55 structures must be shared. This approach is supported by the SBCL
56 @acronym{FFI}, and used automatically by the when passing integers and
60 The Lisp program can directly manipulate foreign objects through the
61 use of extensions to the Lisp language.
65 SBCL, like CMUCL before it, relies primarily on the automatic
66 conversion and direct manipulation approaches. The @code{SB-ALIEN}
67 package provices a facility wherein foreign values of simple scalar
68 types are automatically converted and complex types are directly
69 manipulated in their foreign representation. Additionally the
70 lower-level System Area Pointers (or @acronym{SAP}s) can be used where
71 necessary to provide untyped access to foreign memory.
73 Any foreign objects that can't automatically be converted into Lisp
74 values are represented by objects of type @code{alien-value}. Since
75 Lisp is a dynamically typed language, even foreign objects must have a
76 run-time type; this type information is provided by encapsulating the
77 raw pointer to the foreign data within an @code{alien-value} object.
79 The type language and operations on foreign types are
80 intentionally similar to those of the C language.
82 @node Foreign Types, Operations On Foreign Values, Introduction to the Foreign Function Interface, The Foreign Function Interface
83 @comment node-name, next, previous, up
84 @section Foreign Types
85 @c AKA "Alien Types" in the CMU CL manual
87 Alien types have a description language based on nested list
88 structure. For example the C type
97 has the corresponding SBCL @acronym{FFI} type
102 (b (array (* (struct foo)) 100)))
107 * Defining Foreign Types::
108 * Foreign Types and Lisp Types::
109 * Foreign Type Specifiers::
112 @node Defining Foreign Types, Foreign Types and Lisp Types, Foreign Types, Foreign Types
113 @comment node-name, next, previous, up
114 @subsection Defining Foreign Types
116 Types may be either named or anonymous. With structure and union
117 types, the name is part of the type specifier, allowing recursively
118 defined types such as:
121 (struct foo (a (* (struct foo))))
124 An anonymous structure or union type is specified by using the name
125 @code{nil}. The @code{with-alien} macro defines a local scope which
126 ``captures'' any named type definitions. Other types are not
127 inherently named, but can be given named abbreviations using the
128 @code{define-alien-type} macro.
130 @node Foreign Types and Lisp Types, Foreign Type Specifiers, Defining Foreign Types, Foreign Types
131 @comment node-name, next, previous, up
132 @subsection Foreign Types and Lisp Types
134 The foreign types form a subsystem of the SBCL type system. An
135 @code{alien} type specifier provides a way to use any foreign type as a
136 Lisp type specifier. For example,
139 (typep @var{foo} '(alien (* int)))
142 can be used to determine whether @var{foo} is a pointer to a foreign
143 @code{int}. @code{alien} type specifiers can be used in the same ways
144 as ordinary Lisp type specifiers (like @code{string}.) Alien type
145 declarations are subject to the same precise type checking as any
146 other declaration. @xref{Precise Type Checking}.
148 Note that the type identifiers used in the foreign type system overlap
149 with native Lisp type specifiers in some cases. For example, the type
150 specifier @code{(alien single-float)} is identical to
151 @code{single-float}, since foreign floats are automatically converted
152 to Lisp floats. When @code{type-of} is called on an alien value that
153 is not automatically converted to a Lisp value, then it will return an
154 @code{alien} type specifier.
156 @node Foreign Type Specifiers, , Foreign Types and Lisp Types, Foreign Types
157 @comment node-name, next, previous, up
158 @subsection Foreign Type Specifiers
160 Note: All foreign type names are exported from the @code{sb-alien}
161 package. Some foreign type names are also symbols in
162 the @code{common-lisp} package, in which case they are
163 reexported from the @code{sb-alien} package, so that
164 e.g. it is legal to refer to @code{sb-alien:single-float}.
166 These are the basic foreign type specifiers:
170 The foreign type specifier @code{(* @var{foo})} describes a pointer to
171 an object of type @var{foo}. A pointed-to type @var{foo} of @code{t}
172 indicates a pointer to anything, similar to @code{void *} in
173 ANSI C. A null alien pointer can be detected with the
174 @code{sb-alien:null-alien} function.
177 The foreign type specifier @code{(array @var{foo} &optional
178 dimensions)} describes array of the specified @code{dimensions},
179 holding elements of type @var{foo}. Note that (unlike in C) @code{(*
180 @var{foo})} and @code{(array @var{foo})} are considered to be
181 different types when type checking is done. If equivalence of pointer
182 and array types is desired, it may be explicitly coerced using
183 @code{sb-alien:cast}.
185 Arrays are accessed using @code{sb-alien:deref}, passing the indices
186 as additional arguments. Elements are stored in column-major order
187 (as in C), so the first dimension determines only the size of the
188 memory block, and not the layout of the higher dimensions. An array
189 whose first dimension is variable may be specified by using @code{nil}
190 as the first dimension. Fixed-size arrays can be allocated as array
191 elements, structure slots or @code{sb-alien:with-alien}
192 variables. Dynamic arrays can only be allocated using
193 @code{sb-alien:make-alien}.
196 The foreign type specifier @code{(sb-alien:struct @var{name} &rest
197 @var{fields})} describes a structure type with the specified
198 @var{name} and @var{fields}. Fields are allocated at the same offsets
199 used by the implementation's C compiler. If @var{name} is @code{nil}
200 then the structure is anonymous.
202 If a named foreign @code{struct} specifier is passed to
203 @code{define-alien-type} or @code{with-alien}, then this defines,
204 respectively, a new global or local foreign structure type. If no
205 @var{fields} are specified, then the fields are taken
206 from the current (local or global) alien structure type definition of
210 The foreign type specifier @code{(sb-alien:union @var{name} &rest
211 @var{fields})} is similar to @code{sb-alien:struct}, but describes a
212 union type. All fields are allocated at the same offset, and the size
213 of the union is the size of the largest field. The programmer must
214 determine which field is active from context.
217 The foreign type specifier @code{(sb-alien:enum @var{name} &rest
218 @var{specs})} describes an enumeration type that maps between integer
219 values and keywords. If @var{name} is @code{nil}, then the type is
220 anonymous. Each element of the @var{specs} list is either a Lisp
221 keyword, or a list @code{(@var{keyword} @var{value})}. @var{value} is
222 an integer. If @var{value} is not supplied, then it defaults to one
223 greater than the value for the preceding spec (or to zero if it is the
227 The foreign type specifier @code{(sb-alien:signed &optional
228 @var{bits})} specifies a signed integer with the specified number of
229 @var{bits} precision. The upper limit on integer
230 precision is determined by the machine's word size. If
231 @var{bits} is not specified, the maximum size will be
235 The foreign type specifier @code{(integer &optional @var{bits})}
236 is equivalent to the corresponding type specifier using
237 @code{sb-alien:signed} instead of @code{integer}.
240 The foreign type specifier @code{(sb-alien:unsigned &optional
241 @var{bits})} is like corresponding type specifier using
242 @code{sb-alien:signed} except that the variable is treated as an
246 The foreign type specifier @code{(boolean &optional @var{bits})} is
247 similar to an enumeration type, but maps from Lisp @code{nil} and
248 @code{t} to C @code{0} and @code{1} respectively. @var{bits}
249 determines the amount of storage allocated to hold the truth value.
252 The foreign type specifier @code{single-float} describes a
253 floating-point number in IEEE single-precision format.
256 The foreign type specifier @code{double-float} describes a
257 floating-point number in IEEE double-precision format.
260 The foreign type specifier @code{(function @var{result-type} &rest
261 @var{arg-types})} describes a foreign function that takes arguments of
262 the specified @var{arg-types} and returns a result of type
263 @var{result-type}. Note that the only context where a foreign
264 @code{function} type is directly specified is in the argument to
265 @code{sb-alien:alien-funcall}. In all other contexts, foreign
266 functions are represented by foreign function pointer types: @code{(*
267 (function @dots{}))}.
270 The foreign type specifier @code{sb-alien:system-area-pointer}
271 describes a pointer which is represented in Lisp as a
272 @code{system-area-pointer} object. SBCL exports this type from
273 @code{sb-alien} because CMUCL did, but tentatively (as of the first
274 draft of this section of the manual, SBCL 0.7.6) it is deprecated,
275 since it doesn't seem to be required by user code.
278 The foreign type specifier @code{sb-alien:void} is used in function
279 types to declare that no useful value is returned. Using
280 @code{alien-funcall} to call a @code{void} foreign function will
284 The foreign type specifier @code{sb-alien:c-string} is similar to
285 @code{(* char)}, but is interpreted as a null-terminated string, and
286 is automatically converted into a Lisp string when accessed; or if the
287 pointer is C @code{NULL} or @code{0}, then accessing it gives Lisp
288 @code{nil}. Lisp strings are stored with a trailing NUL
289 termination, so no copying (either by the user or the implementation)
290 is necessary when passing them to foreign code.
292 Assigning a Lisp string to a @code{c-string} structure field or
293 variable stores the contents of the string to the memory already
294 pointed to by that variable. When a foreign object of type @code{(*
295 char)} is assigned to a @code{c-string}, then the
296 @code{c-string} pointer is assigned to. This allows
297 @code{c-string} pointers to be initialized. For example:
300 (cl:in-package "CL-USER") ; which USEs package "SB-ALIEN"
302 (define-alien-type nil (struct foo (str c-string)))
304 (defun make-foo (str)
305 (let ((my-foo (make-alien (struct foo))))
306 (setf (slot my-foo 'str) (make-alien char (length str))
307 (slot my-foo 'str) str)
311 Storing Lisp @code{NIL} in a @code{c-string} writes C @code{NULL} to
315 @code{sb-alien} also exports translations of these C type
316 specifiers as foreign type specifiers: @code{sb-alien:char},
317 @code{sb-alien:short}, @code{sb-alien:int},
318 @code{sb-alien:long}, @code{sb-alien:unsigned-char},
319 @code{sb-alien:unsigned-short},
320 @code{sb-alien:unsigned-int},
321 @code{sb-alien:unsigned-long}, @code{sb-alien:float}, and
322 @code{sb-alien:double}.
326 @node Operations On Foreign Values, Foreign Variables, Foreign Types, The Foreign Function Interface
327 @comment node-name, next, previous, up
328 @section Operations On Foreign Values
329 @c AKA "Alien Operations" in the CMU CL manual
331 This section describes how to read foreign values as Lisp values, how
332 to coerce foreign values to different kinds of foreign values, and how
333 to dynamically allocate and free foreign variables.
336 * Accessing Foreign Values::
337 * Coercing Foreign Values::
338 * Foreign Dynamic Allocation::
341 @node Accessing Foreign Values, Coercing Foreign Values, Operations On Foreign Values, Operations On Foreign Values
342 @comment node-name, next, previous, up
343 @subsection Accessing Foreign Values
345 @defun sb-alien:deref @var{pointer-or-array} &rest @var{indices}
347 The @code{sb-alien:deref} function returns the value pointed to by a
348 foreign pointer, or the value of a foreign array element. When
349 dereferencing a pointer, an optional single index can be specified to
350 give the equivalent of C pointer arithmetic; this index is scaled by
351 the size of the type pointed to. When dereferencing an array, the
352 number of indices must be the same as the number of dimensions in the
353 array type. @code{deref} can be set with @code{setf} to assign a new
358 @defun sb-alien:slot @var{struct-or-union} &rest @var{slot-names}
360 The @code{sb-alien:slot} function extracts the value of the slot named
361 @var{slot-name} from a foreign @code{struct} or @code{union}. If
362 @var{struct-or-union} is a pointer to a structure or union, then it is
363 automatically dereferenced. @code{sb-alien:slot} can be set with
364 @code{setf} to assign a new value. Note that @var{slot-name} is
365 evaluated, and need not be a compile-time constant (but only constant
366 slot accesses are efficiently compiled).
370 @subsubsection Untyped memory
372 As noted at the beginning of the chapter, the System Area Pointer
373 facilities allow untyped access to foreign memory. @acronym{SAP}s can
374 be converted to and from the usual typed foreign values using
375 @code{sap-alien} and @code{alien-sap} (described elsewhere), and also
376 to and from integers - raw machine addresses. They should thus be
377 used with caution; corrupting the Lisp heap or other memory with
378 @acronym{SAP}s is trivial.
380 @defun sb-sys:int-sap @var{machine-address}
382 Creates a @acronym{SAP} pointing at the virtual address
383 @var{machine-address}.
386 @defun sb-sys:sap-ref-32 @var{sap} @var{offset}
388 Access the value of the memory location at @var{offset} bytes from
389 @var{sap}. This form may also be used with @code{setf} to alter the
390 memory at that location.
393 @defun sb-sys:sap= @var{sap1} @var{sap2}
395 Compare @var{sap1} and @var{sap2} for equality.
398 Similarly named functions exist for accessing other sizes of word,
399 other comparisons, and other conversions. The reader is invited to
400 use @code{apropos} and @code{describe} for more details
403 (apropos "sap" :sb-sys)
407 @node Coercing Foreign Values, Foreign Dynamic Allocation, Accessing Foreign Values, Operations On Foreign Values
408 @comment node-name, next, previous, up
409 @subsection Coercing Foreign Values
411 @defun sb-alien:addr @var{alien-expr}
413 The @code{sb-alien:addr} macro returns a pointer to the location
414 specified by @var{alien-expr}, which must be either a foreign
415 variable, a use of @code{sb-alien:deref}, a use of
416 @code{sb-alien:slot}, or a use of @code{sb-alien:extern-alien}.
419 @defun sb-alien:cast @var{foreign-value} @var{new-type}
421 The @code{sb-alien:cast} macro converts @var{foreign-value} to a new
422 foreign value with the specified @var{new-type}. Both types, old and
423 new, must be foreign pointer, array or function types. Note that the
424 resulting Lisp foreign variable object is not @code{eq} to the
425 argument, but it does refer to the same foreign data bits.
428 @defun sb-alien:sap-alien @var{sap} @var{type}
430 The @code{sb-alien:sap-alien} function converts @var{sap} (a system
431 area pointer) to a foreign value with the specified
432 @var{type}. @var{type} is not evaluated. </para>
434 The @var{type} must be some foreign pointer, array, or record type.
437 @defun sb-alien:alien-sap @var{foreign-value} @var{type}
439 The @code{sb-alien:alien-sap} function returns the @acronym{SAP} which
440 points to @var{alien-value}'s data.
442 The @var{foreign-value} must be of some foreign pointer, array, or
447 @node Foreign Dynamic Allocation, , Coercing Foreign Values, Operations On Foreign Values
448 @comment node-name, next, previous, up
449 @subsection Foreign Dynamic Allocation
451 Lisp code can call the C standard library functions @code{malloc} and
452 @code{free} to dynamically allocate and deallocate foreign
453 variables. The Lisp code shares the same allocator with foreign C
454 code, so it's OK for foreign code to call @code{free} on the result of
455 Lisp @code{sb-alien:make-alien}, or for Lisp code to call
456 @code{sb-alien:free-alien} on foreign objects allocated by C
459 @defmac sb-alien:make-alien @var{type} @var{size}
461 The @code{sb-alien:make-alien} macro
462 returns a dynamically allocated foreign value of the specified
463 @var{type} (which is not evaluated.) The allocated memory is not
464 initialized, and may contain arbitrary junk. If supplied,
465 @var{size} is an expression to evaluate to compute the size of the
466 allocated object. There are two major cases:
470 When @var{type} is a foreign array type, an array of that type is
471 allocated and a pointer to it is returned. Note that you must use
472 @code{deref} to change the result to an array before you can use
473 @code{deref} to read or write elements:
476 (cl:in-package "CL-USER") ; which USEs package "SB-ALIEN"
477 (defvar *foo* (make-alien (array char 10)))
478 (type-of *foo*) @result{} (alien (* (array (signed 8) 10)))
479 (setf (deref (deref foo) 0) 10) @result{} 10
482 If supplied, @var{size} is used as the first dimension for the
486 When @var{type} is any other foreign type, then an object for that
487 type is allocated, and a pointer to it is returned. So
488 @code{(make-alien int)} returns a @code{(* int)}. If @var{size} is
489 specified, then a block of that many objects is allocated, with the
490 result pointing to the first one.
496 @defun sb-alien:free-alien @var{foreign-value}
498 The @code{sb-alien:free-alien} function
499 frees the storage for @var{foreign-value},
500 which must have been allocated with Lisp @code{make-alien}
503 See also the @code{sb-alien:with-alien} macro, which allocates foreign
507 @node Foreign Variables, Foreign Data Structure Examples, Operations On Foreign Values, The Foreign Function Interface
508 @comment node-name, next, previous, up
509 @section Foreign Variables
510 @c AKA "Alien Variables" in the CMU CL manual
512 Both local (stack allocated) and external (C global) foreign variables
516 * Local Foreign Variables::
517 * External Foreign Variables::
520 @node Local Foreign Variables, External Foreign Variables, Foreign Variables, Foreign Variables
521 @comment node-name, next, previous, up
522 @subsection Local Foreign Variables
524 @defmac sb-alien:with-alien @var{var-definitions} &body @var{body}
526 The @code{with-alien} macro establishes local foreign variables with
527 the specified alien types and names. This form is analogous to
528 defining a local variable in C: additional storage is allocated, and
529 the initial value is copied. This form is less analogous to
530 @code{LET}-allocated Lisp variables, since the variables can't be
531 captured in closures: they live only for the dynamic extent of the
532 body, and referring to them outside is a gruesome error.
534 The @var{var-definitions} argument is a list of
535 variable definitions, each of the form
537 (@var{name} @var{type} &optional @var{initial-value})
540 The names of the variables are established as symbol-macros; the
541 bindings have lexical scope, and may be assigned with @code{setq} or
544 The @code{with-alien} macro also establishes a new scope for named
545 structures and unions. Any @var{type} specified for a variable may
546 contain named structure or union types with the slots specified.
547 Within the lexical scope of the binding specifiers and body, a locally
548 defined foreign structure type @var{foo} can be referenced by its name
549 using @code{(struct @var{foo})}.
552 @node External Foreign Variables, , Local Foreign Variables, Foreign Variables
553 @comment node-name, next, previous, up
554 @subsection External Foreign Variables
556 External foreign names are strings, and Lisp names are symbols. When
557 an external foreign value is represented using a Lisp variable, there
558 must be a way to convert from one name syntax into the other. The
559 macros @code{extern-alien}, @code{define-alien-variable} and
560 @code{define-alien-routine} use this conversion heuristic:
565 Alien names are converted to Lisp names by uppercasing and replacing
566 underscores with hyphens.
569 Conversely, Lisp names are converted to alien names by lowercasing and
570 replacing hyphens with underscores.
573 Both the Lisp symbol and alien string names may be separately
574 specified by using a list of the form
577 (alien-string lisp-symbol)
582 @defmac sb-alien:define-alien-variable @var{name} @var{type}
584 The @code{define-alien-variable} macro defines @var{name} as an
585 external foreign variable of the specified foreign @code{type}.
586 @var{name} and @code{type} are not evaluated. The Lisp name of the
587 variable (see above) becomes a global alien variable. Global alien
588 variables are effectively ``global symbol macros''; a reference to the
589 variable fetches the contents of the external variable. Similarly,
590 setting the variable stores new contents -- the new contents must be
591 of the declared @code{type}. Someday, they may well be implemented
592 using the @acronym{ANSI} @code{define-symbol-macro} mechanism, but as
593 of SBCL 0.7.5, they are still implemented using an older more-or-less
594 parallel mechanism inherited from CMUCL.
596 For example, to access a C-level counter @var{foo}, one could write
599 (define-alien-variable "foo" int)
600 ;; Now it is possible to get the value of the C variable foo simply by
601 ;; referencing that Lisp variable:
608 @defun sb-alien:get-errno
610 Since in modern C libraries, the @code{errno} ``variable'' is typically
611 no longer a variable, but some bizarre artificial construct
612 which behaves superficially like a variable within a given thread,
613 it can no longer reliably be accessed through the ordinary
614 @code{define-alien-variable} mechanism. Instead, SBCL provides
615 the operator @code{sb-alien:get-errno} to allow Lisp code to read it.
618 @defmac sb-alien:extern-alien @var{name} @var{type}
620 The @code{extern-alien} macro returns an alien with the specified
621 @var{type} which points to an externally defined value. @var{name} is
622 not evaluated, and may be either a string or a symbol. @var{type} is
623 an unevaluated alien type specifier.
626 @node Foreign Data Structure Examples, Loading Unix Object Files, Foreign Variables, The Foreign Function Interface
627 @comment node-name, next, previous, up
628 @section Foreign Data Structure Examples
629 @c AKA "Alien Data Structure Example" in the CMU CL manual
631 Now that we have alien types, operations and variables, we can
632 manipulate foreign data structures. This C declaration
641 can be translated into the following alien type:
644 (define-alien-type nil
647 (b (array (* (struct foo)) 100))))
650 Once the @code{foo} alien type has been defined as above, the C
658 can be translated in this way:
661 (with-alien ((f (struct foo)))
662 (slot (deref (slot f 'b) 7) 'a)
664 ;; Do something with f...
668 Or consider this example of an external C variable and some accesses:
677 extern struct c_struct *my_struct;
680 my_struct = my_struct->n;
683 which can be manipulated in Lisp like this:
686 (define-alien-type nil
694 (define-alien-variable "my_struct" (* c-struct))
695 (incf (slot my-struct 'x))
696 (setf (slot my-struct 'a) 5)
697 (setq my-struct (slot my-struct 'n))
700 @node Loading Unix Object Files, Foreign Function Calls, Foreign Data Structure Examples, The Foreign Function Interface
701 @comment node-name, next, previous, up
702 @section Loading Unix Object Files
704 Foreign object files can be loaded into the running Lisp process by
705 calling the functions @code{load-foreign} or @code{load-1-foreign}.
707 The @code{sb-alien:load-1-foreign} function is the more primitive of
708 the two operations. It loads a single object file into the currently
709 running Lisp. The external symbols defining routines and variables are
710 made available for future external references (e.g. by
711 @code{extern-alien}). Forward references to foreign symbols aren't
712 supported: @code{load-1-foreign} must be run before any of the defined
713 symbols are referenced.
715 @code{sb-alien:load-foreign} is built in terms of
716 @code{load-1-foreign} and some other machinery like
717 @code{sb-ext:run-program}. It accepts a list of files and libraries,
718 and runs the linker on the files and libraries, creating an absolute
719 Unix object file which is then processed by @code{load-1-foreign}.
722 Note: As of SBCL 0.7.5, all foreign code (code loaded with
723 @code{load-1-function} or @code{load-function}) is lost when a Lisp
724 core is saved with @code{sb-ext:save-lisp-and-die}, and no attempt is
725 made to restore it when the core is loaded. Historically this has been
726 an annoyance both for SBCL users and for CMUCL users. It's hard to
727 solve this problem completely cleanly, but some generally-reliable
728 partial solution might be useful. Once someone in either camp gets
729 sufficiently annoyed to create it, SBCL is likely to adopt some
730 mechanism for automatically restoring foreign code when a saved core
735 @node Foreign Function Calls, Step-By-Step Example of the Foreign Function Interface, Loading Unix Object Files, The Foreign Function Interface
736 @comment node-name, next, previous, up
737 @section Foreign Function Calls
739 The foreign function call interface allows a Lisp program to call
740 many functions written in languages that use the C calling convention.
742 Lisp sets up various signal handling routines and other environment
743 information when it first starts up, and expects these to be in place
744 at all times. The C functions called by Lisp should not change the
745 environment, especially the signal handlers: the signal handlers
746 installed by Lisp typically have interesting flags set (e.g to request
747 machine context information, or for signal delivery on an alternate
748 stack) which the Lisp runtime relies on for correct operation.
749 Precise details of how this works may change without notice between
750 versions; the source, or the brain of a friendly SBCL developer, is
751 the only documentation. Users of a Lisp built with the
752 @code{:sb-thread} feature should also read the section about threads,
756 * The alien-funcall Primitive::
757 * The define-alien-routine Macro::
758 * define-alien-routine Example::
759 * Calling Lisp From C::
762 @node The alien-funcall Primitive, The define-alien-routine Macro, Foreign Function Calls, Foreign Function Calls
763 @comment node-name, next, previous, up
764 @subsection The @code{alien-funcall} Primitive
766 @defun sb-alien:alien-funcall @var{alien-function} &rest @var{arguments}
768 The @code{alien-funcall} function is the foreign function call
769 primitive: @var{alien-function} is called with the supplied
770 @var{arguments} and its C return value is returned as a Lisp value.
771 The @var{alien-function} is an arbitrary run-time expression; to refer
772 to a constant function, use @code{extern-alien} or a value defined by
773 @code{define-alien-routine}.
775 The type of @code{alien-function} must be @code{(alien (function
776 ...))} or @code{(alien (* (function ...)))}. The function type is
777 used to determine how to call the function (as though it was declared
778 with a prototype.) The type need not be known at compile time, but
779 only known-type calls are efficiently compiled. Limitations:
784 Structure type return values are not implemented.
787 Passing of structures by value is not implemented.
793 Here is an example which allocates a @code{(struct foo)}, calls a
794 foreign function to initialize it, then returns a Lisp vector of all
795 the @code{(* (struct foo))} objects filled in by the foreign call:
798 ;; Allocate a foo on the stack.
799 (with-alien ((f (struct foo)))
800 ;; Call some C function to fill in foo fields.
801 (alien-funcall (extern-alien "mangle_foo" (function void (* foo)))
803 ;; Find how many foos to use by getting the A field.
804 (let* ((num (slot f 'a))
805 (result (make-array num)))
806 ;; Get a pointer to the array so that we don't have to keep extracting it:
807 (with-alien ((a (* (array (* (struct foo)) 100)) (addr (slot f 'b))))
808 ;; Loop over the first N elements and stash them in the result vector.
810 (setf (svref result i) (deref (deref a) i)))
815 @node The define-alien-routine Macro, define-alien-routine Example, The alien-funcall Primitive, Foreign Function Calls
816 @comment node-name, next, previous, up
817 @subsection The @code{define-alien-routine} Macro
819 @defmac sb-alien:define-alien-routine @var{name} @var{result-type} &rest @var{arg-specifiers}
821 The @code{define-alien-routine} macro is a convenience for
822 automatically generating Lisp interfaces to simple foreign functions.
823 The primary feature is the parameter style specification, which
824 translates the C pass-by-reference idiom into additional return
827 @var{name} is usually a string external symbol, but may also be a
828 symbol Lisp name or a list of the foreign name and the Lisp name. If
829 only one name is specified, the other is automatically derived as for
830 @code{extern-alien}. @var{result-type} is the alien type of the
833 Each element of the @var{arg-specifiers} list
834 specifies an argument to the foreign function, and is
837 (aname atype &optional style)
840 @var{aname} is the symbol name of the argument to the constructed
841 function (for documentation). @var{atype} is the alien type of
842 corresponding foreign argument. The semantics of the actual call are
843 the same as for @code{alien-funcall}. @var{style} specifies how this
844 argument should be handled at call and return time, and should be one
850 @code{:in} specifies that the argument is passed by value. This is the
851 default. @code{:in} arguments have no corresponding return value from
855 @code{:copy} is similar to @code{:in}, but the argument is copied to a
856 pre-allocated object and a pointer to this object is passed to the
860 @code{:out} specifies a pass-by-reference output value. The type of
861 the argument must be a pointer to a fixed-sized object (such as an
862 integer or pointer). @code{:out} and @code{:in-out} style cannot be
863 used with pointers to arrays, records or functions. An object of the
864 correct size is allocated on the stack, and its address is passed to
865 the foreign function. When the function returns, the contents of this
866 location are returned as one of the values of the Lisp function (and
867 the location is automatically deallocated).
870 @code{:in-out} is a combination of @code{:copy} and @code{:out}. The
871 argument is copied to a pre-allocated object and a pointer to this
872 object is passed to the foreign routine. On return, the contents of
873 this location is returned as an additional value.
878 Note: Any efficiency-critical foreign interface function should be inline
879 expanded, which can be done by preceding the
880 @code{define-alien-routine} call with:
883 (declaim (inline lisp-name))
886 In addition to avoiding the Lisp call overhead, this allows
887 pointers, word-integers and floats to be passed using non-descriptor
888 representations, avoiding consing.)
893 @node define-alien-routine Example, Calling Lisp From C, The define-alien-routine Macro, Foreign Function Calls
894 @comment node-name, next, previous, up
895 @subsection @code{define-alien-routine} Example
897 Consider the C function @code{cfoo} with the following calling
904 char *a; /* update */
907 /* body of cfoo(...) */
911 This can be described by the following call to
912 @code{define-alien-routine}:
915 (define-alien-routine "cfoo" void
921 The Lisp function @code{cfoo} will have two arguments (@var{str} and
922 @var{a}) and two return values (@var{a} and @var{i}).
924 @node Calling Lisp From C, , define-alien-routine Example, Foreign Function Calls
925 @comment node-name, next, previous, up
926 @subsection Calling Lisp From C
928 Calling Lisp functions from C is sometimes possible, but is extremely
929 hackish and poorly supported as of SBCL 0.7.5. See @code{funcall0}
930 @dots{} @code{funcall3} in the runtime system. The arguments must be
931 valid SBCL object descriptors (so that e.g. fixnums must be
932 left-shifted by 2.) As of SBCL 0.7.5, the format of object descriptors
933 is documented only by the source code and, in parts, by the old CMUCL
934 @file{INTERNALS} documentation.
936 Note that the garbage collector moves objects, and won't be
937 able to fix up any references in C variables. There are three
938 mechanisms for coping with this:
942 The @code{sb-ext:purify} moves all live Lisp
943 data into static or read-only areas such that it will never be moved
944 (or freed) again in the life of the Lisp session
947 @code{sb-sys:with-pinned-objects} is a macro which arranges for some
948 set of objects to be pinned in memory for the dynamic extent of its
949 body forms. On ports which use the generational garbage collector (as
950 of SBCL 0.8.3, only the x86) this has a page granularity - i.e. the
951 entire 4k page or pages containing the objects will be locked down. On
952 other ports it is implemented by turning off GC for the duration (so
953 could be said to have a whole-world granularity).
956 Disable GC, using the @code{without-gcing} macro or @code{gc-off}
960 @c <!-- FIXME: This is a "changebar" section from the CMU CL manual.
961 @c I (WHN 2002-07-14) am not very familiar with this content, so
962 @c I'm not immediately prepared to try to update it for SBCL, and
963 @c I'm not feeling masochistic enough to work to encourage this
964 @c kind of low-level hack anyway. However, I acknowledge that callbacks
965 @c are sometimes really really necessary, so I include the original
966 @c text in case someone is hard-core enough to benefit from it. If
967 @c anyone brings the information up to date for SBCL, it belong
968 @c either in the main manual or on a CLiki SBCL Internals page.
969 @c LaTeX \subsection{Accessing Lisp Arrays}
971 @c LaTeX Due to the way \cmucl{} manages memory, the amount of memory that can
972 @c LaTeX be dynamically allocated by \code{malloc} or \funref{make-alien} is
973 @c LaTeX limited\footnote{\cmucl{} mmaps a large piece of memory for it's own
974 @c LaTeX use and this memory is typically about 8 MB above the start of the C
975 @c LaTeX heap. Thus, only about 8 MB of memory can be dynamically
976 @c LaTeX allocated.}.
978 @c Empirically determined to be considerably >8Mb on this x86 linux
979 @c machine, but I don't know what the actual values are - dan 2003.09.01
981 @c Note that this technique is used in SB-GROVEL in the SBCL contrib
984 @c LaTeX To overcome this limitation, it is possible to access the content of
985 @c LaTeX Lisp arrays which are limited only by the amount of physical memory
986 @c LaTeX and swap space available. However, this technique is only useful if
987 @c LaTeX the foreign function takes pointers to memory instead of allocating
988 @c LaTeX memory for itself. In latter case, you will have to modify the
989 @c LaTeX foreign functions.
991 @c LaTeX This technique takes advantage of the fact that \cmucl{} has
992 @c LaTeX specialized array types (\pxlref{specialized-array-types}) that match
993 @c LaTeX a typical C array. For example, a \code{(simple-array double-float
994 @c LaTeX (100))} is stored in memory in essentially the same way as the C
995 @c LaTeX array \code{double x[100]} would be. The following function allows us
996 @c LaTeX to get the physical address of such a Lisp array:
997 @c LaTeX \begin{example}
998 @c LaTeX (defun array-data-address (array)
999 @c LaTeX "Return the physical address of where the actual data of an array is
1002 @c LaTeX ARRAY must be a specialized array type in CMU Lisp. This means ARRAY
1003 @c LaTeX must be an array of one of the following types:
1005 @c LaTeX double-float
1006 @c LaTeX single-float
1007 @c LaTeX (unsigned-byte 32)
1008 @c LaTeX (unsigned-byte 16)
1009 @c LaTeX (unsigned-byte 8)
1010 @c LaTeX (signed-byte 32)
1011 @c LaTeX (signed-byte 16)
1012 @c LaTeX (signed-byte 8)
1014 @c LaTeX (declare (type (or #+signed-array (array (signed-byte 8))
1015 @c LaTeX #+signed-array (array (signed-byte 16))
1016 @c LaTeX #+signed-array (array (signed-byte 32))
1017 @c LaTeX (array (unsigned-byte 8))
1018 @c LaTeX (array (unsigned-byte 16))
1019 @c LaTeX (array (unsigned-byte 32))
1020 @c LaTeX (array single-float)
1021 @c LaTeX (array double-float))
1023 @c LaTeX (optimize (speed 3) (safety 0))
1024 @c LaTeX (ext:optimize-interface (safety 3)))
1025 @c LaTeX ;; with-array-data will get us to the actual data. However, because
1026 @c LaTeX ;; the array could have been displaced, we need to know where the
1027 @c LaTeX ;; data starts.
1028 @c LaTeX (lisp::with-array-data ((data array)
1031 @c LaTeX (declare (ignore end))
1032 @c LaTeX ;; DATA is a specialized simple-array. Memory is laid out like this:
1034 @c LaTeX ;; byte offset Value
1035 @c LaTeX ;; 0 type code (should be 70 for double-float vector)
1036 @c LaTeX ;; 4 4 * number of elements in vector
1037 @c LaTeX ;; 8 1st element of vector
1040 @c LaTeX (let ((addr (+ 8 (logandc1 7 (kernel:get-lisp-obj-address data))))
1041 @c LaTeX (type-size (let ((type (array-element-type data)))
1042 @c LaTeX (cond ((or (equal type '(signed-byte 8))
1043 @c LaTeX (equal type '(unsigned-byte 8)))
1045 @c LaTeX ((or (equal type '(signed-byte 16))
1046 @c LaTeX (equal type '(unsigned-byte 16)))
1048 @c LaTeX ((or (equal type '(signed-byte 32))
1049 @c LaTeX (equal type '(unsigned-byte 32)))
1051 @c LaTeX ((equal type 'single-float)
1053 @c LaTeX ((equal type 'double-float)
1056 @c LaTeX (error "Unknown specialized array element type"))))))
1057 @c LaTeX (declare (type (unsigned-byte 32) addr)
1058 @c LaTeX (optimize (speed 3) (safety 0) (ext:inhibit-warnings 3)))
1059 @c LaTeX (system:int-sap (the (unsigned-byte 32)
1060 @c LaTeX (+ addr (* type-size start)))))))
1061 @c LaTeX \end{example}
1063 @c LaTeX Assume we have the C function below that we wish to use:
1064 @c LaTeX \begin{example}
1065 @c LaTeX double dotprod(double* x, double* y, int n)
1068 @c LaTeX double sum = 0;
1070 @c LaTeX for (k = 0; k < n; ++k) \{
1071 @c LaTeX sum += x[k] * y[k];
1074 @c LaTeX \end{example}
1075 @c LaTeX The following example generates two large arrays in Lisp, and calls the C
1076 @c LaTeX function to do the desired computation. This would not have been
1077 @c LaTeX possible using \code{malloc} or \code{make-alien} since we need about
1078 @c LaTeX 16 MB of memory to hold the two arrays.
1079 @c LaTeX \begin{example}
1080 @c LaTeX (define-alien-routine "dotprod" double
1081 @c LaTeX (x (* double-float) :in)
1082 @c LaTeX (y (* double-float) :in)
1083 @c LaTeX (n int :in))
1085 @c LaTeX (let ((x (make-array 1000000 :element-type 'double-float))
1086 @c LaTeX (y (make-array 1000000 :element-type 'double-float)))
1087 @c LaTeX ;; Initialize X and Y somehow
1088 @c LaTeX (let ((x-addr (system:int-sap (array-data-address x)))
1089 @c LaTeX (y-addr (system:int-sap (array-data-address y))))
1090 @c LaTeX (dotprod x-addr y-addr 1000000)))
1091 @c LaTeX \end{example}
1092 @c LaTeX In this example, it may be useful to wrap the inner \code{let}
1093 @c LaTeX expression in an \code{unwind-protect} that first turns off garbage
1094 @c LaTeX collection and then turns garbage collection on afterwards. This will
1095 @c LaTeX prevent garbage collection from moving \code{x} and \code{y} after we
1096 @c LaTeX have obtained the (now erroneous) addresses but before the call to
1097 @c LaTeX \code{dotprod} is made.
1102 @node Step-By-Step Example of the Foreign Function Interface, , Foreign Function Calls, The Foreign Function Interface
1103 @comment node-name, next, previous, up
1104 @section Step-By-Step Example of the Foreign Function Interface
1106 This section presents a complete example of an interface to a somewhat
1107 complicated C function.
1109 Suppose you have the following C function which you want to be able to
1110 call from Lisp in the file @file{test.c}
1119 struct c_struct *c_function (i, s, r, a)
1126 struct c_struct *r2;
1128 printf("i = %d\n", i);
1129 printf("s = %s\n", s);
1130 printf("r->x = %d\n", r->x);
1131 printf("r->s = %s\n", r->s);
1132 for (j = 0; j < 10; j++) printf("a[%d] = %d.\n", j, a[j]);
1133 r2 = (struct c_struct *) malloc (sizeof(struct c_struct));
1135 r2->s = "a C string";
1140 It is possible to call this C function from Lisp using the file
1141 @file{test.lisp} containing
1144 (cl:defpackage "TEST-C-CALL" (:use "CL" "SB-ALIEN" "SB-C-CALL"))
1145 (cl:in-package "TEST-C-CALL")
1147 ;;; Define the record C-STRUCT in Lisp.
1148 (define-alien-type nil
1153 ;;; Define the Lisp function interface to the C routine. It returns a
1154 ;;; pointer to a record of type C-STRUCT. It accepts four parameters:
1155 ;;; I, an int; S, a pointer to a string; R, a pointer to a C-STRUCT
1156 ;;; record; and A, a pointer to the array of 10 ints.
1158 ;;; The INLINE declaration eliminates some efficiency notes about heap
1159 ;;; allocation of alien values.
1160 (declaim (inline c-function))
1161 (define-alien-routine c-function
1162 (* (struct c-struct))
1165 (r (* (struct c-struct)))
1168 ;;; a function which sets up the parameters to the C function and
1169 ;;; actually calls it
1171 (with-alien ((ar (array int 10))
1172 (c-struct (struct c-struct)))
1173 (dotimes (i 10) ; Fill array.
1174 (setf (deref ar i) i))
1175 (setf (slot c-struct 'x) 20)
1176 (setf (slot c-struct 's) "a Lisp string")
1178 (with-alien ((res (* (struct c-struct))
1179 (c-function 5 "another Lisp string" (addr c-struct) ar)))
1180 (format t "~&back from C function~%")
1181 (multiple-value-prog1
1182 (values (slot res 'x)
1185 ;; Deallocate result. (after we are done referring to it:
1186 ;; "Pillage, *then* burn.")
1187 (free-alien res)))))
1190 To execute the above example, it is necessary to compile the C
1191 routine, e.g.: @samp{cc -c test.c} (In order to enable incremental
1192 loading with some linkers, you may need to say @samp{cc -G 0 -c
1195 Once the C code has been compiled, you can start up Lisp and load it
1196 in: @samp{sbcl} Lisp should start up with its normal prompt.
1198 Within Lisp, compile the Lisp file. (This step can be done
1199 separately. You don't have to recompile every time.)
1200 @samp{(compile-file "test.lisp")}
1202 Within Lisp, load the foreign object file to define the necessary
1203 symbols: @samp{(load-foreign "test.o")}. This must be done before
1204 loading any code that refers to these symbols.
1206 Now you can load the compiled Lisp (``fasl'') file into Lisp:
1207 @samp{(load "test.fasl")}
1208 And once the Lisp file is loaded, you can call the
1209 Lisp routine that sets up the parameters and calls the C
1211 @samp{(test-c-call::call-cfun)}
1213 The C routine should print the following information to standard output:
1217 s = another Lisp string
1219 r->s = a Lisp string
1232 After return from the C function,
1233 the Lisp wrapper function should print the following output:
1236 back from C function
1239 And upon return from the Lisp wrapper function,
1240 before the next prompt is printed, the
1241 Lisp read-eval-print loop should print the following return values: