X-Git-Url: http://repo.macrolet.net/gitweb/?a=blobdiff_plain;f=doc%2Fffi.sgml;h=e48d16cf77df9b57a23a1a90292a7af2d87feb41;hb=4cf9c8955fc99aa5718eb4b265360578d0de29e0;hp=cc33fe570ac2abb37fe7eb5061b6335c55ee1458;hpb=a530bbe337109d898d5b4a001fc8f1afa3b5dc39;p=sbcl.git diff --git a/doc/ffi.sgml b/doc/ffi.sgml index cc33fe5..e48d16c 100644 --- a/doc/ffi.sgml +++ b/doc/ffi.sgml @@ -1,32 +1,1173 @@ The Foreign Function Interface</> -<para>FIXME: The material in the &CMUCL; manual about the foreign -function interface should be reviewed, reformatted in DocBook, -lightly edited for &SBCL;, and substituted into this manual. But in -the meantime, the original &CMUCL; manual is still 95+% correct for -the &SBCL; version of the foreign function interface. (The main -difference is that the package names have changed from -<quote><literal>ALIEN</></> and <quote><literal>C-CALL</></> to -<quote><literal>SB-ALIEN</></> and <quote><literal>SB-C-CALL</></>.) - <!-- FIXME: Oh, and I seem to remember that the CMUCL manual - was out of date about how to test for a null pointer, - there's a builtin operator to do it, you don't need to - do the nasty idiom the manual says you need to do. --> - <!-- FIXME: Also, the CMU CL alien documentation claims you - can just do (DEF-ALIEN-VARIABLE "errno" INT), which fails - with modern multithreading hacks. --> - <!-- FIXME: Also, LOAD-FOREIGN isn't implemented as of sbcl-0.6.7, - but LOAD-1-FOREIGN is. --> -See the sections +<para>This chapter describes &SBCL;'s interface to C programs and +libraries (and, since C interfaces are a sort of <foreignphrase>lingua +franca</> of the Unix world, to other programs and libraries in +general.)</para> + +<note><para>In the modern Lisp world, the usual term for this +functionality is Foreign Function Interface, or <acronym>FFI</>, where +despite the mention of <quote>function</> in this term, <acronym>FFI</> also +refers to direct manipulation of C data structures as well as +functions. The traditional &CMUCL; terminology is Alien Interface, and +while that older terminology is no longer used much in the system +documentation, it still reflected in names in the +implementation, notably in the name of the <literal>SB-ALIEN</> +package.</para></note> + +<sect1><title>Introduction to the Foreign Function Interface</> +<!-- AKA "Introduction to Aliens" in the CMU CL manual --> + +<para> +Because of Lisp's emphasis on dynamic memory allocation and garbage +collection, Lisp implementations use non-C-like memory representations +for objects. This representation mismatch creates friction when a Lisp +program must share objects with programs which expect C data. There +are three common approaches to establishing communication: +<itemizedlist> + <listitem><para>The burden can be placed on the foreign program + (and programmer) by requiring the knowledge and use of the + representations used internally by the Lisp implementation. + This can require a considerable amount of <quote>glue</> code on the + C side, and that code tends to be sensitively dependent on the + internal implementation details of the Lisp system.</para></listitem> + <listitem><para>The Lisp system can automatically convert objects + back and forth between the Lisp and foreign representations. + This is convenient, but translation becomes prohibitively slow + when large or complex data structures must be shared. This approach + is supported by the &SBCL; <acronym>FFI</>, and used automatically + by the when passing integers and strings.</para></listitem> + <listitem><para>The Lisp program can directly manipulate foreign + objects through the use of extensions to the Lisp language. + </para></listitem> +</itemizedlist> + +<para>&SBCL;, like &CMUCL; before it, +relies primarily on the automatic conversion and direct manipulation +approaches. Foreign values of simple scalar types are automatically +converted, complex types are directly manipulated in their foreign +representation. Furthermore, Lisp strings are represented internally +with null termination bytes so that they can be passed directly to +C interfaces without allocating new zero-terminated copies.</para> + +<para>Any foreign objects that can't automatically be converted into +Lisp values are represented by objects of type <type>alien-value</>. +Since Lisp is a dynamically typed language, even foreign objects must +have a run-time type; this type information is provided by +encapsulating the raw pointer to the foreign data within an +<type>alien-value</> object.</para> + +<para>The type language and operations on foreign types are +intentionally similar to those of the C language.</para> + +</sect1> + +<sect1><title>Foreign Types</> +<!-- AKA "Alien Types" in the CMU CL manual --> + +<para>Alien types have a description language based on nested list +structure. For example the C type +<programlisting>struct foo { + int a; + struct foo *b[100]; +};</programlisting> +has the corresponding &SBCL; FFI type +<programlisting>(struct foo + (a int) + (b (array (* (struct foo)) 100)))</programlisting> +</para> + +<sect2><title>Defining Foreign Types</> + +<para> +Types may be either named or anonymous. With structure and union +types, the name is part of the type specifier, allowing recursively +defined types such as: +<programlisting>(struct foo (a (* (struct foo))))</programlisting> +An anonymous structure or union type is specified by using the name +<literal>nil</>. The <function>with-alien</> macro defines a local +scope which <quote>captures</> any named type definitions. Other types +are not inherently named, but can be given named abbreviations using +the <function>define-alien-type</> macro. +</para> + +</sect2> + +<sect2><title>Foreign Types and Lisp Types</> + +<para> +The foreign types form a subsystem of the &SBCL; type system. An +<type>alien</> type specifier provides a way to use any foreign type as a +Lisp type specifier. For example, +<programlisting>(typep foo '(alien (* int)))</programlisting> +can be used to determine whether <varname>foo</> is a pointer to a foreign +<type>int</>. <type>alien</> type specifiers can be used in the same ways +as ordinary Lisp type specifiers (like <type>string</>.) Alien type +declarations are subject to the same +precise type checking <!-- FIXME: should be linked to id="precisetypechecking" --> +as any other declaration. +</para> + +<para> +Note that the type identifiers used in the +foreign type system overlap with native Lisp type +specifiers in some cases. For example, the type specifier +<type>(alien single-float)</type> is identical to <type>single-float</>, since +foreign floats are automatically converted to Lisp floats. When +<function>type-of</> is called on an alien value that is not automatically +converted to a Lisp value, then it will return an <type>alien</> type +specifier. +</para> + +</sect2> + +<sect2><title>Foreign Type Specifiers</> + +<note><para> +All foreign type names are exported from the <literal>sb-alien</> +package. Some foreign type names are also symbols in +the <literal>common-lisp</> package, in which case they are +reexported from the <literal>sb-alien</> package, so that +e.g. it is legal to refer to <type>sb-alien:single-float</>. +</para></note> + +<para> +These are the basic foreign type specifiers: +<!-- FIXME: There must be some better way of formatting definitions + in DocBook than this. I haven't found it yet, but suggestions + or patches would be welcome. --> +<itemizedlist> + <listitem> + <para> + The foreign type specifier <type>(* foo)</> describes a + pointer to an object of type <type>foo</>. A pointed-to type + <type>foo</> of <type>t</> indicates a pointer to anything, + similar to <type>void *</> in ANSI C. A null alien pointer can + be detected with the <function>sb-alien:null-alien</> + function. + </para> + </listitem> + <listitem> + <para> + The foreign type specifier <type>(array foo &optional dimensions)</> + describes array of the specified <literal>dimensions</>, holding + elements of type <type>foo</>. Note that (unlike in C) <type>(* foo)</> + <type>(array foo)}</> are considered to be different types when + type checking is done. If equivalence of pointer and array types + is desired, it may be explicitly coerced using + <function>sb-alien:cast</>. + </para> + <para> + Arrays are accessed using <function>sb-alien:deref</>, passing + the indices as additional arguments. Elements are stored in + column-major order (as in C), so the first dimension determines + only the size of the memory block, and not the layout of the + higher dimensions. An array whose first dimension is variable + may be specified by using <literal>nil</> as the first dimension. + Fixed-size arrays can be allocated as array elements, structure + slots or <function>sb-alien:with-alien</> variables. Dynamic + arrays can only be allocated using <function>sb-alien:make-alien</>. + </para> + </listitem> + <listitem> + <para> + The foreign type specifier + <type>(sb-alien:struct name &rest fields)</> + describes a structure type with the specified <varname>name</> and + <varname>fields</>. Fields are allocated at the same offsets + used by the implementation's C compiler. If <varname>name</> + is <literal>nil</> then the structure is anonymous. + </para> + <para> + If a named foreign <type>struct</> specifier is passed to + <function>define-alien-type</> or <function>with-alien</>, + then this defines, respectively, a new global or local foreign + structure type. If no <varname>fields</> are specified, then + the fields are taken from the current (local or global) alien + structure type definition of <varname>name</>. + </para> + </listitem> + <listitem> + <para> + The foreign type specifier + <type>(sb-alien:union name &rest fields)</> + is similar to <type>sb-alien:struct</>, but describes a union type. + All fields are allocated at the same offset, and the size of the + union is the size of the largest field. The programmer must + determine which field is active from context. + </para> + </listitem> + <listitem> + <para> + The foreign type specifier <type>(sb-alien:enum name &rest specs)</> + describes an enumeration type that maps between integer values + and keywords. If <varname>name</> is <literal>nil</>, then the + type is anonymous. Each element of the <varname>specs</> + list is either a Lisp keyword, or a list <literal>(keyword value)</>. + <varname>value</> is an integer. If <varname>value</> is not + supplied, then it defaults to one greater than the value for + the preceding spec (or to zero if it is the first spec.) + <para> + </listitem> + <listitem> + <para> + The foreign type specifier <type>(sb-alien:signed &optional bits)</> + specifies a signed integer with the specified number of + <varname>bits</> precision. The upper limit on integer + precision is determined by the machine's word + size. If <varname>bits</> is not specified, the maximum + size will be used. + </para> + </listitem> + <listitem> + <para> + The foreign type specifier <type>(integer &optional bits)</> is + equivalent to the corresponding type specifier using + <type>sb-alien:signed</> instead of <type>integer</>. + </para> + </listitem> + <listitem> + <para> + The foreign type specifier + <type>(sb-alien:unsigned &optional bits)</> + is like corresponding type specifier using <type>sb-alien:signed</> + except that the variable is treated as an unsigned integer. + </para> + </listitem> + <listitem> + <para> + The foreign type specifier <type>(boolean &optional bits)</> is + similar to an enumeration type, but maps from Lisp <literal>nil</> + and <literal>t</> to C <literal>0</> and <literal>1</> + respectively. <varname>bits</> determines the amount of + storage allocated to hold the truth value. + </para> + </listitem> + <listitem> + <para> + The foreign type specifier <type>single-float</> describes a + floating-point number in IEEE single-precision format. + </para> + </listitem> + <listitem> + <para> + The foreign type specifier <type>double-float</> describes a + floating-point number in IEEE double-precision format. + </para> + </listitem> + <listitem> + <para> + The foreign type specifier + <type>(function result-type &rest arg-types)</> + describes a foreign function that takes arguments of the specified + <varname>arg-types</> and returns a result of type <type>result-type</>. + Note that the only context where a foreign <type>function</> type + is directly specified is in the argument to + <function>sb-alien:alien-funcall</>. + In all other contexts, foreign functions are represented by + foreign function pointer types: <type>(* (function ...))</>. + </para> + </listitem> + <listitem> + <para> + The foreign type specifier <type>sb-alien:system-area-pointer</> + describes a pointer which is represented in Lisp as a + <type>system-area-pointer</> object. &SBCL; exports this type from + <literal>sb-alien</> because &CMUCL; did, but tentatively (as of + the first draft of this section of the manual, &SBCL; 0.7.6) it is + deprecated, since it doesn't seem to be required by user code. + </para> + </listitem> + <listitem> + <para> + The foreign type specifier <type>sb-alien:void</> is + used in function types to declare that no useful value + is returned. Using <function>alien-funcall</> + to call a <type>void</> foreign function will return + zero values. + </para> + </listitem> + <listitem> + <para> + The foreign type specifier <type>sb-alien:c-string</> + is similar to <type>(* char)</>, but is interpreted as a + null-terminated string, and is automatically converted into a + Lisp string when accessed; or if the pointer is C <literal>NULL</> + or <literal>0</>, then accessing it gives Lisp <literal>nil</>. + </para> + <para> + Assigning a Lisp string to a <type>c-string</> structure field or + variable stores the contents of the string to the memory already + pointed to by that variable. When a foreign object of type + <type>(* char)</> is assigned to a <type>c-string</>, then the + <type>c-string</> pointer is assigned to. This allows + <type>c-string</> pointers to be initialized. For example: + <programlisting>(cl:in-package "CL-USER") ; which USEs package "SB-ALIEN" + (define-alien-type nil (struct foo (str c-string))) + (defun make-foo (str) (let ((my-foo (make-alien (struct foo)))) + (setf (slot my-foo 'str) (make-alien char (length str)) + (slot my-foo 'str) str) my-foo))</programlisting> + Storing Lisp <literal>NIL</> in a <type>c-string</> writes C + <literal>NULL</> to the variable. + </para> + </listitem> + <listitem> + <para> + <literal>sb-alien</> also exports translations of these C type + specifiers as foreign type specifiers: + <type>sb-alien:char</>, + <type>sb-alien:short</>, + <type>sb-alien:int</>, + <type>sb-alien:long</>, + <type>sb-alien:unsigned-char</>, + <type>sb-alien:unsigned-short</>, + <type>sb-alien:unsigned-int</>, + <type>sb-alien:unsigned-long</>, + <type>sb-alien:float</>, and + <type>sb-alien:double</>. + </para> + </listitem> + +</itemizedlist> + +</sect1> + +<sect1><title>Operations On Foreign Values</> +<!-- AKA "Alien Operations" in the CMU CL manual --> + +<para>This section describes how to read foreign values as Lisp +values, how to coerce foreign values to different kinds of foreign values, and +how to dynamically allocate and free foreign variables.</para> + +<sect2><title>Accessing Foreign Values</> + +<synopsis>(sb-alien:deref pointer-or-array &rest indices)</> + +<para>The <function>sb-alien:deref</> function returns the value pointed to by +a foreign pointer, or the value of a foreign array element. When +dereferencing a pointer, an optional single index can be specified to +give the equivalent of C pointer arithmetic; this index is scaled by +the size of the type pointed to. When dereferencing an array, the +number of indices must be the same as the number of dimensions in the +array type. <function>deref</> can be set with <function>setf</> to +assign a new value.</para> + +<synopsis>(sb-alien:slot struct-or-union &rest slot-names)</> + +<para>The <function>sb-alien:slot</> function extracts the value of +the slot named <varname>slot-name</> from a foreign <type>struct</> or +<type>union</>. If <varname>struct-or-union</> is a pointer to a +structure or union, then it is automatically dereferenced. +<function>sb-alien:slot</> can be set with <function>setf</> to assign +a new value. Note that <varname>slot-name</> is evaluated, and need +not be a compile-time constant (but only constant slot accesses are +efficiently compiled.)</para> + +</sect2> + +<sect2><title>Coercing Foreign Values</> + +<synopsis>(sb-alien:addr alien-expr)</> + +<para> +The <function>sb-alien:addr</> macro +returns a pointer to the location specified by +<varname>alien-expr</>, which must be either a foreign variable, a use of +<function>sb-alien:deref</>, a use of <function>sb-alien:slot</>, or a use of +<function>sb-alien:extern-alien</>. +</para> + +<synopsis>(sb-alien:cast foreign-value new-type)</> + +<para>The <function>sb-alien:cast</> +converts <varname>foreign-value</> to a new foreign value with the specified +<varname>new-type</>. Both types, old and new, must be foreign pointer, +array or function types. Note that the resulting Lisp +foreign variable object +is not <function>eq</> to the +argument, but it does refer to the same foreign data bits.</para> + +<synopsis>(sb-alien:sap-alien sap type)</> + +<para>The <function>sb-alien:sap-alien</> function converts <varname>sap</> +(a system area pointer) to a foreign value with the specified +<varname>type</>. <varname>type</> is not evaluated. +As of &SBCL; 0.7.6, it looks as though this and other SAP functionality +may become deprecated, since it shouldn't be needed by user code. +</para> + +<para>The <varname>type</> must be some foreign pointer, array, or +record type.</para> + +<synopsis>(sb-alien:alien-sap foreign-value type)</> + +<para>The <function>sb-alien:alien-sap</> function +returns the SAP which points to <varname>alien-value</>'s data. +As of &SBCL; 0.7.6, it looks as though this and other SAP functionality +may become deprecated, since it shouldn't be needed by user code. +</para> + +<para>The <varname>foreign-value</> must be of some foreign pointer, +array, or record type.</para> + +</sect2> + +<sect2><title>Foreign Dynamic Allocation</> + +<para>Lisp code can call the C standard library functions +<function>malloc</> and <function>free</> to dynamically allocate and +deallocate foreign variables. The Lisp code shares the same allocator +with foreign C code, so it's OK for foreign code to call +<function>free</> on the result of Lisp +<function>sb-alien:make-alien</>, or for Lisp code to call +<function>sb-alien:free-alien</> on foreign objects allocated by C +code.</para> + +<synopsis>(sb-alien:make-alien type size)</> + +<para>The <function>sb-alien:make-alien</> macro +returns a dynamically allocated foreign value of the specified +<varname>type</> (which is not evaluated.) The allocated memory is not +initialized, and may contain arbitrary junk. If supplied, +<varname>size</> is an expression to evaluate to compute the size of the +allocated object. There are two major cases: +<itemizedlist> + <listitem> + <para>When <varname>type</> is a foreign array type, an array of + that type is allocated and a pointer to it is returned. Note that you + must use <function>deref</> to change the result to an array before you + can use <function>deref</> to read or write elements: + <programlisting> + (cl:in-package "CL-USER") ; which USEs package "SB-ALIEN" + (defvar *foo* (make-alien (array char 10))) + (type-of *foo*) => (alien (* (array (signed 8) 10))) + (setf (deref (deref foo) 0) 10) => 10</programlisting> + If supplied, <varname>size</> is used as the first dimension for the + array.</para> + </listitem> + <listitem> + <para>When <varname>type</> is any other foreign type, then an + object for that type is allocated, and a pointer to it is + returned. So <function>(make-alien int)</> returns a <type>(* int)</>. + If <varname>size</> is specified, then a block of that many + objects is allocated, with the result pointing to the first one.</para> + </listitem> +</itemizedlist> +</para> + +<synopsis>(sb-alien:free-alien foreign-value)</> + +<para>The <function>sb-alien:free-alien</> function +frees the storage for <varname>foreign-value</>, +which must have been allocated with Lisp <function>make-alien</> +or C <function>malloc</>.</para> + +<para>See also the <function>sb-alien:with-alien</> macro, which +allocates foreign values on the stack.</para> + +</sect1> + +<sect1><title>Foreign Variables</> +<!-- AKA "Alien Variables" in the CMU CL manual --> + +<para> +Both local (stack allocated) and external (C global) foreign variables are +supported. +</para> + +<sect2><title>Local Foreign Variables</> + +<synopsis>(sb-alien:with-alien var-definitions &body body)</> + +<para>The <function>with-alien</> +macro establishes local +foreign variables +with the specified +alien types and names. +This form is analogous to defining a local variable in C: additional +storage is allocated, and the initial value is copied. +This form is less +analogous to LET-allocated Lisp variables, since the variables +can't be captured in closures: they live only for the dynamic extent +of the body, and referring to them outside is a gruesome error. +</para> + +<para>The <varname>var-definitions</> argument is a list of +variable definitions, each of the form +<programlisting>(name type &optional initial-value)</programlisting> +The names of the variables are established as symbol-macros; the bindings have +lexical scope, and may be assigned with <function>setq</> +or <function>setf</>. +</para> + +<para>The <function>with-alien</> macro also establishes +a new scope for named structures +and unions. Any <varname>type</> specified for a variable may contain +named structure or union types with the slots specified. Within the +lexical scope of the binding specifiers and body, a locally defined +foreign structure type <type>foo</> can be referenced by its name using +<type>(struct foo)</>. +</para> + +</sect2> + +<sect2><title>External Foreign Variables</> + +<para> +External foreign names are strings, and Lisp names are symbols. When +an external foreign value is represented using a Lisp variable, there +must be a way to convert from one name syntax into the other. The +macros <function>extern-alien</>, <function>define-alien-variable</> and +<function>define-alien-routine</> use this conversion heuristic: <itemizedlist> - <listitem><para>Type Translations</></> - <listitem><para>System Area Pointers</></> - <listitem><para>Alien Objects</></> - <listitem><para>Alien Types</></> - <listitem><para>Alien Operations</></> - <listitem><para>Alien Variables</></> - <listitem><para>Alien Function Calls</></> + <listitem><para>Alien names are converted to Lisp names by uppercasing and + replacing underscores with hyphens.</para></listitem> + <listitem><para>Conversely, Lisp names are converted to alien names by + lowercasing and replacing hyphens with underscores.</para></listitem> + <listitem><para>Both the Lisp symbol and alien string names may be + separately specified by using a list of the form + <programlisting>(alien-string lisp-symbol)</></para></listitem> </itemizedlist> + +<synopsis>(sb-alien:define-alien-variable name type)</> + +<para> +The <function>define-alien-variable</> macro +defines <varname>name</> as an external foreign variable of the +specified foreign <type>type</>. <varname>name</> and <type>type</> are not +evaluated. The Lisp name of the variable (see above) becomes a +global alien variable. Global alien variables +are effectively ``global symbol macros''; a reference to the +variable fetches the contents of the external variable. Similarly, +setting the variable stores new contents---the new contents must be +of the declared <type>type</>. Someday, they may well be implemented +using the &ANSI; <function>define-symbol-macro</> mechanism, but +as of &SBCL; 0.7.5, they are still implemented using an older +more-or-less parallel mechanism inherited from &CMUCL;. +</para> + +<para> +For example, to access a C-level counter <varname>foo</>, one could +write +<programlisting> +(define-alien-variable "foo" int) +;; Now it is possible to get the value of the C variable foo simply by +;; referencing that Lisp variable: +(print foo) +(setf foo 14) +(incf foo)</programlisting> </para> -</chapter> \ No newline at end of file +<synopsis>(sb-alien:get-errno)</> + +<para> +Since in modern C libraries, the <varname>errno</> "variable" is typically +no longer a variable, but some bizarre artificial construct +which behaves superficially like a variable within a given thread, +it can no longer reliably be accessed through the ordinary +<varname>define-alien-variable</> mechanism. Instead, &SBCL; provides +the operator <function>sb-alien:get-errno</> to allow Lisp code to read it. +</para> + +<synopsis>(sb-alien:extern-alien name type)</> + +<para> +The <function>extern-alien</> macro +returns an alien with the specified <type>type</> which +points to an externally defined value. <varname>name</> is not evaluated, +and may be either a string or a symbol. <type>type</> is +an unevaluated alien type specifier. +</para> + +</sect1> + +<sect1><title>Foreign Data Structure Examples</> +<!-- AKA "Alien Data Structure Example" in the CMU CL manual --> + +<para> +Now that we have alien types, operations and variables, we can manipulate +foreign data structures. This C declaration +<programlisting> +struct foo { + int a; + struct foo *b[100]; +};</programlisting> +can be translated into the following alien type: +<programlisting>(define-alien-type nil + (struct foo + (a int) + (b (array (* (struct foo)) 100))))</programlisting> +</para> + +<para> +Once the <type>foo</> alien type has been defined as above, +the C expression +<programlisting> +struct foo f; +f.b[7].a</programlisting> +can be translated in this way: +<programlisting> +(with-alien ((f (struct foo))) + (slot (deref (slot f 'b) 7) 'a) + ;; + ;; Do something with f... + )</programlisting> +</para> + +<para> +Or consider this example of an external C variable and some accesses: +<programlisting> +struct c_struct { + short x, y; + char a, b; + int z; + c_struct *n; +}; +extern struct c_struct *my_struct; +my_struct->x++; +my_struct->a = 5; +my_struct = my_struct->n;</programlisting> +which can be manipulated in Lisp like this: +<programlisting> +(define-alien-type nil + (struct c-struct + (x short) + (y short) + (a char) + (b char) + (z int) + (n (* c-struct)))) +(define-alien-variable "my_struct" (* c-struct)) +(incf (slot my-struct 'x)) +(setf (slot my-struct 'a) 5) +(setq my-struct (slot my-struct 'n))</programlisting> +</para> + +</sect1> + +<sect1><title>Loading Unix Object Files</> + +<para> +Foreign object files can be loaded into the running Lisp process by +calling the functions <function>load-foreign</> or +<function>load-1-foreign</>. +</para> + +<para> The <function>sb-alien:load-1-foreign</> function is the more +primitive of the two operations. It loads a single object file. into +the currently running Lisp. The external symbols defining routines and +variables are made available for future external references (e.g. by +<function>extern-alien</>). Forward references to foreign symbols +aren't supported: <function>load-1-foreign</> must be run before any +of the defined symbols are referenced. +</para> + +<para><function>sb-alien:load-foreign</> is built in terms of +<function>load-1-foreign</> and some other machinery +like <function>sb-ext:run-program</>. +It accepts a list of files and libraries, +and runs the linker on the files and +libraries, creating an absolute Unix object file which is then +processed by <function>load-1-foreign</>.</para> + +<note><para>As of &SBCL; 0.7.5, all foreign code (code loaded +with <function>load-1-function</> or <function>load-function</>) is +lost when a Lisp core is saved with +<function>sb-ext:save-lisp-and-die</>, and no attempt is made to +restore it when the core is loaded. Historically this has been an +annoyance both for &SBCL; users and for &CMUCL; users. +It's hard to solve this problem completely cleanly, but some +generally-reliable partial solution might be useful. Once someone in +either camp gets sufficiently annoyed to create it, &SBCL; is +likely to adopt some mechanism for automatically restoring foreign +code when a saved core is loaded.</para></note> + +</sect1> + +<sect1><title>Foreign Function Calls</> + +<para> +The foreign function call interface allows a Lisp program to call +functions written in other languages using the C calling convention. +</para> + +<para> +Lisp sets up various interrupt handling routines and other environment +information when it first starts up, and expects these to be in place +at all times. The C functions called by Lisp should either not change +the environment, especially the interrupt entry points, or should make +sure that these entry points are restored when the C function returns +to Lisp. If a C function makes changes without restoring things to the +way they were when the C function was entered, there is no telling +what will happen. +</para> + +<sect2><title>The <function>alien-funcall</> Primitive + +(sb-alien:alien-funcall alien-function &rest arguments) + + +The alien-funcall function is the foreign function call +primitive: alien-function is called with the supplied +arguments and its C return value is returned as a Lisp value. +The alien-function is an arbitrary +run-time expression; to refer to a constant function, use +extern-alien or a value defined by +define-alien-routine. + + + +The type of alien-function +must be (alien (function ...)) +or (alien (* (function ...))). +The function type is used to +determine how to call the function (as though it was declared with +a prototype.) The type need not be known at compile time, but only +known-type calls are efficiently compiled. Limitations: + + Structure type return values are not implemented. + Passing of structures by value is not implemented. + + + +Here is an example which allocates a (struct foo), calls a foreign +function to initialize it, then returns a Lisp vector of all the +(* (struct foo)) objects filled in by the foreign call: + +;; Allocate a foo on the stack. +(with-alien ((f (struct foo))) + ;; Call some C function to fill in foo fields. + (alien-funcall (extern-alien "mangle_foo" (function void (* foo))) + (addr f)) + ;; Find how many foos to use by getting the A field. + (let* ((num (slot f 'a)) + (result (make-array num))) + ;; Get a pointer to the array so that we don't have to keep extracting it: + (with-alien ((a (* (array (* (struct foo)) 100)) (addr (slot f 'b)))) + ;; Loop over the first N elements and stash them in the result vector. + (dotimes (i num) + (setf (svref result i) (deref (deref a) i))) + ;; Voila. + result))) + + + + +The <function>define-alien-routine</> Macro</> + +<synopsis>(sb-alien:define-alien-routine} name result-type &rest arg-specifiers)</> + +<para> +The <function>define-alien-routine</> macro is a convenience +for automatically generating Lisp +interfaces to simple foreign functions. The primary feature is the +parameter style specification, which translates the C +pass-by-reference idiom into additional return values. +</para> + +<para> +<varname>name</> is usually a string external symbol, but may also be a +symbol Lisp name or a list of the foreign name and the Lisp name. +If only one name is specified, the other is automatically derived +as for <function>extern-alien</>. +<varname>result-type</> is the alien type of the return value. +</para> + +<para> +Each element of the <varname>arg-specifiers</> list +specifies an argument to the foreign function, and is +of the form +<programlisting>(aname atype &optional style)</programlisting> +<varname>aname</> is the symbol name of the argument to the constructed +function (for documentation). <varname>atype</> is the alien type of +corresponding foreign argument. The semantics of the actual call +are the same as for <function>alien-funcall</>. <varname>style</> +specifies how this argument should be handled at call and return time, +and should be one of the following +<itemizedlist> + <listitem><para><varname>:in</>specifies that the argument is + passed by value. This is the default. <varname>:in</> arguments + have no corresponding return value from the Lisp function. + </para></listitem> + <listitem><para><varname>:copy</> is similar to <varname>:in</>, + but the argument is copied + to a pre-allocated object and a pointer to this object is passed + to the foreign routine.</para></listitem> + <listitem><para><varname>:out</> specifies a pass-by-reference + output value. The type of the argument must be a pointer to + a fixed-sized object (such as an integer or pointer). + <varname>:out</> and <varname>:in-out</> style cannot + be used with pointers to arrays, records or functions. An + object of the correct size is allocated on the stack, and + its address is passed to the foreign function. When the + function returns, the contents + of this location are returned as one of the values of the Lisp + function (and the location is automatically deallocated). + </para></listitem> + <listitem><para><varname>:in-out</> is a combination of + <varname>:copy</> and <varname>:out</>. + The argument is copied to a pre-allocated object and a pointer to + this object is passed to the foreign routine. On return, the + contents of this location is returned as an additional value. + </para></listitem> +</itemizedlist> +</para> + +<note> +<para> +Any efficiency-critical foreign interface function should be inline +expanded, which can be done by preceding the +<function>define-alien-routine</> call with: +<programlisting>(declaim (inline lisp-name))</programlisting> +In addition to avoiding the Lisp call overhead, this allows +pointers, word-integers and floats to be passed using non-descriptor +representations, avoiding consing.) +</para> +</note> + +</sect2> + +<sect2><title><function>define-alien-routine</> Example + + +Consider the C function cfoo +with the following calling convention: + +void +cfoo (str, a, i) + char *str; + char *a; /* update */ + int *i; /* out */ +{ + /* body of cfoo(...) */ +} +This can be described by the following call to +define-alien-routine: + +(define-alien-routine "cfoo" void + (str c-string) + (a char :in-out) + (i int :out)) +The Lisp function cfoo will have +two arguments (str and a) +and two return values (a and i). + + + +Calling Lisp From C</> + +<para> +Calling Lisp functions from C is sometimes possible, but is extremely +hackish and poorly supported as of &SBCL; 0.7.5. +See <function>funcall0</> ... <function>funcall3</> in +the runtime system. The +arguments must be valid &SBCL; object descriptors (so that +e.g. fixnums must be +left-shifted by 2.) As of &SBCL; 0.7.5, the format +of object descriptors is documented only by the source code and +by the old &CMUCL; "INTERNALS" documentation.</para> + +<para> Note that the garbage collector moves objects, and won't be +able to fix up any references in C variables, so either turn GC off or +don't keep Lisp pointers in C data unless they are to statically +allocated objects. It is possible to use the +<function>sb-ext:purify</> function to place live data structures in +static space so that they won't move during GC. </para> + +<!-- FIXME: This is a "changebar" section from the CMU CL manual. + I (WHN 2002-07-14) am not very familiar with this content, so + I'm not immediately prepared to try to update it for SBCL, and + I'm not feeling masochistic enough to work to encourage this + kind of low-level hack anyway. However, I acknowledge that callbacks + are sometimes really really necessary, so I include the original + text in case someone is hard-core enough to benefit from it. If + anyone brings the information up to date for SBCL, it belong + either in the main manual or on a CLiki SBCL Internals page. +LaTeX \subsection{Accessing Lisp Arrays} +LaTeX +LaTeX Due to the way \cmucl{} manages memory, the amount of memory that can +LaTeX be dynamically allocated by \code{malloc} or \funref{make-alien} is +LaTeX limited\footnote{\cmucl{} mmaps a large piece of memory for it's own +LaTeX use and this memory is typically about 8 MB above the start of the C +LaTeX heap. Thus, only about 8 MB of memory can be dynamically +LaTeX allocated.}. +LaTeX +LaTeX To overcome this limitation, it is possible to access the content of +LaTeX Lisp arrays which are limited only by the amount of physical memory +LaTeX and swap space available. However, this technique is only useful if +LaTeX the foreign function takes pointers to memory instead of allocating +LaTeX memory for itself. In latter case, you will have to modify the +LaTeX foreign functions. +LaTeX +LaTeX This technique takes advantage of the fact that \cmucl{} has +LaTeX specialized array types (\pxlref{specialized-array-types}) that match +LaTeX a typical C array. For example, a \code{(simple-array double-float +LaTeX (100))} is stored in memory in essentially the same way as the C +LaTeX array \code{double x[100]} would be. The following function allows us +LaTeX to get the physical address of such a Lisp array: +LaTeX \begin{example} +LaTeX (defun array-data-address (array) +LaTeX "Return the physical address of where the actual data of an array is +LaTeX stored. +LaTeX +LaTeX ARRAY must be a specialized array type in CMU Lisp. This means ARRAY +LaTeX must be an array of one of the following types: +LaTeX +LaTeX double-float +LaTeX single-float +LaTeX (unsigned-byte 32) +LaTeX (unsigned-byte 16) +LaTeX (unsigned-byte 8) +LaTeX (signed-byte 32) +LaTeX (signed-byte 16) +LaTeX (signed-byte 8) +LaTeX " +LaTeX (declare (type (or #+signed-array (array (signed-byte 8)) +LaTeX #+signed-array (array (signed-byte 16)) +LaTeX #+signed-array (array (signed-byte 32)) +LaTeX (array (unsigned-byte 8)) +LaTeX (array (unsigned-byte 16)) +LaTeX (array (unsigned-byte 32)) +LaTeX (array single-float) +LaTeX (array double-float)) +LaTeX array) +LaTeX (optimize (speed 3) (safety 0)) +LaTeX (ext:optimize-interface (safety 3))) +LaTeX ;; with-array-data will get us to the actual data. However, because +LaTeX ;; the array could have been displaced, we need to know where the +LaTeX ;; data starts. +LaTeX (lisp::with-array-data ((data array) +LaTeX (start) +LaTeX (end)) +LaTeX (declare (ignore end)) +LaTeX ;; DATA is a specialized simple-array. Memory is laid out like this: +LaTeX ;; +LaTeX ;; byte offset Value +LaTeX ;; 0 type code (should be 70 for double-float vector) +LaTeX ;; 4 4 * number of elements in vector +LaTeX ;; 8 1st element of vector +LaTeX ;; ... ... +LaTeX ;; +LaTeX (let ((addr (+ 8 (logandc1 7 (kernel:get-lisp-obj-address data)))) +LaTeX (type-size (let ((type (array-element-type data))) +LaTeX (cond ((or (equal type '(signed-byte 8)) +LaTeX (equal type '(unsigned-byte 8))) +LaTeX 1) +LaTeX ((or (equal type '(signed-byte 16)) +LaTeX (equal type '(unsigned-byte 16))) +LaTeX 2) +LaTeX ((or (equal type '(signed-byte 32)) +LaTeX (equal type '(unsigned-byte 32))) +LaTeX 4) +LaTeX ((equal type 'single-float) +LaTeX 4) +LaTeX ((equal type 'double-float) +LaTeX 8) +LaTeX (t +LaTeX (error "Unknown specialized array element type")))))) +LaTeX (declare (type (unsigned-byte 32) addr) +LaTeX (optimize (speed 3) (safety 0) (ext:inhibit-warnings 3))) +LaTeX (system:int-sap (the (unsigned-byte 32) +LaTeX (+ addr (* type-size start))))))) +LaTeX \end{example} +LaTeX +LaTeX Assume we have the C function below that we wish to use: +LaTeX \begin{example} +LaTeX double dotprod(double* x, double* y, int n) +LaTeX \{ +LaTeX int k; +LaTeX double sum = 0; +LaTeX +LaTeX for (k = 0; k < n; ++k) \{ +LaTeX sum += x[k] * y[k]; +LaTeX \} +LaTeX \} +LaTeX \end{example} +LaTeX The following example generates two large arrays in Lisp, and calls the C +LaTeX function to do the desired computation. This would not have been +LaTeX possible using \code{malloc} or \code{make-alien} since we need about +LaTeX 16 MB of memory to hold the two arrays. +LaTeX \begin{example} +LaTeX (define-alien-routine "dotprod" double +LaTeX (x (* double-float) :in) +LaTeX (y (* double-float) :in) +LaTeX (n int :in)) +LaTeX +LaTeX (let ((x (make-array 1000000 :element-type 'double-float)) +LaTeX (y (make-array 1000000 :element-type 'double-float))) +LaTeX ;; Initialize X and Y somehow +LaTeX (let ((x-addr (system:int-sap (array-data-address x))) +LaTeX (y-addr (system:int-sap (array-data-address y)))) +LaTeX (dotprod x-addr y-addr 1000000))) +LaTeX \end{example} +LaTeX In this example, it may be useful to wrap the inner \code{let} +LaTeX expression in an \code{unwind-protect} that first turns off garbage +LaTeX collection and then turns garbage collection on afterwards. This will +LaTeX prevent garbage collection from moving \code{x} and \code{y} after we +LaTeX have obtained the (now erroneous) addresses but before the call to +LaTeX \code{dotprod} is made. +LaTeX +--> + +</sect1> + +<sect1><title>Step-By-Step Example of the Foreign Function Interface</> + +<para> +This section presents a complete example of an interface to a somewhat +complicated C function. +</para> + +<para> +Suppose you have the following C function which you want to be able to +call from Lisp in the file <filename>test.c</> +<programlisting> +struct c_struct +{ + int x; + char *s; +}; + +struct c_struct *c_function (i, s, r, a) + int i; + char *s; + struct c_struct *r; + int a[10]; +{ + int j; + struct c_struct *r2; + + printf("i = %d\n", i); + printf("s = %s\n", s); + printf("r->x = %d\n", r->x); + printf("r->s = %s\n", r->s); + for (j = 0; j < 10; j++) printf("a[%d] = %d.\n", j, a[j]); + r2 = (struct c_struct *) malloc (sizeof(struct c_struct)); + r2->x = i + 5; + r2->s = "a C string"; + return(r2); +};</programlisting> +</para> + +<para> +It is possible to call this C function from Lisp using the file +<filename>test.lisp</> containing +<programlisting> +(cl:defpackage "TEST-C-CALL" (:use "CL" "SB-ALIEN" "SB-C-CALL")) +(cl:in-package "TEST-C-CALL") + +;;; Define the record C-STRUCT in Lisp. +(define-alien-type nil + (struct c-struct + (x int) + (s c-string))) + +;;; Define the Lisp function interface to the C routine. It returns a +;;; pointer to a record of type C-STRUCT. It accepts four parameters: +;;; I, an int; S, a pointer to a string; R, a pointer to a C-STRUCT +;;; record; and A, a pointer to the array of 10 ints. +;;; +;;; The INLINE declaration eliminates some efficiency notes about heap +;;; allocation of alien values. +(declaim (inline c-function)) +(define-alien-routine c-function + (* (struct c-struct)) + (i int) + (s c-string) + (r (* (struct c-struct))) + (a (array int 10))) + +;;; a function which sets up the parameters to the C function and +;;; actually calls it +(defun call-cfun () + (with-alien ((ar (array int 10)) + (c-struct (struct c-struct))) + (dotimes (i 10) ; Fill array. + (setf (deref ar i) i)) + (setf (slot c-struct 'x) 20) + (setf (slot c-struct 's) "a Lisp string") + + (with-alien ((res (* (struct c-struct)) + (c-function 5 "another Lisp string" (addr c-struct) ar))) + (format t "~&back from C function~%") + (multiple-value-prog1 + (values (slot res 'x) + (slot res 's)) + + ;; Deallocate result. (after we are done referring to it: + ;; "Pillage, *then* burn.") + (free-alien res)))))</programlisting> +</para> + +<para> +To execute the above example, it is necessary to compile the C routine, +e.g.: +<userinput>cc -c test.c</> +(In order to enable incremental loading with some linkers, you may need +to say +<userinput>cc -G 0 -c test.c</>) +</para> + +<para> +Once the C code has been compiled, you can start up Lisp and load it in: +<userinput>sbcl</>. +Lisp should start up with its normal prompt.</para> + +<para> +Within Lisp, +compile the Lisp file. (This step can be done separately. You don't +have to recompile every time.) +<userinput>(compile-file "test.lisp")</> + +<para> +Within Lisp, load the foreign object file to define the necessary +symbols: +<userinput>(load-foreign "test.o")</>. +This must be done before loading any code that refers +to these symbols. +<para> + +<para> +Now you can load the compiled Lisp ("fasl") file into Lisp: +<userinput>(load "test.fasl")</> +And once the Lisp file is loaded, you can call the +Lisp routine that sets up the parameters and calls the C +function: +<userinput>(test-c-call::call-cfun)</> +</para> + +<para> +The C routine should print the following information to standard output: +<!-- FIXME: What should be here is a verbatim environment for computer + output, but since I don't know one in DocBook, I made do with + PROGRAMLISTING for now... --> +<programlisting>i = 5 +s = another Lisp string +r->x = 20 +r->s = a Lisp string +a[0] = 0. +a[1] = 1. +a[2] = 2. +a[3] = 3. +a[4] = 4. +a[5] = 5. +a[6] = 6. +a[7] = 7. +a[8] = 8. +a[9] = 9.</programlisting> +After return from the C function, +the Lisp wrapper function should print the following output: +<programlisting>back from C function</programlisting> +And upon return from the Lisp wrapper function, +before the next prompt is printed, the +Lisp read-eval-print loop should print the following return values: +<!-- FIXME: As above, it's not a program listing, but computer output... --> +<programlisting> +10 +"a C string" +</programlisting> +</para> + +</sect1> + +</chapter>