1 \input texinfo @c -*- texinfo -*-
7 @c We use @&key, etc to escape & from TeX in lambda lists --
8 @c so we need to define them for info as well.
9 @macro &allow-other-keys
26 @dircategory Software development
28 * asdf: (asdf). Another System Definition Facility (for Common Lisp)
32 This manual describes ASDF, a system definition facility
33 for Common Lisp programs and libraries.
35 You can find the latest version of this manual at
36 @url{http://common-lisp.net/project/asdf/asdf.html}.
38 ASDF Copyright @copyright{} 2001-2010 Daniel Barlow and contributors.
40 This manual Copyright @copyright{} 2001-2010 Daniel Barlow and contributors.
42 This manual revised @copyright{} 2009-2010 Robert P. Goldman and Francois-Rene Rideau.
44 Permission is hereby granted, free of charge, to any person obtaining
45 a copy of this software and associated documentation files (the
46 ``Software''), to deal in the Software without restriction, including
47 without limitation the rights to use, copy, modify, merge, publish,
48 distribute, sublicense, and/or sell copies of the Software, and to
49 permit persons to whom the Software is furnished to do so, subject to
50 the following conditions:
52 The above copyright notice and this permission notice shall be
53 included in all copies or substantial portions of the Software.
55 THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND,
56 EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
57 MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
58 NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
59 LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
60 OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
61 WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
68 @title ASDF: Another System Definition Facility
70 @c The following two commands start the copyright page.
72 @vskip 0pt plus 1filll
76 @c Output the table of contents at the beginning.
79 @c -------------------
83 @node Top, Introduction, (dir), (dir)
84 @top asdf: another system definition facility
93 * Defining systems with defsystem::
94 * The object model of ASDF::
95 * Controlling where ASDF searches for systems::
96 * Controlling where ASDF saves compiled files::
98 * Miscellaneous additional functionality::
99 * Getting the latest version::
104 * Function and Class Index::
108 @c --- The Detailed Node Listing ---
110 @c Defining systems with defsystem
112 @c * The defsystem form::
113 @c * A more involved example::
114 @c * The defsystem grammar::
115 @c * Other code in .asd files::
117 @c The object model of ASDF
124 @c * Predefined operations of ASDF::
125 @c * Creating new operations::
129 @c * Common attributes of components::
130 @c * Pre-defined subclasses of component::
131 @c * Creating new component types::
135 @c * Pre-defined subclasses of component::
136 @c * Creating new component types::
143 @c -------------------
145 @node Introduction, Loading ASDF, Top, Top
146 @comment node-name, next, previous, up
147 @chapter Introduction
148 @cindex ASDF-related features
150 @cindex Testing for ASDF
151 @cindex ASDF versions
155 ASDF is Another System Definition Facility:
156 a tool for specifying how systems of Common Lisp software
157 are comprised of components (sub-systems and files),
158 and how to operate on these components in the right order
159 so that they can be compiled, loaded, tested, etc.
161 ASDF presents three faces:
162 one for users of Common Lisp software who want to reuse other people's code,
163 one for writers of Common Lisp software who want to specify how to build their systems,
164 one for implementers of Common Lisp extensions who want to extend the build system.
165 @xref{Using ASDF,,Loading a system},
166 to learn how to use ASDF to load a system.
167 @xref{Defining systems with defsystem},
168 to learn how to define a system of your own.
169 @xref{The object model of ASDF}, for a description of
170 the ASDF internals and how to extend ASDF.
173 We have released ASDF 2.000 on May 31st 2010.
174 It hopefully will have been it included
175 in all CL maintained implementations shortly afterwards.
176 @xref{FAQ,,``What has changed between ASDF 1 and ASDF 2?''}.
179 @node Loading ASDF, Configuring ASDF, Introduction, Top
180 @comment node-name, next, previous, up
181 @chapter Loading ASDF
182 @vindex *central-registry*
185 @findex compile-system
187 @cindex system directory designator
192 @c * Installing ASDF::
196 @section Loading a pre-installed ASDF
198 Many Lisp implementations include a copy of ASDF.
199 You can usually load this copy using Common Lisp's @code{require} function:
205 Consult your Lisp implementation's documentation for details.
207 Hopefully, ASDF 2 will soon be bundled with every Common Lisp implementation,
208 and you can load it that way.
211 @section Checking whether ASDF is loaded
213 To check whether ASDF is properly loaded in your current Lisp image,
214 you can run this form:
220 If it returns a string,
221 that is the version of ASDF that is currently installed.
223 If it raises an error,
224 then either ASDF is not loaded, or
225 you are using an old version of ASDF.
227 You can check whether an old version is loaded
228 by checking if the ASDF package is present.
229 The form below will allow you to programmatically determine
230 whether a recent version is loaded, an old version is loaded,
234 (or #+asdf2 (asdf:asdf-version) #+asdf :old)
237 If it returns a version number, that's the version of ASDF installed.
238 If it returns the keyword @code{:OLD},
239 then you're using an old version of ASDF (from before 1.635).
240 If it returns @code{NIL} then ASDF is not installed.
242 If you are running a version older than 2.000,
243 we recommend that you load a newer ASDF using the method below.
246 @section Upgrading ASDF
248 If your implementation does provide ASDF 2 or later,
249 and you want to upgrade to a more recent version,
250 just install ASDF like any other package
251 (see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below),
252 configure ASDF as usual (see @pxref{Configuring ASDF} below),
257 (asdf:load-system :asdf)
260 If on the other hand, your implementation only provides an old ASDF,
261 you will require a special configuration step and an old-style loading:
265 (push #p"@var{/path/to/new/asdf/}" asdf:*central-registry*)
266 (asdf:oos 'asdf:load-op :asdf)
269 Don't forget the trailing @code{/} at the end of your pathname.
271 Also, note that older versions of ASDF won't redirect their output,
272 or at least won't do it according to your usual ASDF 2 configuration.
273 You therefore need write access on the directory
274 where you install the new ASDF,
275 and make sure you're not using it
276 for multiple mutually incompatible implementations.
277 At worst, you may have to have multiple copies of the new ASDF,
278 e.g. one per implementation installation, to avoid clashes.
280 Finally, note that there are some limitations to upgrading ASDF:
283 Any ASDF extension is invalidated, and will need to be reloaded.
285 It is safer if you upgrade ASDF and its extensions as a special step
286 at the very beginning of whatever script you are running,
287 before you start using ASDF to load anything else.
291 @section Loading an otherwise installed ASDF
293 If your implementation doesn't include ASDF,
294 if for some reason the upgrade somehow fails,
295 does not or cannot apply to your case,
296 you will have to install the file @file{asdf.lisp}
297 somewhere and load it with:
300 (load "/path/to/your/installed/asdf.lisp")
303 The single file @file{asdf.lisp} is all you normally need to use ASDF.
305 You can extract this file from latest release tarball on the
306 @url{http://common-lisp.net/project/asdf/,ASDF website}.
307 If you are daring and willing to report bugs, you can get
308 the latest and greatest version of ASDF from its git repository.
309 @xref{Getting the latest version}.
311 For maximum convenience you might want to have ASDF loaded
312 whenever you start your Lisp implementation,
313 for example by loading it from the startup script or dumping a custom core
314 --- check your Lisp implementation's manual for details.
317 @node Configuring ASDF, Using ASDF, Loading ASDF, Top
318 @comment node-name, next, previous, up
320 @chapter Configuring ASDF
322 @section Configuring ASDF to find your systems
324 So it may compile and load your systems, ASDF must be configured to find
325 the @file{.asd} files that contain system definitions.
327 Since ASDF 2, the preferred way to configure where ASDF finds your systems is
328 the @code{source-registry} facility,
329 fully described in its own chapter of this manual.
330 @xref{Controlling where ASDF searches for systems}.
332 The default location for a user to install Common Lisp software is under
333 @file{~/.local/share/common-lisp/source/}.
334 If you install software there, you don't need further configuration.
335 If you're installing software yourself at a location that isn't standard,
336 you have to tell ASDF where you installed it. See below.
337 If you're using some tool to install software,
338 the authors of that tool should already have configured ASDF.
340 The simplest way to add a path to your search path,
341 say @file{/home/luser/.asd-link-farm/}
342 is to create the directory
343 @file{~/.config/common-lisp/source-registry.conf.d/}
344 and there create a file with any name of your choice but the type @file{conf},
345 for instance @file{42-asd-link-farm.conf}
348 @kbd{(:directory "/home/luser/.asd-link-farm/")}
350 If you want all the subdirectories under @file{/home/luser/lisp/}
351 to be recursively scanned for @file{.asd} files, instead use:
353 @kbd{(:tree "/home/luser/lisp/")}
355 Note that your Operating System distribution or your system administrator
356 may already have configured system-managed libraries for you.
358 The required @file{.conf} extension allows you to have disabled files
359 or editor backups (ending in @file{~}), and works portably
360 (for instance, it is a pain to allow both empty and non-empty extension on CLISP).
361 Excluded are files the name of which start with a @file{.} character.
362 It is customary to start the filename with two digits
363 that specify the order in which the directories will be scanned.
365 ASDF will automatically read your configuration
366 the first time you try to find a system.
367 You can reset the source-registry configuration with:
370 (asdf:clear-source-registry)
373 And you probably should do so before you dump your Lisp image,
374 if the configuration may change
375 between the machine where you save it at the time you save it
376 and the machine you resume it at the time you resume it.
379 @section Configuring ASDF to find your systems -- old style
381 The old way to configure ASDF to find your systems is by
382 @code{push}ing directory pathnames onto the variable
383 @code{asdf:*central-registry*}.
385 You must configure this variable between the time you load ASDF
386 and the time you first try to use it.
387 Loading and configuring ASDF presumably happen
388 as part of some initialization script that builds or starts
389 your Common Lisp software system.
390 (For instance, some SBCL users used to put it in their @file{~/.sbclrc}.)
392 The @code{asdf:*central-registry*} is empty by default in ASDF 2,
393 but is still supported for compatibility with ASDF 1.
394 When used, it takes precedence over the above source-registry@footnote{
395 It is possible to further customize
396 the system definition file search.
397 That's considered advanced use, and covered later:
399 @code{*system-definition-search-functions*}.
400 @xref{Defining systems with defsystem}.}.
402 For instance, if you wanted ASDF to find the @file{.asd} file
403 @file{/home/me/src/foo/foo.asd} your initialization script
404 could after it loads ASDF with @code{(require :asdf)}
408 (push "/home/me/src/foo/" asdf:*central-registry*)
411 Note the trailing slash: when searching for a system,
412 ASDF will evaluate each entry of the central registry
413 and coerce the result to a pathname@footnote{
414 ASDF will indeed call @code{EVAL} on each entry.
415 It will also skip entries that evaluate to @code{NIL}.
417 Strings and pathname objects are self-evaluating,
418 in which case the @code{EVAL} step does nothing;
419 but you may push arbitrary SEXP onto the central registry,
420 that will be evaluated to compute e.g. things that depend
421 on the value of shell variables or the identity of the user.
423 The variable @code{asdf:*central-registry*} is thus a list of
424 ``system directory designators''.
425 A @dfn{system directory designator} is a form
426 which will be evaluated whenever a system is to be found,
427 and must evaluate to a directory to look in.
428 By ``directory'' here, we mean
429 ``designator for a pathname with a supplied DIRECTORY component''.
431 at which point the presence of the trailing directory name separator
432 is necessary to tell Lisp that you're discussing a directory
435 Typically, however, there are a lot of @file{.asd} files, and
436 a common idiom was to have to put
437 a bunch of @emph{symbolic links} to @file{.asd} files
438 in a common directory
439 and push @emph{that} directory (the ``link farm'')
441 @code{asdf:*central-registry*}
442 instead of pushing each of the many involved directories
443 to the @code{asdf:*central-registry*}.
444 ASDF knows how to follow such @emph{symlinks}
445 to the actual file location when resolving the paths of system components
446 (on Windows, you can use Windows shortcuts instead of POSIX symlinks).
448 For example, if @code{#p"/home/me/cl/systems/"} (note the trailing slash)
449 is a member of @code{*central-registry*}, you could set up the
450 system @var{foo} for loading with asdf with the following
451 commands at the shell:
454 $ cd /home/me/cl/systems/
455 $ ln -s ~/src/foo/foo.asd .
458 This old style for configuring ASDF is not recommended for new users,
459 but it is supported for old users, and for users who want to programmatically
460 control what directories are added to the ASDF search path.
463 @section Configuring where ASDF stores object files
464 @findex clear-output-locations
466 ASDF lets you configure where object files will be stored.
467 Sensible defaults are provided and
468 you shouldn't normally have to worry about it.
470 This allows the same source code repository may be shared
471 between several versions of several Common Lisp implementations,
472 between several users using different compilation options
473 and without write privileges on shared source directories, etc.
474 This also allows to keep source directories uncluttered
475 by plenty of object files.
477 Starting with ASDF 2, the @code{asdf-output-translations} facility
478 was added to ASDF itself, that controls where object files will be stored.
479 This facility is fully described in a chapter of this manual,
480 @ref{Controlling where ASDF saves compiled files}.
482 The simplest way to add a translation to your search path,
483 say from @file{/foo/bar/baz/quux/}
484 to @file{/where/i/want/my/fasls/}
485 is to create the directory
486 @file{~/.config/common-lisp/asdf-output-translations.conf.d/}
487 and there create a file with any name of your choice and the type @file{conf},
488 for instance @file{42-bazquux.conf}
491 @kbd{("/foo/bar/baz/quux/" "/where/i/want/my/fasls/")}
493 To disable output translations for source under a given directory,
494 say @file{/toto/tata/}
495 you can create a file @file{40-disable-toto.conf}
498 @kbd{("/toto/tata/")}
500 To wholly disable output translations for all directories,
501 you can create a file @file{00-disable.conf}
506 Note that your Operating System distribution or your system administrator
507 may already have configured translations for you.
508 In absence of any configuration, the default is to redirect everything
509 under an implementation-dependent subdirectory of @file{~/.cache/common-lisp/}.
510 @xref{Controlling where ASDF searches for systems}, for full details.
512 The required @file{.conf} extension allows you to have disabled files
513 or editor backups (ending in @file{~}), and works portably
514 (for instance, it is a pain to allow both empty and non-empty extension on CLISP).
515 Excluded are files the name of which start with a @file{.} character.
516 It is customary to start the filename with two digits
517 that specify the order in which the directories will be scanned.
519 ASDF will automatically read your configuration
520 the first time you try to find a system.
521 You can reset the source-registry configuration with:
524 (asdf:clear-output-translations)
527 And you probably should do so before you dump your Lisp image,
528 if the configuration may change
529 between the machine where you save it at the time you save it
530 and the machine you resume it at the time you resume it.
532 Finally note that before ASDF 2,
533 other ASDF add-ons offered the same functionality,
534 each in subtly different and incompatible ways:
535 ASDF-Binary-Locations, cl-launch, common-lisp-controller.
536 ASDF-Binary-Locations is now not needed anymore and should not be used.
537 cl-launch 3.000 and common-lisp-controller 7.2 have been updated
538 to just delegate this functionality to ASDF.
540 @node Using ASDF, Defining systems with defsystem, Configuring ASDF, Top
541 @comment node-name, next, previous, up
545 @section Loading a system
547 The system @var{foo} is loaded (and compiled, if necessary)
548 by evaluating the following Lisp form:
551 (asdf:load-system :@var{foo})
554 On some implementations (namely ABCL, Clozure CL, CMUCL, ECL and SBCL),
555 ASDF hooks into the @code{CL:REQUIRE} facility
556 and you can just use:
562 In older versions of ASDF, you needed to use
563 @code{(asdf:oos 'asdf:load-op :@var{foo})}.
564 If your ASDF is too old to provide @code{asdf:load-system} though
565 we recommend that you upgrade to ASDF 2.
566 @xref{Loading ASDF,,Loading an otherwise installed ASDF}.
569 @section Other Operations
571 ASDF provides three commands for the most common system operations:
572 @code{load-system}, @code{compile-system} or @code{test-system}.
574 Because ASDF is an extensible system
575 for defining @emph{operations} on @emph{components},
576 it also provides a generic function @code{operate}
577 (which is usually abbreviated by @code{oos}).
578 You'll use @code{oos} whenever you want to do something beyond
579 compiling, loading and testing.
581 Output from ASDF and ASDF extensions are supposed to be sent
582 to the CL stream @code{*standard-output*},
583 and so rebinding that stream around calls to @code{asdf:operate}
584 should redirect all output from ASDF operations.
586 Reminder: before ASDF can operate on a system, however,
587 it must be able to find and load that system's definition.
588 @xref{Configuring ASDF,,Configuring ASDF to find your systems}.
597 Load ASDF itself into your Lisp image, either through
598 @code{(require :asdf)} or else through
599 @code{(load "/path/to/asdf.lisp")}.
602 Make sure ASDF can find system definitions
603 thanks to proper source-registry configuration.
606 Load a system with @code{(load-system :my-system)}
607 or use some other operation on some system of your choice.
613 That's all you need to know to use ASDF to load systems written by others.
614 The rest of this manual deals with writing system definitions
615 for Common Lisp software you write yourself,
616 including how to extend ASDF to define new operation and component types.
619 @node Defining systems with defsystem, The object model of ASDF, Using ASDF, Top
620 @comment node-name, next, previous, up
621 @chapter Defining systems with defsystem
623 This chapter describes how to use asdf to define systems and develop
628 * The defsystem form::
629 * A more involved example::
630 * The defsystem grammar::
631 * Other code in .asd files::
634 @node The defsystem form, A more involved example, Defining systems with defsystem, Defining systems with defsystem
635 @comment node-name, next, previous, up
636 @section The defsystem form
638 Systems can be constructed programmatically
639 by instantiating components using @code{make-instance}.
640 Most of the time, however, it is much more practical to use
641 a static @code{defsystem} form.
642 This section begins with an example of a system definition,
643 then gives the full grammar of @code{defsystem}.
645 Let's look at a simple system.
646 This is a complete file that would
647 usually be saved as @file{hello-lisp.asd}:
652 (defsystem "hello-lisp"
653 :description "hello-lisp: a sample Lisp system."
655 :author "Joe User <joe@@example.com>"
656 :licence "Public Domain"
657 :components ((:file "packages")
658 (:file "macros" :depends-on ("packages"))
659 (:file "hello" :depends-on ("macros"))))
662 Some notes about this example:
667 The file starts with an @code{in-package} form
668 to use package @code{asdf}.
669 You could instead start your definition by using
670 a qualified name @code{asdf:defsystem}.
673 If in addition to simply using @code{defsystem},
674 you are going to define functions,
675 create ASDF extension, globally bind symbols, etc.,
676 it is recommended that to avoid namespace pollution between systems,
677 you should create your own package for that purpose,
678 for instance replacing the above @code{(in-package :asdf)} with:
681 (defpackage :foo-system
684 (in-package :foo-system)
688 The @code{defsystem} form defines a system named @code{hello-lisp}
689 that contains three source files:
690 @file{packages}, @file{macros} and @file{hello}.
693 The file @file{macros} depends on @file{packages}
694 (presumably because the package it's in is defined in @file{packages}),
695 and the file @file{hello} depends on @file{macros}
696 (and hence, transitively on @file{packages}).
697 This means that ASDF will compile and load @file{packages} and @file{macros}
698 before starting the compilation of file @file{hello}.
701 The files are located in the same directory
702 as the file with the system definition.
703 ASDF resolves symbolic links (or Windows shortcuts)
704 before loading the system definition file and
705 stores its location in the resulting system@footnote{
706 It is possible, though almost never necessary, to override this behaviour.}.
707 This is a good thing because the user can move the system sources
708 without having to edit the system definition.
712 @node A more involved example, The defsystem grammar, The defsystem form, Defining systems with defsystem
713 @comment node-name, next, previous, up
714 @section A more involved example
716 Let's illustrate some more involved uses of @code{defsystem} via a
717 slightly convoluted example:
722 :components ((:module "foo" :components ((:file "bar") (:file"baz")
724 :perform (compile-op :after (op c)
726 :explain (compile-op :after (op c)
727 (explain-something c)))
731 The method-form tokens need explaining: essentially, this part:
734 :perform (compile-op :after (op c)
736 :explain (compile-op :after (op c)
737 (explain-something c))
743 (defmethod perform :after ((op compile-op) (c (eql ...)))
745 (defmethod explain :after ((op compile-op) (c (eql ...)))
746 (explain-something c))
749 where @code{...} is the component in question;
750 note that although this also supports @code{:before} methods,
751 they may not do what you want them to ---
752 a @code{:before} method on perform @code{((op compile-op) (c (eql ...)))}
753 will run after all the dependencies and sub-components have been processed,
754 but before the component in question has been compiled.
756 @node The defsystem grammar, Other code in .asd files, A more involved example, Defining systems with defsystem
757 @comment node-name, next, previous, up
758 @section The defsystem grammar
761 system-definition := ( defsystem system-designator @var{option}* )
763 option := :components component-list
764 | :pathname pathname-specifier
765 | :default-component-class
766 | :perform method-form
767 | :explain method-form
768 | :output-files method-form
769 | :operation-done-p method-form
770 | :depends-on ( @var{dependency-def}* )
771 | :serial [ t | nil ]
772 | :in-order-to ( @var{dependency}+ )
774 component-list := ( @var{component-def}* )
776 component-def := ( component-type simple-component-name @var{option}* )
778 component-type := :system | :module | :file | :static-file | other-component-type
780 other-component-type := symbol-by-name (@pxref{The defsystem grammar,,Component types})
782 dependency-def := simple-component-name
784 | ( :version simple-component-name version-specifier)
786 dependency := (dependent-op @var{requirement}+)
787 requirement := (required-op @var{required-component}+)
788 | (feature feature-name)
789 dependent-op := operation-name
790 required-op := operation-name | feature
792 simple-component-name := string
795 pathname-specifier := pathname | string | symbol
797 method-form := (operation-name qual lambda-list @&rest body)
798 qual := method qualifier
801 @subsection Component names
803 Component names (@code{simple-component-name})
804 may be either strings or symbols.
806 @subsection Component types
808 Component type names, even if expressed as keywords, will be looked up
809 by name in the current package and in the asdf package, if not found in
810 the current package. So a component type @code{my-component-type}, in
811 the current package @code{my-system-asd} can be specified as
812 @code{:my-component-type}, or @code{my-component-type}.
814 @subsection Pathname specifiers
815 @cindex pathname specifiers
817 A pathname specifier (@code{pathname-specifier})
818 may be a pathname, a string or a symbol.
819 When no pathname specifier is given for a component,
820 which is the usual case, the component name itself is used.
822 If a string is given, which is the usual case,
823 the string will be interpreted as a Unix-style pathname
824 where @code{/} characters will be interpreted as directory separators.
825 Usually, Unix-style relative pathnames are used
826 (i.e. not starting with @code{/}, as opposed to absolute pathnames);
827 they are relative to the path of the parent component.
828 Finally, depending on the @code{component-type},
829 the pathname may be interpreted as either a file or a directory,
831 a file type may be added corresponding to the @code{component-type},
832 or else it will be extracted from the string itself (if applicable).
834 For instance, the @code{component-type} @code{:module}
835 wants a directory pathname, and so a string @code{"foo/bar"}
836 will be interpreted as the pathname @file{#p"foo/bar/"}.
837 On the other hand, the @code{component-type} @code{:file}
838 wants a file of type @code{lisp}, and so a string @code{"foo/bar"}
839 will be interpreted as the pathname @file{#p"foo/bar.lisp"},
840 and a string @code{"foo/bar.quux"}
841 will be interpreted as the pathname @file{#p"foo/bar.quux.lisp"}.
842 Finally, the @code{component-type} @code{:static-file}
843 wants a file without specifying a type, and so a string @code{"foo/bar"}
844 will be interpreted as the pathname @file{#p"foo/bar"},
845 and a string @code{"foo/bar.quux"}
846 will be interpreted as the pathname @file{#p"foo/bar.quux"}.
848 ASDF does not interpret the string @code{".."} to designate the parent
849 directory. This string will be passed through to the underlying
850 operating system for interpretation. We @emph{believe} that this will
851 work on all platforms where ASDF is deployed, but do not guarantee this
852 behavior. A pathname object with a relative directory component of
853 @code{:up} or @code{:back} is the only guaranteed way to specify a
856 If a symbol is given, it will be translated into a string,
857 and downcased in the process.
858 The downcasing of symbols is unconventional,
859 but was selected after some consideration.
860 Observations suggest that the type of systems we want to support
861 either have lowercase as customary case (Unix, Mac, windows)
862 or silently convert lowercase to uppercase (lpns),
863 so this makes more sense than attempting to use @code{:case :common}
864 as argument to @code{make-pathname},
865 which is reported not to work on some implementations.
867 Pathname objects may be given to override the path for a component.
868 Such objects are typically specified using reader macros such as @code{#p}
869 or @code{#.(make-pathname ...)}.
870 Note however, that @code{#p...} is a shorthand for @code{#.(parse-namestring ...)}
871 and that the behavior of @code{parse-namestring} is completely non-portable,
872 unless you are using Common Lisp @code{logical-pathname}s
873 (@pxref{The defsystem grammar,,Warning about logical pathnames}, below).
874 Pathnames made with @code{#.(make-pathname ...)}
875 can usually be done more easily with the string syntax above.
876 The only case that you really need a pathname object is to override
877 the component-type default file type for a given component.
878 Therefore, pathname objects should only rarely be used.
879 Unhappily, ASDF 1 didn't properly support
880 parsing component names as strings specifying paths with directories,
881 and the cumbersome @code{#.(make-pathname ...)} syntax had to be used.
883 Note that when specifying pathname objects,
884 ASDF does not do any special interpretation of the pathname
885 influenced by the component type, unlike the procedure for
886 pathname-specifying strings.
887 On the one hand, you have to be careful to provide a pathname that correctly
888 fulfills whatever constraints are required from that component type
889 (e.g. naming a directory or a file with appropriate type);
890 on the other hand, you can circumvent the file type that would otherwise
891 be forced upon you if you were specifying a string.
894 @subsection Warning about logical pathnames
895 @cindex logical pathnames
897 We recommend that you not use logical pathnames
898 in your asdf system definitions at this point,
899 but logical pathnames @emph{are} supported.
901 To use logical pathnames,
902 you will have to provide a pathname object as a @code{:pathname} specifier
903 to components that use it, using such syntax as
904 @code{#p"LOGICAL-HOST:absolute;path;to;component.lisp"}.
906 You only have to specify such logical pathname for your system or
907 some top-level component. Sub-components' relative pathnames, specified
908 using the string syntax
909 for names, will be properly merged with the pathnames of their parents.
910 The specification of a logical pathname host however is @emph{not}
911 otherwise directly supported in the ASDF syntax
912 for pathname specifiers as strings.
914 The @code{asdf-output-translation} layer will
915 avoid trying to resolve and translate logical-pathnames.
916 The advantage of this is that you can define yourself what translations you want to use
917 with the logical pathname facility.
918 The disadvantage is that if you do not define such translations, any
919 system that uses logical pathnames will be have differently under
920 asdf-output-translations than other systems you use.
922 If you wish to use logical pathnames you will have to configure the
923 translations yourself before they may be used.
924 ASDF currently provides no specific support
925 for defining logical pathname translations.
928 @subsection Serial dependencies
929 @cindex serial dependencies
931 If the @code{:serial t} option is specified for a module,
932 ASDF will add dependencies for each each child component,
933 on all the children textually preceding it.
934 This is done as if by @code{:depends-on}.
938 :components ((:file "a") (:file "b") (:file "c"))
944 :components ((:file "a")
945 (:file "b" :depends-on ("a"))
946 (:file "c" :depends-on ("a" "b")))
950 @subsection Source location
952 The @code{:pathname} option is optional in all cases for systems
953 defined via @code{defsystem},
954 and in the usual case the user is recommended not to supply it.
956 Instead, ASDF follows a hairy set of rules that are designed so that
960 will load a system from disk
961 and have its pathname default to the right place.
963 This pathname information will not be overwritten with
964 @code{*default-pathname-defaults*}
965 (which could be somewhere else altogether)
966 if the user loads up the @file{.asd} file into his editor
967 and interactively re-evaluates that form.
970 If a system is being loaded for the first time,
971 its top-level pathname will be set to:
975 The host/device/directory parts of @code{*load-truename*},
978 @code{*default-pathname-defaults*}, otherwise.
981 If a system is being redefined, the top-level pathname will be
985 changed, if explicitly supplied or obtained from @code{*load-truename*}
986 (so that an updated source location is reflected in the system definition)
988 changed if it had previously been set from @code{*default-pathname-defaults*}
990 left as before, if it had previously been set from @code{*load-truename*}
991 and @code{*load-truename*} is currently unbound
992 (so that a developer can evaluate a @code{defsystem} form
993 from within an editor without clobbering its source location)
996 @node Other code in .asd files, , The defsystem grammar, Defining systems with defsystem
997 @section Other code in .asd files
999 Files containing @code{defsystem} forms
1000 are regular Lisp files that are executed by @code{load}.
1001 Consequently, you can put whatever Lisp code you like into these files
1002 (e.g., code that examines the compile-time environment
1003 and adds appropriate features to @code{*features*}).
1004 However, some conventions should be followed,
1005 so that users can control certain details of execution
1006 of the Lisp in @file{.asd} files:
1010 Any informative output
1011 (other than warnings and errors,
1012 which are the condition system's to dispose of)
1013 should be sent to the standard CL stream @code{*standard-output*},
1014 so that users can easily control the disposition
1015 of output from ASDF operations.
1019 @node The object model of ASDF, Controlling where ASDF searches for systems, Defining systems with defsystem, Top
1020 @comment node-name, next, previous, up
1021 @chapter The object model of ASDF
1023 ASDF is designed in an object-oriented way from the ground up.
1024 Both a system's structure and the operations that can be performed on systems
1026 ASDF is extensible to new operations and to new component types.
1027 This allows the addition of behaviours:
1028 for example, a new component could be added for Java JAR archives,
1029 and methods specialised on @code{compile-op} added for it
1030 that would accomplish the relevant actions.
1032 This chapter deals with @emph{components}, the building blocks of a system,
1033 and @emph{operations}, the actions that can be performed on a system.
1042 @node Operations, Components, The object model of ASDF, The object model of ASDF
1043 @comment node-name, next, previous, up
1047 An @dfn{operation} object of the appropriate type is instantiated
1048 whenever the user wants to do something with a system like
1051 @item compile all its files
1052 @item load the files into a running lisp environment
1053 @item copy its source files somewhere else
1056 Operations can be invoked directly, or examined
1057 to see what their effects would be without performing them.
1058 @emph{FIXME: document how!}
1059 There are a bunch of methods specialised on operation and component type
1060 that actually do the grunt work.
1062 The operation object contains whatever state is relevant for this purpose
1063 (perhaps a list of visited nodes, for example)
1064 but primarily is a nice thing to specialise operation methods on
1065 and easier than having them all be @code{EQL} methods.
1067 Operations are invoked on systems via @code{operate}.
1069 @deffn {Generic function} @code{operate} @var{operation} @var{system} @&rest @var{initargs}
1070 @deffnx {Generic function} @code{oos} @var{operation} @var{system} @&rest @var{initargs}
1071 @code{operate} invokes @var{operation} on @var{system}.
1072 @code{oos} is a synonym for @code{operate}.
1074 @var{operation} is a symbol that is passed, along with the supplied
1075 @var{initargs}, to @code{make-instance} to create the operation object.
1076 @var{system} is a system designator.
1078 The @var{initargs} are passed to the @code{make-instance} call
1079 when creating the operation object.
1080 Note that dependencies may cause the operation
1081 to invoke other operations on the system or its components:
1082 the new operations will be created
1083 with the same @var{initargs} as the original one.
1088 * Predefined operations of ASDF::
1089 * Creating new operations::
1092 @node Predefined operations of ASDF, Creating new operations, Operations, Operations
1093 @comment node-name, next, previous, up
1094 @subsection Predefined operations of ASDF
1096 All the operations described in this section are in the @code{asdf} package.
1097 They are invoked via the @code{operate} generic function.
1100 (asdf:operate 'asdf:@var{operation-name} :@var{system-name} @{@var{operation-options ...}@})
1103 @deffn Operation @code{compile-op} @&key @code{proclamations}
1105 This operation compiles the specified component.
1106 If proclamations are supplied, they will be proclaimed.
1107 This is a good place to specify optimization settings.
1109 When creating a new component type,
1110 you should provide methods for @code{compile-op}.
1112 When @code{compile-op} is invoked,
1113 component dependencies often cause some parts of the system
1114 to be loaded as well as compiled.
1115 Invoking @code{compile-op}
1116 does not necessarily load all the parts of the system, though;
1117 use @code{load-op} to load a system.
1120 @deffn Operation @code{load-op} @&key @code{proclamations}
1122 This operation loads a system.
1124 The default methods for @code{load-op} compile files before loading them.
1125 For parity, your own methods on new component types should probably do so too.
1128 @deffn Operation @code{load-source-op}
1130 This operation will load the source for the files in a module
1131 even if the source files have been compiled.
1132 Systems sometimes have knotty dependencies
1133 which require that sources are loaded
1134 before they can be compiled.
1135 This is how you do that.
1137 If you are creating a component type,
1138 you need to implement this operation --- at least, where meaningful.
1142 @deffn Operation @code{test-op}
1144 This operation will perform some tests on the module.
1145 The default method will do nothing.
1146 The default dependency is to require
1147 @code{load-op} to be performed on the module first.
1148 The default @code{operation-done-p} is that the operation is @emph{never} done
1150 we assume that if you invoke the @code{test-op},
1151 you want to test the system, even if you have already done so.
1153 The results of this operation are not defined by ASDF.
1154 It has proven difficult to define how the test operation
1155 should signal its results to the user
1156 in a way that is compatible with all of the various test libraries
1157 and test techniques in use in the community.
1160 @c @deffn Operation test-system-version @&key minimum
1162 @c Asks the system whether it satisfies a version requirement.
1164 @c The default method accepts a string, which is expected to contain of a
1165 @c number of integers separated by #\. characters. The method is not
1166 @c recursive. The component satisfies the version dependency if it has
1167 @c the same major number as required and each of its sub-versions is
1168 @c greater than or equal to the sub-version number required.
1171 @c (defun version-satisfies (x y)
1172 @c (labels ((bigger (x y)
1173 @c (cond ((not y) t)
1175 @c ((> (car x) (car y)) t)
1176 @c ((= (car x) (car y))
1177 @c (bigger (cdr x) (cdr y))))))
1178 @c (and (= (car x) (car y))
1179 @c (or (not (cdr y)) (bigger (cdr x) (cdr y))))))
1182 @c If that doesn't work for your system, you can override it. I hope
1183 @c you have as much fun writing the new method as @verb{|#lisp|} did
1184 @c reimplementing this one.
1187 @c @deffn Operation feature-dependent-op
1189 @c An instance of @code{feature-dependent-op} will ignore any components
1190 @c which have a @code{features} attribute, unless the feature combination
1191 @c it designates is satisfied by @code{*features*}. This operation is
1192 @c not intended to be instantiated directly, but other operations may
1197 @node Creating new operations, , Predefined operations of ASDF, Operations
1198 @comment node-name, next, previous, up
1199 @subsection Creating new operations
1201 ASDF was designed to be extensible in an object-oriented fashion.
1202 To teach ASDF new tricks, a programmer can implement the behaviour he wants
1203 by creating a subclass of @code{operation}.
1205 ASDF's pre-defined operations are in no way ``privileged'',
1206 but it is requested that developers never use the @code{asdf} package
1207 for operations they develop themselves.
1208 The rationale for this rule is that we don't want to establish a
1209 ``global asdf operation name registry'',
1210 but also want to avoid name clashes.
1212 An operation must provide methods for the following generic functions
1213 when invoked with an object of type @code{source-file}:
1214 @emph{FIXME describe this better}
1218 @item @code{output-files}
1219 The @code{output-files} method determines where the method will put its files.
1220 It returns two values, a list of pathnames, and a boolean.
1221 If the boolean is @code{T} then the pathnames are marked
1222 not be translated by enclosing @code{:around} methods.
1223 If the boolean is @code{NIL} then enclosing @code{:around} methods
1224 may translate these pathnames, e.g. to ensure object files
1225 are somehow stored in some implementation-dependent cache.
1226 @item @code{perform}
1227 The @code{perform} method must call @code{output-files}
1228 to find out where to put its files,
1229 because the user is allowed to override.
1230 @item @code{output-files}
1231 for local policy @code{explain}
1232 @item @code{operation-done-p},
1233 if you don't like the default one
1237 Operations that print output should send that output to the standard
1238 CL stream @code{*standard-output*}, as the Lisp compiler and loader do.
1240 @node Components, , Operations, The object model of ASDF
1241 @comment node-name, next, previous, up
1245 @cindex system designator
1246 @vindex *system-definition-search-functions*
1248 A @dfn{component} represents a source file or
1249 (recursively) a collection of components.
1250 A @dfn{system} is (roughly speaking) a top-level component
1251 that can be found via @code{find-system}.
1253 A @dfn{system designator} is a string or symbol
1254 and behaves just like any other component name
1255 (including with regard to the case conversion rules for component names).
1258 @defun find-system system-designator &optional (error-p t)
1260 Given a system designator, @code{find-system} finds and returns a system.
1261 If no system is found, an error of type
1262 @code{missing-component} is thrown,
1263 or @code{nil} is returned if @code{error-p} is false.
1265 To find and update systems, @code{find-system} funcalls each element
1266 in the @code{*system-definition-search-functions*} list,
1267 expecting a pathname to be returned.
1268 The resulting pathname is loaded if either of the following conditions is true:
1272 there is no system of that name in memory
1274 the file's @code{last-modified} time exceeds the @code{last-modified} time
1275 of the system in memory
1278 When system definitions are loaded from @file{.asd} files,
1279 a new scratch package is created for them to load into,
1280 so that different systems do not overwrite each others operations.
1281 The user may also wish to (and is recommended to)
1282 include @code{defpackage} and @code{in-package} forms
1283 in his system definition files, however,
1284 so that they can be loaded manually if need be.
1286 The default value of @code{*system-definition-search-functions*}
1287 is a list of two functions.
1288 The first function looks in each of the directories given
1289 by evaluating members of @code{*central-registry*}
1290 for a file whose name is the name of the system and whose type is @file{asd}.
1291 The first such file is returned,
1292 whether or not it turns out to actually define the appropriate system.
1293 The second function does something similar,
1294 for the directories specified in the @code{source-registry}.
1295 Hence, it is strongly advised to define a system
1296 @var{foo} in the corresponding file @var{foo.asd}.
1301 * Common attributes of components::
1302 * Pre-defined subclasses of component::
1303 * Creating new component types::
1306 @node Common attributes of components, Pre-defined subclasses of component, Components, Components
1307 @comment node-name, next, previous, up
1308 @subsection Common attributes of components
1310 All components, regardless of type, have the following attributes.
1311 All attributes except @code{name} are optional.
1315 A component name is a string or a symbol.
1316 If a symbol, its name is taken and lowercased.
1318 Unless overridden by a @code{:pathname} attribute,
1319 the name will be interpreted as a pathname specifier according
1320 to a Unix-style syntax.
1321 @xref{The defsystem grammar,,Pathname specifiers}.
1323 @subsubsection Version identifier
1325 This optional attribute is used by the @code{test-system-version} operation.
1326 @xref{Predefined operations of ASDF}.
1327 For the default method of @code{test-system-version},
1328 the version should be a string of integers separated by dots,
1329 for example @samp{1.0.11}.
1332 This operation, planned for ASDF 1,
1333 is still not implement yet as of ASDF 2.
1334 Don't hold your breath.
1337 @subsubsection Required features
1339 @emph{FIXME: This subsection seems to contradict the
1340 @code{defsystem} grammar subsection,
1341 which doesn't provide any obvious way to specify required features.
1342 Furthermore, in 2009, discussions on the
1343 @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1344 suggested that the specification of required features may be broken,
1345 and that no one may have been using them for a while.
1347 @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1348 if you are interested in getting this features feature fixed.}
1350 Traditionally defsystem users have used reader conditionals
1351 to include or exclude specific per-implementation files.
1352 This means that any single implementation cannot read the entire system,
1353 which becomes a problem if it doesn't wish to compile it,
1354 but instead for example to create an archive file containing all the sources,
1355 as it will omit to process the system-dependent sources for other systems.
1357 Each component in an asdf system may therefore specify features using
1358 the same syntax as @code{#+} does, and it will (somehow) be ignored for
1359 certain operations unless the feature conditional is a member of
1363 @subsubsection Dependencies
1365 This attribute specifies dependencies of the component on its siblings.
1366 It is optional but often necessary.
1368 There is an excitingly complicated relationship between the initarg
1369 and the method that you use to ask about dependencies
1371 Dependencies are between (operation component) pairs.
1372 In your initargs for the component, you can say
1375 :in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
1376 (load-op (load-op "foo")))
1379 This means the following things:
1382 before performing compile-op on this component, we must perform
1383 load-op on @var{a} and @var{b}, and compile-op on @var{c},
1385 before performing @code{load-op}, we have to load @var{foo}
1388 The syntax is approximately
1391 (this-op {(other-op required-components)}+)
1393 required-components := component-name
1394 | (required-components required-components)
1396 component-name := string
1397 | (:version string minimum-version-object)
1402 This is on a par with what ACL defsystem does.
1403 mk-defsystem is less general: it has an implied dependency
1406 for all x, (load x) depends on (compile x)
1409 and using a @code{:depends-on} argument to say that @var{b} depends on
1410 @var{a} @emph{actually} means that
1413 (compile b) depends on (load a)
1416 This is insufficient for e.g. the McCLIM system, which requires that
1417 all the files are loaded before any of them can be compiled ]
1421 In ASDF, the dependency information for a given component and operation
1422 can be queried using @code{(component-depends-on operation component)},
1423 which returns a list
1426 ((load-op "a") (load-op "b") (compile-op "c") ...)
1429 @code{component-depends-on} can be subclassed for more specific
1430 component/operation types: these need to @code{(call-next-method)}
1431 and append the answer to their dependency, unless
1432 they have a good reason for completely overriding the default dependencies.
1434 If it weren't for CLISP, we'd be using @code{LIST} method
1435 combination to do this transparently.
1436 But, we need to support CLISP.
1437 If you have the time for some CLISP hacking,
1438 I'm sure they'd welcome your fixes.
1439 @c Doesn't CLISP now support LIST method combination?
1441 @subsubsection pathname
1443 This attribute is optional and if absent (which is the usual case),
1444 the component name will be used.
1446 @xref{The defsystem grammar,,Pathname specifiers},
1447 for an explanation of how this attribute is interpreted.
1449 Note that the @code{defsystem} macro (used to create a ``top-level'' system)
1450 does additional processing to set the filesystem location of
1451 the top component in that system.
1452 This is detailed elsewhere. @xref{Defining systems with defsystem}.
1454 The answer to the frequently asked question
1455 ``how do I create a system definition
1456 where all the source files have a @file{.cl} extension''
1460 (defmethod source-file-type ((c cl-source-file) (s (eql (find-system 'my-sys))))
1464 @subsubsection properties
1466 This attribute is optional.
1468 Packaging systems often require information about files or systems
1469 in addition to that specified by ASDF's pre-defined component attributes.
1470 Programs that create vendor packages out of ASDF systems therefore
1471 have to create ``placeholder'' information to satisfy these systems.
1472 Sometimes the creator of an ASDF system may know the additional
1473 information and wish to provide it directly.
1475 @code{(component-property component property-name)} and
1476 associated @code{setf} method will allow
1477 the programmatic update of this information.
1478 Property names are compared as if by @code{EQL},
1479 so use symbols or keywords or something.
1482 * Pre-defined subclasses of component::
1483 * Creating new component types::
1486 @node Pre-defined subclasses of component, Creating new component types, Common attributes of components, Components
1487 @comment node-name, next, previous, up
1488 @subsection Pre-defined subclasses of component
1490 @deffn Component source-file
1492 A source file is any file that the system does not know how to
1493 generate from other components of the system.
1495 Note that this is not necessarily the same thing as
1496 ``a file containing data that is typically fed to a compiler''.
1497 If a file is generated by some pre-processor stage
1498 (e.g. a @file{.h} file from @file{.h.in} by autoconf)
1499 then it is not, by this definition, a source file.
1500 Conversely, we might have a graphic file
1501 that cannot be automatically regenerated,
1502 or a proprietary shared library that we received as a binary:
1503 these do count as source files for our purposes.
1505 Subclasses of source-file exist for various languages.
1506 @emph{FIXME: describe these.}
1509 @deffn Component module
1511 A module is a collection of sub-components.
1513 A module component has the following extra initargs:
1517 @code{:components} the components contained in this module
1520 @code{:default-component-class}
1521 All children components which don't specify their class explicitly
1522 are inferred to be of this type.
1525 @code{:if-component-dep-fails}
1526 This attribute takes one of the values
1527 @code{:fail}, @code{:try-next}, @code{:ignore},
1528 its default value is @code{:fail}.
1529 The other values can be used for implementing conditional compilation
1530 based on implementation @code{*features*},
1531 for the case where it is not necessary for all files in a module to be
1533 @emph{FIXME: such conditional compilation has been reported
1534 to be broken in 2009.}
1537 @code{:serial} When this attribute is set,
1538 each subcomponent of this component is assumed to depend on all subcomponents
1539 before it in the list given to @code{:components}, i.e.
1540 all of them are loaded before a compile or load operation is performed on it.
1544 The default operation knows how to traverse a module, so
1545 most operations will not need to provide methods specialised on modules.
1547 @code{module} may be subclassed to represent components such as
1548 foreign-language linked libraries or archive files.
1551 @deffn Component system
1553 @code{system} is a subclass of @code{module}.
1555 A system is a module with a few extra attributes for documentation
1556 purposes; these are given elsewhere.
1557 @xref{The defsystem grammar}.
1559 Users can create new classes for their systems:
1560 the default @code{defsystem} macro takes a @code{:class} keyword argument.
1563 @node Creating new component types, , Pre-defined subclasses of component, Components
1564 @comment node-name, next, previous, up
1565 @subsection Creating new component types
1567 New component types are defined by subclassing one of the existing
1568 component classes and specializing methods on the new component class.
1570 @emph{FIXME: this should perhaps be explained more throughly,
1571 not only by example ...}
1573 As an example, suppose we have some implementation-dependent
1574 functionality that we want to isolate
1575 in one subdirectory per Lisp implementation our system supports.
1576 We create a subclass of
1577 @code{cl-source-file}:
1580 (defclass unportable-cl-source-file (cl-source-file)
1584 A hypothetical function @code{system-dependent-dirname}
1585 gives us the name of the subdirectory.
1586 All that's left is to define how to calculate the pathname
1587 of an @code{unportable-cl-source-file}.
1590 (defmethod component-pathname ((component unportable-cl-source-file))
1591 (let ((pathname (call-next-method))
1592 (name (string-downcase (system-dependent-dirname))))
1594 (make-pathname :directory (list :relative name))
1598 The new component type is used in a @code{defsystem} form in this way:
1605 (:unportable-cl-source-file "threads"
1606 :depends-on ("packages" ...))
1611 @node Controlling where ASDF searches for systems, Controlling where ASDF saves compiled files, The object model of ASDF, Top
1612 @comment node-name, next, previous, up
1613 @chapter Controlling where ASDF searches for systems
1615 @section Configurations
1617 Configurations specify paths where to find system files.
1622 The search registry may use some hardcoded wrapping registry specification.
1623 This allows some implementations (notably SBCL) to specify where to find
1624 some special implementation-provided systems that
1625 need to precisely match the version of the implementation itself.
1628 An application may explicitly initialize the source-registry configuration
1629 using the configuration API
1630 (@pxref{Controlling where ASDF searches for systems,Configuration API,Configuration API}, below)
1631 in which case this takes precedence.
1632 It may itself compute this configuration from the command-line,
1633 from a script, from its own configuration file, etc.
1636 The source registry will be configured from
1637 the environment variable @code{CL_SOURCE_REGISTRY} if it exists.
1640 The source registry will be configured from
1641 user configuration file
1642 @file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf}
1644 @file{~/.config/common-lisp/source-registry.conf})
1648 The source registry will be configured from
1649 user configuration directory
1650 @file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf.d/}
1652 @file{~/.config/common-lisp/source-registry.conf.d/})
1656 The source registry will be configured from
1657 system configuration file
1658 @file{/etc/common-lisp/source-registry.conf}
1662 The source registry will be configured from
1663 system configuration directory
1664 @file{/etc/common-lisp/source-registry.conf.d/}
1668 The source registry will be configured from a default configuration.
1669 This configuration may allow for implementation-specific systems
1670 to be found, for systems to be found the current directory
1671 (at the time that the configuration is initialized) as well as
1672 @code{:directory} entries for @file{$XDG_DATA_DIRS/common-lisp/systems/} and
1673 @code{:tree} entries for @file{$XDG_DATA_DIRS/common-lisp/source/}.
1677 Each of these configuration is specified as a SEXP
1678 in a trival domain-specific language (defined below).
1679 Additionally, a more shell-friendly syntax is available
1680 for the environment variable (defined yet below).
1682 Each of these configurations is only used if the previous
1683 configuration explicitly or implicitly specifies that it
1684 includes its inherited configuration.
1686 Additionally, some implementation-specific directories
1687 may be automatically prepended to whatever directories are specified
1688 in configuration files, no matter if the last one inherits or not.
1690 @section XDG base directory
1692 Note that we purport to respect the XDG base directory specification
1693 as to where configuration files are located,
1694 where data files are located,
1695 where output file caches are located.
1696 Mentions of XDG variables refer to that document.
1698 @url{http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html}
1700 This specification allows the user to specify some environment variables
1701 to customize how applications behave to his preferences.
1703 On Windows platforms, when not using Cygwin,
1704 instead of the XDG base directory specification,
1705 we try to use folder configuration from the registry regarding
1706 @code{Common AppData} and similar directories.
1707 However, support querying the Windows registry is limited as of ASDF 2,
1708 and on many implementations, we may fall back to always using the defaults
1709 without consulting the registry.
1712 @section Backward Compatibility
1714 For backward compatibility as well as for a practical backdoor for hackers,
1715 ASDF will first search for @code{.asd} files in the directories specified in
1716 @code{asdf:*central-registry*}
1717 before it searches in the source registry above.
1719 @xref{Configuring ASDF,,Configuring ASDF to find your systems -- old style}.
1721 By default, @code{asdf:*central-registry*} will be empty.
1723 This old mechanism will therefore not affect you if you don't use it,
1724 but will take precedence over the new mechanism if you do use it.
1726 @section Configuration DSL
1728 Here is the grammar of the SEXP DSL for source-registry configuration:
1731 ;; A configuration is single SEXP starting with keyword :source-registry
1732 ;; followed by a list of directives.
1733 CONFIGURATION := (:source-registry DIRECTIVE ...)
1735 ;; A directive is one of the following:
1737 ;; INHERITANCE DIRECTIVE:
1738 ;; Your configuration expression MUST contain
1739 ;; exactly one of either of these:
1740 :inherit-configuration | ; splices inherited configuration (often specified last)
1741 :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
1743 ;; add a single directory to be scanned (no recursion)
1744 (:directory DIRECTORY-PATHNAME-DESIGNATOR) |
1746 ;; add a directory hierarchy, recursing but excluding specified patterns
1747 (:tree DIRECTORY-PATHNAME-DESIGNATOR) |
1749 ;; override the defaults for exclusion patterns
1750 (:exclude PATTERN ...) |
1751 ;; augment the defaults for exclusion patterns
1752 (:also-exclude PATTERN ...) |
1754 ;; splice the parsed contents of another config file
1755 (:include REGULAR-FILE-PATHNAME-DESIGNATOR) |
1757 ;; This directive specifies that some default must be spliced.
1760 PATTERN := a string without wildcards, that will be matched exactly
1761 against the name of a any subdirectory in the directory component
1762 of a path. e.g. @code{"_darcs"} will match @file{#p"/foo/bar/_darcs/src/bar.asd"}
1765 For instance, as a simple case, my @file{~/.config/common-lisp/source-registry.conf},
1766 which is the default place ASDF looks for this configuration,
1770 (:tree "/home/fare/cl/")
1771 :inherit-configuration)
1775 @section Configuration Directories
1777 Configuration directories consist in files each contains
1778 a list of directives without any enclosing @code{(:source-registry ...)} form.
1779 The files will be sorted by namestring as if by @code{string<} and
1780 the lists of directives of these files with be concatenated in order.
1781 An implicit @code{:inherit-configuration} will be included
1782 at the @emph{end} of the list.
1784 This allows for packaging software that has file granularity
1785 (e.g. Debian's @code{dpkg} or some future version of @code{clbuild})
1786 to easily include configuration information about distributed software.
1788 The convention is that, for sorting purposes,
1789 the names of files in such a directory begin with two digits
1790 that determine the order in which these entries will be read.
1791 Also, the type of these files is conventionally @code{"conf"}
1792 and as a limitation to some implementations (e.g. GNU clisp),
1793 the type cannot be @code{NIL}.
1795 Directories may be included by specifying a directory pathname
1796 or namestring in an @code{:include} directive, e.g.:
1799 (:include "/foo/bar/")
1802 Hence, to achieve the same effect as
1803 my example @file{~/.config/common-lisp/source-registry.conf} above,
1804 I could simply create a file
1805 @file{~/.config/common-lisp/source-registry.conf.d/33-home-fare-cl.conf}
1806 alone in its directory with the following contents:
1808 (:tree "/home/fare/cl/")
1812 @section Shell-friendly syntax for configuration
1814 When considering environment variable @code{CL_SOURCE_REGISTRY}
1815 ASDF will skip to next configuration if it's an empty string.
1816 It will @code{READ} the string as a SEXP in the DSL
1817 if it begins with a paren @code{(}
1818 and it will be interpreted much like @code{TEXINPUTS}
1819 list of paths, where
1821 * paths are separated
1822 by a @code{:} (colon) on Unix platforms (including cygwin),
1823 by a @code{;} (semicolon) on other platforms (mainly, Windows).
1825 * each entry is a directory to add to the search path.
1827 * if the entry ends with a double slash @code{//}
1828 then it instead indicates a tree in the subdirectories
1829 of which to recurse.
1831 * if the entry is the empty string (which may only appear once),
1832 then it indicates that the inherited configuration should be
1836 @section Search Algorithm
1838 In case that isn't clear, the semantics of the configuration is that
1839 when searching for a system of a given name,
1840 directives are processed in order.
1842 When looking in a directory, if the system is found, the search succeeds,
1843 otherwise it continues.
1845 When looking in a tree, if one system is found, the search succeeds.
1846 If multiple systems are found, the consequences are unspecified:
1847 the search may succeed with any of the found systems,
1848 or an error may be raised.
1849 ASDF currently returns the first system found,
1850 XCVB currently raised an error.
1851 If none is found, the search continues.
1853 Exclude statements specify patterns of subdirectories
1854 the systems from which to ignore.
1855 Typically you don't want to use copies of files kept by such
1856 version control systems as Darcs.
1857 Exclude statements are not propagated to further included or inherited
1858 configuration files or expressions;
1859 instead the defaults are reset around every configuration statement
1860 to the default defaults from @code{asdf::*default-source-registry-exclusions*}.
1862 Include statements cause the search to recurse with the path specifications
1863 from the file specified.
1865 An inherit-configuration statement cause the search to recurse with the path
1866 specifications from the next configuration
1867 (@pxref{Controlling where ASDF searches for systems,,Configurations} above).
1870 @section Caching Results
1872 The implementation is allowed to either eagerly compute the information
1873 from the configurations and file system, or to lazily re-compute it
1874 every time, or to cache any part of it as it goes.
1875 To explicitly flush any information cached by the system, use the API below.
1878 @section Configuration API
1880 The specified functions are exported from your build system's package.
1881 Thus for ASDF the corresponding functions are in package ASDF,
1882 and for XCVB the corresponding functions are in package XCVB.
1884 @defun initialize-source-registry @&optional PARAMETER
1885 will read the configuration and initialize all internal variables.
1886 You may extend or override configuration
1887 from the environment and configuration files
1888 with the given @var{PARAMETER}, which can be
1889 @code{NIL} (no configuration override),
1890 or a SEXP (in the SEXP DSL),
1891 a string (as in the string DSL),
1892 a pathname (of a file or directory with configuration),
1893 or a symbol (fbound to function that when called returns one of the above).
1896 @defun clear-source-registry
1897 undoes any source registry configuration
1898 and clears any cache for the search algorithm.
1899 You might want to call that before you
1900 dump an image that would be resumed with a different configuration,
1901 and return an empty configuration.
1902 Note that this does not include clearing information about
1903 systems defined in the current image, only about
1904 where to look for systems not yet defined.
1907 @defun ensure-source-registry @&optional PARAMETER
1908 checks whether a source registry has been initialized.
1909 If not, initialize it with the given @var{PARAMETER}.
1915 If this mechanism is successful, in the future, we may declare
1916 @code{asdf:*central-registry*} obsolete and eventually remove it.
1917 Any hook into implementation-specific search mechanisms will by then
1918 have been integrated in the @code{:default-configuration} which everyone
1919 should either explicitly use or implicit inherit. Some shell syntax
1920 for it should probably be added somehow.
1922 But we're not there yet. For now, let's see how practical this new
1926 @section Rejected ideas
1928 Alternatives I considered and rejected included:
1931 @item Keep @code{asdf:*central-registry*} as the master with its current semantics,
1932 and somehow the configuration parser expands the new configuration
1933 language into a expanded series of directories of subdirectories to
1934 lookup, pre-recursing through specified hierarchies. This is kludgy,
1935 and leaves little space of future cleanups and extensions.
1937 @item Keep @code{asdf:*central-registry*} remains the master but extend its semantics
1938 in completely new ways, so that new kinds of entries may be implemented
1939 as a recursive search, etc. This seems somewhat backwards.
1941 @item Completely remove @code{asdf:*central-registry*}
1942 and break backwards compatibility.
1943 Hopefully this will happen in a few years after everyone migrate to
1944 a better ASDF and/or to XCVB, but it would be very bad to do it now.
1946 @item Replace @code{asdf:*central-registry*} by a symbol-macro with appropriate magic
1947 when you dereference it or setf it. Only the new variable with new
1948 semantics is handled by the new search procedure.
1949 Complex and still introduces subtle semantic issues.
1953 I've been suggested the below features, but have rejected them,
1954 for the sake of keeping ASDF no more complex than strictly necessary.
1958 More syntactic sugar: synonyms for the configuration directives, such as
1959 @code{(:add-directory X)} for @code{(:directory X)}, or @code{(:add-directory-hierarchy X)}
1960 or @code{(:add-directory X :recurse t)} for @code{(:tree X)}.
1963 The possibility to register individual files instead of directories.
1966 Integrate Xach Beane's tilde expander into the parser,
1967 or something similar that is shell-friendly or shell-compatible.
1968 I'd rather keep ASDF minimal. But maybe this precisely keeps it
1969 minimal by removing the need for evaluated entries that ASDF has?
1970 i.e. uses of @code{USER-HOMEDIR-PATHNAME} and @code{$SBCL_HOME}
1971 Hopefully, these are already superseded by the @code{:default-registry}
1974 Using the shell-unfriendly syntax @code{/**} instead of @code{//} to specify recursion
1975 down a filesystem tree in the environment variable.
1976 It isn't that Lisp friendly either.
1986 @section Credits for the source-registry
1988 Thanks a lot to Stelian Ionescu for the initial idea.
1990 Thanks to Rommel Martinez for the initial implementation attempt.
1992 All bad design ideas and implementation bugs are to mine, not theirs.
1993 But so are good design ideas and elegant implementation tricks.
1995 --- Francois-Rene Rideau @email{fare@@tunes.org}, Mon, 22 Feb 2010 00:07:33 -0500
1999 @node Controlling where ASDF saves compiled files, Error handling, Controlling where ASDF searches for systems, Top
2000 @comment node-name, next, previous, up
2001 @chapter Controlling where ASDF saves compiled files
2002 @cindex asdf-output-translations
2003 @vindex ASDF_OUTPUT_TRANSLATIONS
2005 Each Common Lisp implementation has its own format
2006 for compiled files (fasls for short, short for ``fast loading'').
2007 If you use multiple implementations
2008 (or multiple versions of the same implementation),
2009 you'll soon find your source directories
2010 littered with various @file{fasl}s, @file{dfsl}s, @file{cfsl}s and so on.
2011 Worse yet, some implementations use the same file extension
2012 while changing formats from version to version (or platform to platform)
2013 which means that you'll have to recompile binaries
2014 as you switch from one implementation to the next.
2016 ASDF 2 includes the @code{asdf-output-translations} facility
2017 to mitigate the problem.
2019 @section Configurations
2021 Configurations specify mappings from input locations to output locations.
2022 Once again we rely on the XDG base directory specification for configuration.
2023 @xref{Controlling where ASDF searches for systems,,XDG base directory}.
2028 Some hardcoded wrapping output translations configuration may be used.
2029 This allows special output translations (or usually, invariant directories)
2030 to be specified corresponding to the similar special entries in the source registry.
2033 An application may explicitly initialize the output-translations
2034 configuration using the Configuration API
2035 in which case this takes precedence.
2036 (@pxref{Controlling where ASDF saves compiled files,,Configuration API}.)
2037 It may itself compute this configuration from the command-line,
2038 from a script, from its own configuration file, etc.
2041 The source registry will be configured from
2042 the environment variable @code{ASDF_OUTPUT_TRANSLATIONS} if it exists.
2045 The source registry will be configured from
2046 user configuration file
2047 @file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf}
2049 @file{~/.config/common-lisp/asdf-output-translations.conf})
2053 The source registry will be configured from
2054 user configuration directory
2055 @file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf.d/}
2057 @file{~/.config/common-lisp/asdf-output-translations.conf.d/})
2061 The source registry will be configured from
2062 system configuration file
2063 @file{/etc/common-lisp/asdf-output-translations.conf}
2067 The source registry will be configured from
2068 system configuration directory
2069 @file{/etc/common-lisp/asdf-output-translations.conf.d/}
2074 Each of these configurations is specified as a SEXP
2075 in a trival domain-specific language (defined below).
2076 Additionally, a more shell-friendly syntax is available
2077 for the environment variable (defined yet below).
2079 Each of these configurations is only used if the previous
2080 configuration explicitly or implicitly specifies that it
2081 includes its inherited configuration.
2083 Note that by default, a per-user cache is used for output files.
2084 This allows the seamless use of shared installations of software
2085 between several users, and takes files out of the way of the developers
2086 when they browse source code,
2087 at the expense of taking a small toll when developers have to clean up
2088 output files and find they need to get familiar with output-translations first.
2091 @section Backward Compatibility
2093 @c FIXME -- I think we should provide an easy way
2094 @c to get behavior equivalent to A-B-L and
2095 @c I will propose a technique for doing this.
2097 We purposefully do NOT provide backward compatibility with earlier versions of
2098 @code{ASDF-Binary-Locations} (8 Sept 2009),
2099 @code{common-lisp-controller} (7.0) or
2100 @code{cl-launch} (2.35),
2101 each of which had similar general capabilities.
2102 The previous APIs of these programs were not designed
2103 for configuration by the end-user
2104 in an easy way with configuration files.
2105 Recent versions of same packages use
2106 the new @code{asdf-output-translations} API as defined below:
2107 @code{common-lisp-controller} (7.2) and @code{cl-launch} (3.000).
2108 @code{ASDF-Binary-Locations} is fully superseded and not to be used anymore.
2110 This incompatibility shouldn't inconvenience many people.
2111 Indeed, few people use and customize these packages;
2112 these few people are experts who can trivially adapt to the new configuration.
2113 Most people are not experts, could not properly configure these features
2114 (except inasmuch as the default configuration of
2115 @code{common-lisp-controller} and/or @code{cl-launch}
2116 might have been doing the right thing for some users),
2117 and yet will experience software that ``just works'',
2118 as configured by the system distributor, or by default.
2120 Nevertheless, if you are a fan of @code{ASDF-Binary-Locations},
2121 we provide a limited emulation mode:
2123 @defun asdf:enable-asdf-binary-locations-compatibility @&key centralize-lisp-binaries default-toplevel-directory include-per-user-information map-all-source-files source-to-target-mappings
2124 This function will initialize the new @code{asdf-output-translations} facility in a way
2125 that emulates the behavior of the old @code{ASDF-Binary-Locations} facility.
2126 Where you would previously set global variables
2127 @var{*centralize-lisp-binaries*},
2128 @var{*default-toplevel-directory*},
2129 @var{*include-per-user-information*},
2130 @var{*map-all-source-files*} or @var{*source-to-target-mappings*}
2131 you will now have to pass the same values as keyword arguments to this function.
2132 Note however that as an extension the @code{:source-to-target-mappings} keyword argument
2133 will accept any valid pathname designator for @code{asdf-output-translations}
2134 instead of just strings and pathnames.
2137 If you insist, you can also keep using the old @code{ASDF-Binary-Locations}
2138 (the one available as an extension to load of top of ASDF,
2139 not the one built into a few old versions of ASDF),
2140 but first you must disable @code{asdf-output-translations}
2141 with @code{(asdf:disable-output-translations)},
2142 or you might experience ``interesting'' issues.
2144 Also, note that output translation is enabled by default.
2145 To disable it, use @code{(asdf:disable-output-translations)}.
2148 @section Configuration DSL
2150 Here is the grammar of the SEXP DSL
2151 for @code{asdf-output-translations} configuration:
2154 ;; A configuration is single SEXP starting with keyword :source-registry
2155 ;; followed by a list of directives.
2156 CONFIGURATION := (:output-translations DIRECTIVE ...)
2158 ;; A directive is one of the following:
2160 ;; INHERITANCE DIRECTIVE:
2161 ;; Your configuration expression MUST contain
2162 ;; exactly one of either of these:
2163 :inherit-configuration | ; splices inherited configuration (often specified last)
2164 :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
2166 ;; include a configuration file or directory
2167 (:include PATHNAME-DESIGNATOR) |
2169 ;; enable global cache in ~/.common-lisp/cache/sbcl-1.0.35-x86-64/ or something.
2170 :enable-user-cache |
2171 ;; Disable global cache. Map / to /
2174 ;; add a single directory to be scanned (no recursion)
2175 (DIRECTORY-DESIGNATOR DIRECTORY-DESIGNATOR)
2177 ;; use a function to return the translation of a directory designator
2178 (DIRECTORY-DESIGNATOR (:function TRANSLATION-FUNCTION))
2180 DIRECTORY-DESIGNATOR :=
2181 T | ;; as source matches anything, as destination leaves pathname unmapped.
2182 ABSOLUTE-COMPONENT-DESIGNATOR |
2183 (ABSOLUTE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...)
2185 ABSOLUTE-COMPONENT-DESIGNATOR :=
2186 NULL | ;; As source: skip this entry. As destination: same as source
2187 :ROOT | ;; magic: paths that are relative to the root of the source host and device
2188 STRING | ;; namestring (directory is assumed, better be absolute or bust, ``/**/*.*'' added)
2189 PATHNAME | ;; pathname (better be an absolute directory or bust)
2190 :HOME | ;; designates the user-homedir-pathname ~/
2191 :USER-CACHE | ;; designates the default location for the user cache
2192 :SYSTEM-CACHE | ;; designates the default location for the system cache
2193 :CURRENT-DIRECTORY ;; the current directory
2195 RELATIVE-COMPONENT-DESIGNATOR :=
2196 STRING | ;; namestring, directory is assumed. If the last component, /**/*.* is added
2197 PATHNAME | ;; pathname unless last component, directory is assumed.
2198 :IMPLEMENTATION | ;; a directory based on implementation, e.g. sbcl-1.0.32.30-linux-x86-64
2199 :IMPLEMENTATION-TYPE | ;; a directory based on lisp-implementation-type only, e.g. sbcl
2200 :CURRENT-DIRECTORY | ;; all components of the current directory, without the :absolute
2201 :UID | ;; current UID -- not available on Windows
2202 :USER ;; current USER name -- NOT IMPLEMENTED(!)
2204 TRANSLATION-FUNCTION :=
2205 SYMBOL | ;; symbol of a function that takes two arguments,
2206 ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2207 LAMBDA ;; A form which evalutates to a function taking two arguments consisting of
2208 ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2212 Relative components better be either relative
2213 or subdirectories of the path before them, or bust.
2215 The last component, if not a pathname, is notionally completed by @file{/**/*.*}.
2216 You can specify more fine-grained patterns
2217 by using a pathname object as the last component
2218 e.g. @file{#p"some/path/**/foo*/bar-*.fasl"}
2220 You may use @code{#+features} to customize the configuration file.
2222 The second designator of a mapping may be @code{NIL}, indicating that files are not mapped
2223 to anything but themselves (same as if the second designator was the same as the first).
2225 When the first designator is @code{t},
2226 the mapping always matches.
2227 When the first designator starts with @code{:root},
2228 the mapping matches any host and device.
2229 In either of these cases, if the second designator
2230 isn't @code{t} and doesn't start with @code{:root},
2231 then strings indicating the host and pathname are somehow copied
2232 in the beginning of the directory component of the source pathname
2233 before it is translated.
2235 When the second designator is @code{t}, the mapping is the identity.
2236 When the second designator starts with @code{root},
2237 the mapping preserves the host and device of the original pathname.
2239 @code{:include} statements cause the search to recurse with the path specifications
2240 from the file specified.
2242 If the @code{translate-pathname} mechanism cannot achieve a desired
2243 translation, the user may provide a function which provides the
2244 required algorithim. Such a translation function is specified by
2245 supplying a list as the second @code{directory-designator}
2246 the first element of which is the keyword @code{:function},
2247 and the second element of which is
2248 either a symbol which designates a function or a lambda expression.
2249 The function designated by the second argument must take two arguments,
2250 the first being the pathname of the source file,
2251 the second being the wildcard that was matched.
2252 The result of the function invocation should be the translated pathname.
2254 An @code{:inherit-configuration} statement cause the search to recurse with the path
2255 specifications from the next configuration.
2256 @xref{Controlling where ASDF saves compiled files,,Configurations}, above.
2260 @code{:enable-user-cache} is the same as @code{(t :user-cache)}.
2262 @code{:disable-cache} is the same as @code{(t t)}.
2264 @code{:user-cache} uses the contents of variable @code{asdf::*user-cache*}
2265 which by default is the same as using
2266 @code{(:home ".cache" "common-lisp" :implementation)}.
2268 @code{:system-cache} uses the contents of variable @code{asdf::*system-cache*}
2269 which by default is the same as using
2270 @code{("/var/cache/common-lisp" :uid :implementation-type)}
2271 (on Unix and cygwin), or something semi-sensible on Windows.
2275 @section Configuration Directories
2277 Configuration directories consist in files each contains
2278 a list of directives without any enclosing
2279 @code{(:output-translations ...)} form.
2280 The files will be sorted by namestring as if by @code{string<} and
2281 the lists of directives of these files with be concatenated in order.
2282 An implicit @code{:inherit-configuration} will be included
2283 at the @emph{end} of the list.
2285 This allows for packaging software that has file granularity
2286 (e.g. Debian's @command{dpkg} or some future version of @command{clbuild})
2287 to easily include configuration information about software being distributed.
2289 The convention is that, for sorting purposes,
2290 the names of files in such a directory begin with two digits
2291 that determine the order in which these entries will be read.
2292 Also, the type of these files is conventionally @code{"conf"}
2293 and as a limitation of some implementations, the type cannot be @code{NIL}.
2295 Directories may be included by specifying a directory pathname
2296 or namestring in an @code{:include} directive, e.g.:
2298 (:include "/foo/bar/")
2301 @section Shell-friendly syntax for configuration
2303 When considering environment variable @code{ASDF_OUTPUT_TRANSLATIONS}
2304 ASDF will skip to next configuration if it's an empty string.
2305 It will @code{READ} the string as an SEXP in the DSL
2306 if it begins with a paren @code{(}
2307 and it will be interpreted as a list of directories.
2308 Directories should come by pairs, indicating a mapping directive.
2309 Entries are separated
2310 by a @code{:} (colon) on Unix platforms (including cygwin),
2311 by a @code{;} (semicolon) on other platforms (mainly, Windows).
2313 The magic empty entry,
2314 if it comes in what would otherwise be the first entry in a pair,
2315 indicates the splicing of inherited configuration.
2316 If it comes as the second entry in a pair,
2317 it indicates that the directory specified first is to be left untranslated
2318 (which has the same effect as if the directory had been repeated).
2321 @section Semantics of Output Translations
2323 From the specified configuration,
2324 a list of mappings is extracted in a straightforward way:
2325 mappings are collected in order, recursing through
2326 included or inherited configuration as specified.
2327 To this list is prepended some implementation-specific mappings,
2328 and is appended a global default.
2330 The list is then compiled to a mapping table as follows:
2331 for each entry, in order, resolve the first designated directory
2332 into an actual directory pathname for source locations.
2333 If no mapping was specified yet for that location,
2334 resolve the second designated directory to an output location directory
2335 add a mapping to the table mapping the source location to the output location,
2336 and add another mapping from the output location to itself
2337 (unless a mapping already exists for the output location).
2339 Based on the table, a mapping function is defined,
2340 mapping source pathnames to output pathnames:
2341 given a source pathname, locate the longest matching prefix
2342 in the source column of the mapping table.
2343 Replace that prefix by the corresponding output column
2344 in the same row of the table, and return the result.
2345 If no match is found, return the source pathname.
2346 (A global default mapping the filesystem root to itself
2347 may ensure that there will always be a match,
2348 with same fall-through semantics).
2350 @section Caching Results
2352 The implementation is allowed to either eagerly compute the information
2353 from the configurations and file system, or to lazily re-compute it
2354 every time, or to cache any part of it as it goes.
2355 To explicitly flush any information cached by the system, use the API below.
2358 @section Output location API
2360 The specified functions are exported from package ASDF.
2362 @defun initialize-output-translations @&optional PARAMETER
2363 will read the configuration and initialize all internal variables.
2364 You may extend or override configuration
2365 from the environment and configuration files
2366 with the given @var{PARAMETER}, which can be
2367 @code{NIL} (no configuration override),
2368 or a SEXP (in the SEXP DSL),
2369 a string (as in the string DSL),
2370 a pathname (of a file or directory with configuration),
2371 or a symbol (fbound to function that when called returns one of the above).
2374 @defun disable-output-translations
2375 will initialize output translations in a way
2376 that maps every pathname to itself,
2377 effectively disabling the output translation facility.
2380 @defun clear-output-translations
2381 undoes any output translation configuration
2382 and clears any cache for the mapping algorithm.
2383 You might want to call that before you
2384 dump an image that would be resumed with a different configuration,
2385 and return an empty configuration.
2386 Note that this does not include clearing information about
2387 systems defined in the current image, only about
2388 where to look for systems not yet defined.
2391 @defun ensure-output-translations @&optional PARAMETER
2392 checks whether output translations have been initialized.
2393 If not, initialize them with the given @var{PARAMETER}.
2394 This function will be called before any attempt to operate on a system.
2397 @defun apply-output-translations PATHNAME
2398 Applies the configured output location translations to @var{PATHNAME}
2399 (calls @code{ensure-output-translations} for the translations).
2403 @section Credits for output translations
2405 Thanks a lot to Bjorn Lindberg and Gary King for @code{ASDF-Binary-Locations},
2406 and to Peter van Eynde for @code{Common Lisp Controller}.
2408 All bad design ideas and implementation bugs are to mine, not theirs.
2409 But so are good design ideas and elegant implementation tricks.
2411 --- Francois-Rene Rideau @email{fare@@tunes.org}
2413 @c @section Default locations
2414 @c @findex output-files-for-system-and-operation
2416 @c The default binary location for each Lisp implementation
2417 @c is a subdirectory of each source directory.
2418 @c To account for different Lisps, Operating Systems, Implementation versions,
2419 @c and so on, ASDF borrows code from SLIME
2420 @c to create reasonable custom directory names.
2421 @c Here are some examples:
2425 @c SBCL, version 1.0 on Mac OS X for intel: @code{sbcl-1.0-darwin-x86}
2428 @c Franz Allegro, version 8.0, ANSI Common Lisp: @code{allegro-8.0a-macosx-x86}
2431 @c Franz Allegro, version 8.1, Modern (case sensitive) Common Lisp: @code{allegro-8.1m-macosx-x86}
2434 @c By default, all output file pathnames will be relocated
2435 @c to some thus-named subdirectory of @file{~/.cache/common-lisp/}.
2437 @c See the document @file{README.asdf-output-translations}
2438 @c for a full specification on how to configure @code{asdf-output-translations}.
2440 @node Error handling, Miscellaneous additional functionality, Controlling where ASDF saves compiled files, Top
2441 @comment node-name, next, previous, up
2442 @chapter Error handling
2443 @findex SYSTEM-DEFINITION-ERROR
2444 @findex OPERATION-ERROR
2446 @section ASDF errors
2448 If ASDF detects an incorrect system definition, it will signal a generalised instance of
2449 @code{SYSTEM-DEFINITION-ERROR}.
2451 Operations may go wrong (for example when source files contain errors).
2452 These are signalled using generalised instances of
2453 @code{OPERATION-ERROR}.
2455 @section Compilation error and warning handling
2456 @vindex *compile-file-warnings-behaviour*
2457 @vindex *compile-file-errors-behavior*
2459 ASDF checks for warnings and errors when a file is compiled.
2460 The variables @var{*compile-file-warnings-behaviour*} and
2461 @var{*compile-file-errors-behavior*}
2462 control the handling of any such events.
2463 The valid values for these variables are
2464 @code{:error}, @code{:warn}, and @code{:ignore}.
2466 @node Miscellaneous additional functionality, Getting the latest version, Error handling, Top
2467 @comment node-name, next, previous, up
2468 @chapter Miscellaneous additional functionality
2470 @emph{FIXME: Add discussion of @code{run-shell-command}? Others?}
2472 ASDF includes several additional features that are generally
2473 useful for system definition and development. These include:
2475 @defun system-relative-pathname system name @&key type
2477 It's often handy to locate a file relative to some system.
2478 The @code{system-relative-pathname} function meets this need.
2479 It takes two arguments: the name of a system and a relative pathname.
2480 It returns a pathname built from the location of the system's source file
2481 and the relative pathname. For example
2484 > (asdf:system-relative-pathname 'cl-ppcre #p"regex.data")
2485 #P"/repository/other/cl-ppcre/regex.data"
2488 Instead of a pathname, you can provide a symbol or a string,
2489 and optionally a keyword argument @code{type}.
2490 The arguments will then be interpreted in the same way
2491 as pathname specifiers for components.
2492 @xref{The defsystem grammar,,Pathname specifiers}.
2495 @defun system-source-directory system-designator
2497 ASDF does not provide a turnkey solution for locating
2498 data (or other miscellaneous) files
2499 that are distributed together with the source code of a system.
2500 Programmers can use @code{system-source-directory} to find such files.
2501 Returns a pathname object.
2502 The @var{system-designator} may be a string, symbol, or ASDF system object.
2505 @defun clear-system system-designator
2507 It is sometimes useful to force recompilation of a previously loaded system.
2508 In these cases, it may be useful to @code{(asdf:clear-system :foo)}
2509 to remove the system from the table of currently loaded systems;
2510 the next time the system @code{foo} or one that depends on it is re-loaded,
2511 @code{foo} will then be loaded again.
2512 Alternatively, you could touch @code{foo.asd} or
2513 remove the corresponding fasls from the output file cache.
2514 (It was once conceived that one should provide
2515 a list of systems the recompilation of which to force
2516 as the @code{:force} keyword argument to @code{load-system};
2517 but this has never worked, and though the feature was fixed in ASDF 2.000,
2518 it remains @code{cerror}'ed out as nobody ever used it.)
2520 Note that this does not and cannot by itself undo the previous loading
2521 of the system. Common Lisp has no provision for such an operation,
2522 and its reliance on irreversible side-effects to global datastructures
2523 makes such a thing impossible in the general case.
2524 If the software being re-loaded is not conceived with hot upgrade in mind,
2525 this re-loading may cause many errors, warnings or subtle silent problems,
2526 as packages, generic function signatures, structures, types, macros, constants, etc.
2527 are being redefined incompatibly.
2528 It is up to the user to make sure that reloading is possible and has the desired effect.
2529 In some cases, extreme measures such as recursively deleting packages,
2530 unregistering symbols, defining methods on @code{update-instance-for-redefined-class}
2531 and much more are necessary for reloading to happen smoothly.
2532 ASDF itself goes through notable pains to make such a hot upgrade possible
2533 with respect to its own code, and what it does is ridiculously complex;
2534 look at the beginning of @file{asdf.lisp} to see what it does.
2537 @node Getting the latest version, FAQ, Miscellaneous additional functionality, Top
2538 @comment node-name, next, previous, up
2539 @chapter Getting the latest version
2541 Decide which version you want.
2542 HEAD is the newest version and usually OK, whereas
2543 RELEASE is for cautious people
2544 (e.g. who already have systems using ASDF that they don't want broken),
2545 a slightly older version about which none of the HEAD users have complained.
2546 There is also a STABLE version, which is earlier than release.
2548 You may get the ASDF source repository using git:
2549 @kbd{git clone git://common-lisp.net/projects/asdf/asdf.git}
2551 You will find the above referenced tags in this repository.
2552 You can also browse the repository on
2553 @url{http://common-lisp.net/gitweb?p=projects/asdf/asdf.git}.
2555 Discussion of ASDF development is conducted on the
2557 @kbd{asdf-devel@@common-lisp.net}.
2558 @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel}
2561 @node FAQ, TODO list, Getting the latest version, Top
2562 @comment node-name, next, previous, up
2565 @section ``Where do I report a bug?''
2567 ASDF bugs are tracked on launchpad: @url{https://launchpad.net/asdf}.
2569 If you're unsure about whether something is a bug, of for general discussion,
2570 use the @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
2573 @section ``What has changed between ASDF 1 and ASDF 2?''
2575 @subsection What are ASDF 1 and ASDF 2?
2577 On May 31st 2010, we have released ASDF 2.
2578 ASDF 2 refers to release 2.000 and later.
2579 (Releases between 1.656 and 1.728 were development releases for ASDF 2.)
2580 ASDF 1 to any release earlier than 1.369 or so.
2581 If your ASDF doesn't sport a version, it's an old ASDF 1.
2583 ASDF 2 and its release candidates push
2584 @code{:asdf2} onto @code{*features*} so that if you are writing
2585 ASDF-dependent code you may check for this feature
2586 to see if the new API is present.
2587 @emph{All} versions of ASDF should have the @code{:asdf} feature.
2589 If you are experiencing problems or limitations of any sort with ASDF 1,
2590 we recommend that you should upgrade to ASDF 2,
2591 or whatever is the latest release.
2594 @subsection ASDF can portably name files in subdirectories
2596 Common Lisp namestrings are not portable,
2597 except maybe for logical pathnamestrings,
2598 that themselves have various limitations and require a lot of setup
2599 that is itself ultimately non-portable.
2601 In ASDF 1, the only portable ways to refer to pathnames inside systems and components
2602 were very awkward, using @code{#.(make-pathname ...)} and
2603 @code{#.(merge-pathnames ...)}.
2604 Even the above were themselves were inadequate in the general case
2605 due to host and device issues, unless horribly complex patterns were used.
2606 Plenty of simple cases that looked portable actually weren't,
2607 leading to much confusion and greavance.
2609 ASDF 2 implements its own portable syntax for strings as pathname specifiers.
2610 Naming files within a system definition becomes easy and portable again.
2611 @xref{Miscellaneous additional functionality,asdf:system-relative-pathname},
2612 @code{asdf-utilities:merge-pathnames*},
2613 @code{asdf::merge-component-name-type}.
2615 On the other hand, there are places where systems used to accept namestrings
2616 where you must now use an explicit pathname object:
2617 @code{(defsystem ... :pathname "LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
2618 must now be written with the @code{#p} syntax:
2619 @code{(defsystem ... :pathname #p"LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
2621 @xref{The defsystem grammar,,Pathname specifiers}.
2624 @subsection Output translations
2626 A popular feature added to ASDF was output pathname translation:
2627 @code{asdf-binary-locations}, @code{common-lisp-controller},
2628 @code{cl-launch} and other hacks were all implementing it in ways
2629 both mutually incompatible and difficult to configure.
2631 Output pathname translation is essential to share
2632 source directories of portable systems across multiple implementations
2633 or variants thereof,
2634 or source directories of shared installations of systems across multiple users,
2635 or combinations of the above.
2637 In ASDF 2, a standard mechanism is provided for that,
2638 @code{asdf-output-translations},
2639 with sensible defaults, adequate configuration languages,
2640 a coherent set of configuration files and hooks,
2641 and support for non-Unix platforms.
2643 @xref{Controlling where ASDF saves compiled files}.
2645 @subsection Source Registry Configuration
2647 Configuring ASDF used to require special magic
2648 to be applied just at the right moment,
2649 between the moment ASDF is loaded and the moment it is used,
2650 in a way that is specific to the user,
2651 the implementation he is using and the application he is building.
2653 This made for awkward configuration files and startup scripts
2654 that could not be shared between users, managed by administrators
2655 or packaged by distributions.
2657 ASDF 2 provides a well-documented way to configure ASDF,
2658 with sensible defaults, adequate configuration languages,
2659 and a coherent set of configuration files and hooks.
2661 We believe it's a vast improvement because it decouples
2662 application distribution from library distribution.
2663 The application writer can avoid thinking where the libraries are,
2664 and the library distributor (dpkg, clbuild, advanced user, etc.)
2665 can configure them once and for every application.
2666 Yet settings can be easily overridden where needed,
2667 so whoever needs control has exactly as much as required.
2669 At the same time, ASDF 2 remains compatible
2670 with the old magic you may have in your build scripts
2671 (using @code{*central-registry*} and
2672 @code{*system-definition-search-functions*})
2673 to tailor the ASDF configuration to your build automation needs,
2674 and also allows for new magic, simpler and more powerful magic.
2676 @xref{Controlling where ASDF searches for systems}.
2679 @subsection Usual operations are made easier to the user
2681 In ASDF 1, you had to use the awkward syntax
2682 @code{(asdf:oos 'asdf:load-op :foo)}
2684 and similarly for @code{compile-op}, @code{test-op}.
2686 In ASDF 2, you can use shortcuts for the usual operations:
2687 @code{(asdf:load-system :foo)}, and
2688 similarly for @code{compile-system}, @code{test-system}.
2691 @subsection Many bugs have been fixed
2693 The following issues and many others have been fixed:
2697 The infamous TRAVERSE function has been revamped significantly,
2698 with many bugs squashed.
2699 In particular, dependencies were not correctly propagated
2700 across submodules within a system but now are.
2701 The :version and :feature features and
2702 the :force (system1 .. systemN) feature have been fixed.
2705 Performance has been notably improved for large systems
2706 (say with thousands of components) by using
2707 hash-tables instead of linear search,
2708 and linear-time list accumulation
2709 instead of quadratic-time recursive appends.
2712 Many features used to not be portable,
2713 especially where pathnames were involved.
2714 Windows support was notably quirky because of such non-portability.
2717 The internal test suite used to massively fail on many implementations.
2718 While still incomplete, it now fully passes
2719 on all implementations supported by the test suite,
2720 except for GCL (due to GCL bugs).
2723 Support was lacking for some implementations.
2724 ABCL and GCL were notably wholly broken.
2725 ECL extensions were not integrated in the ASDF release.
2728 The documentation was grossly out of date.
2733 @subsection ASDF itself is versioned
2735 Between new features, old bugs fixed, and new bugs introduced,
2736 there were various releases of ASDF in the wild,
2737 and no simple way to check which release had which feature set.
2738 People using or writing systems had to either make worst-case assumptions
2739 as to what features were available and worked,
2740 or take great pains to have the correct version of ASDF installed.
2742 With ASDF 2, we provide a new stable set of working features
2743 that everyone can rely on from now on.
2744 Use @code{#+asdf2} to detect presence of ASDF 2,
2745 @code{(asdf:version-satisfies (asdf:asdf-version) "2.000")}
2746 to check the availability of a version no earlier than required.
2749 @subsection ASDF can be upgraded
2751 When an old version of ASDF was loaded,
2752 it was very hard to upgrade ASDF in your current image
2753 without breaking everything.
2754 Instead you have to exit the Lisp process and
2755 somehow arrange to start a new one from a simpler image.
2756 Something that can't be done from within Lisp,
2757 making automation of it difficult,
2758 which compounded with difficulty in configuration,
2759 made the task quite hard.
2760 Yet as we saw before, the task would have been required
2761 to not have to live with the worst case or non-portable
2762 subset of ASDF features.
2764 With ASDF 2, it is easy to upgrade
2765 from ASDF 2 to later versions from within Lisp,
2766 and not too hard to upgrade from ASDF 1 to ASDF 2 from within Lisp.
2767 We support hot upgrade of ASDF and any breakage is a bug
2768 that we will do our best to fix.
2769 There are still limitations on upgrade, though,
2770 most notably the fact that after you upgrade ASDF,
2771 you must also reload or upgrade all ASDF extensions.
2773 @subsection Decoupled release cycle
2775 When vendors were releasing their Lisp implementations with ASDF,
2776 they had to basically never change version
2777 because neither upgrade nor downgrade was possible
2778 without breaking something for someone,
2779 and no obvious upgrade path was visible and recommendable.
2781 With ASDF 2, upgrade is possible, easy and can be recommended.
2782 This means that vendors can safely ship a recent version of ASDF,
2783 confident that if a user isn't fully satisfied,
2784 he can easily upgrade ASDF and deal
2785 with a supported recent version of it.
2786 This means that release cycles will be causally decoupled,
2787 the practical consequence of which will mean faster convergence
2788 towards the latest version for everyone.
2791 @subsection Pitfalls of ASDF 2
2793 The main pitfalls in upgrading to ASDF 2 seem to be related
2794 to the output translation mechanism.
2799 Output translations is enabled by default. This may surprise some users,
2800 most of them in pleasant way (we hope), a few of them in an unpleasant way.
2801 It is trivial to disable output translations.
2802 @xref{FAQ,,``How can I wholly disable the compiler output cache?''}.
2805 Some systems in the large have been known not to play well with output translations.
2806 They were relatively easy to fix.
2807 Once again, it is also easy to disable output translations,
2808 or to override its configuration.
2811 The new ASDF output translations are incompatible with ASDF-Binary-Locations.
2812 They replace A-B-L, and there is compatibility mode to emulate
2813 your previous A-B-L configuration.
2814 See @code{asdf:enable-asdf-binary-locations-compatibility} in
2815 @pxref{Controlling where ASDF saves compiled files,,Backward Compatibility}.
2816 But thou shall not load ABL on top of ASDF 2.
2819 ASDF pathname designators are now specified in places where they were unspecified,
2820 and a few small adjustments have to be made to some non-portable defsystems.
2821 Notably, in the @code{:pathname} argument to a @code{defsystem} and its components,
2822 a logical pathname (or implementation-dependent hierarchical pathname)
2823 must now be specified with @code{#p} syntax
2824 where the namestring might have previously sufficed;
2825 moreover when evaluation is desired @code{#.} must be used,
2826 where it wasn't necessary in the toplevel @code{:pathname} argument.
2830 Other issues include the following:
2835 There is a slight performance bug, notably on SBCL,
2836 when initially searching for @file{asd} files,
2837 the implicit @code{(directory "/configured/path/**/*.asd")}
2838 for every configured path @code{(:tree "/configured/path/")}
2839 in your @code{source-registry} configuration can cause a slight pause.
2840 Try to @code{(time (asdf:initialize-source-registry))}
2841 to see how bad it is or isn't on your system.
2842 If you insist on not having this pause,
2843 you can avoid the pause by overriding the default source-registry configuration
2844 and not use any deep @code{:tree} entry but only @code{:directory} entries
2845 or shallow @code{:tree} entries.
2846 Or you can fix your implementation to not be quite that slow
2847 when recursing through directories.
2850 On Windows, only LispWorks supports proper default configuration pathnames
2851 based on the Windows registry.
2852 Other implementations make do.
2853 Windows support is largely untested, so please help report and fix bugs.
2858 @section Issues with installing the proper version of ASDF
2860 @subsection ``My Common Lisp implementation comes with an outdated version of ASDF. What to do?''
2862 We recommend you upgrade ASDF.
2863 @xref{Loading ASDF,,Upgrading ASDF}.
2865 If this does not work, it is a bug, and you should report it.
2866 @xref{FAQ, report-bugs, Where do I report a bug}.
2867 In the meantime, you can load @file{asdf.lisp} directly.
2868 @xref{Loading ASDF,Loading an otherwise installed ASDF}.
2871 @subsection ``I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?''
2873 Starting with current candidate releases of ASDF 2,
2874 it should always be a good time to upgrade to a recent version of ASDF.
2875 You may consult with the maintainer for which specific version they recommend,
2876 but the latest RELEASE should be correct.
2877 We trust you to thoroughly test it with your implementation before you release it.
2878 If there are any issues with the current release,
2879 it's a bug that you should report upstream and that we will fix ASAP.
2881 As to how to include ASDF, we recommend the following:
2885 If ASDF isn't installed yet, then @code{(require :asdf)}
2886 should load the version of ASDF that is bundled with your system.
2887 You may have it load some other version configured by the user,
2888 if you allow such configuration.
2891 If your system provides a mechanism to hook into @code{CL:REQUIRE},
2892 then it would be nice to add ASDF to this hook the same way that
2893 ABCL, CCL, CMUCL, ECL and SBCL do it.
2896 You may, like SBCL, have ASDF be implicitly used to require systems
2897 that are bundled with your Lisp distribution.
2898 If you do have a few magic systems that come with your implementation
2899 in a precompiled way such that one should only use the binary version
2900 that goes with your distribution, like SBCL does,
2901 then you should add them in the beginning of @code{wrapping-source-registry}.
2904 If you have magic systems as above, like SBCL does,
2905 then we explicitly ask you to @emph{NOT} distribute
2906 @file{asdf.asd} as part of those magic systems.
2907 You should still include the file @file{asdf.lisp} in your source distribution
2908 and precompile it in your binary distribution,
2909 but @file{asdf.asd} if included at all,
2910 should be secluded from the magic systems,
2911 in a separate file hierarchy,
2912 or you may otherwise rename the system and its file to e.g.
2913 @code{asdf-ecl} and @file{asdf-ecl.asd}, or
2914 @code{sb-asdf} and @file{sb-asdf.asd}.
2915 Indeed, if you made @file{asdf.asd} a magic system,
2916 then users would no longer be able to upgrade ASDF using ASDF itself
2917 to some version of their preference that
2918 they maintain independently from your Lisp distribution.
2921 If you do not have any such magic systems, or have other non-magic systems
2922 that you want to bundle with your implementation,
2923 then you may add them to the @code{default-source-registry},
2924 and you are welcome to include @file{asdf.asd} amongst them.
2927 Please send us upstream any patches you make to ASDF itself,
2928 so we can merge them back in for the benefit of your users
2929 when they upgrade to the upstream version.
2935 @section Issues with configuring ASDF
2937 @subsection ``How can I customize where fasl files are stored?''
2939 @xref{Controlling where ASDF saves compiled files}.
2941 Note that in the past there was an add-on to ASDF called
2942 @code{ASDF-binary-locations}, developed by Gary King.
2943 That add-on has been merged into ASDF proper,
2944 then superseded by the @code{asdf-output-translations} facility.
2946 Note that use of @code{asdf-output-translations}
2947 can interfere with one aspect of your systems
2948 --- if your system uses @code{*load-truename*} to find files
2949 (e.g., if you have some data files stored with your program),
2950 then the relocation that this ASDF customization performs
2951 is likely to interfere.
2952 Use @code{asdf:system-relative-pathname} to locate a file
2953 in the source directory of some system, and
2954 use @code{asdf:apply-output-translations} to locate a file
2955 whose pathname has been translated by the facility.
2957 @subsection ``How can I wholly disable the compiler output cache?''
2959 To permanently disable the compiler output cache
2960 for all future runs of ASDF, you can:
2963 mkdir -p ~/.config/common-lisp/asdf-output-translations.conf.d/
2964 echo ':disable-cache' > ~/.config/common-lisp/asdf-output-translations.conf.d/99-disable-cache.conf
2967 This assumes that you didn't otherwise configure the ASDF files
2968 (if you did, edit them again),
2969 and don't somehow override the configuration at runtime
2970 with a shell variable (see below) or some other runtime command
2971 (e.g. some call to @code{asdf:initialize-output-translations}).
2973 To disable the compiler output cache in Lisp processes
2974 run by your current shell, try (assuming @code{bash} or @code{zsh})
2975 (on Unix and cygwin only):
2978 export ASDF_OUTPUT_TRANSLATIONS=/:
2981 To disable the compiler output cache just in the current Lisp process,
2982 use (after loading ASDF but before using it):
2985 (asdf:disable-output-translations)
2988 @section Issues with using and extending ASDF to define systems
2990 @subsection ``How can I cater for unit-testing in my system?''
2992 ASDF provides a predefined test operation, @code{test-op}.
2993 @xref{Predefined operations of ASDF, test-op}.
2994 The test operation, however, is largely left to the system definer to specify.
2995 @code{test-op} has been
2996 a topic of considerable discussion on the
2997 @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list},
2999 @url{https://launchpad.net/asdf,launchpad bug-tracker}.
3001 Here are some guidelines:
3005 For a given system, @var{foo}, you will want to define a corresponding
3006 test system, such as @var{foo-test}. The reason that you will want this
3007 separate system is that ASDF does not out of the box supply components
3008 that are conditionally loaded. So if you want to have source files
3009 (with the test definitions) that will not be loaded except when testing,
3010 they should be put elsewhere.
3013 The @var{foo-test} system can be defined in an asd file of its own or
3014 together with @var{foo}. An aesthetic preference against cluttering up
3015 the filesystem with extra asd files should be balanced against the
3016 question of whether one might want to directly load @var{foo-test}.
3017 Typically one would not want to do this except in early stages of
3021 Record that testing is implemented by @var{foo-test}. For example:
3023 (defsystem @var{foo}
3024 :in-order-to ((test-op (test-op @var{foo-test})))
3027 (defsystem @var{foo-test}
3028 :depends-on (@var{foo} @var{my-test-library} ...)
3033 This procedure will allow you to support users who do not wish to
3034 install your test framework.
3036 One oddity of ASDF is that @code{operate} (@pxref{Operations,operate})
3037 does not return a value. So in current versions of ASDF there is no
3038 reliable programmatic means of determining whether or not a set of tests
3039 has passed, or which tests have failed. The user must simply read the
3040 console output. This limitation has been the subject of much
3043 @subsection ``How can I cater for documentation generation in my system?''
3045 The ASDF developers are currently working to add a @code{doc-op}
3046 to the set of predefined ASDF operations.
3047 @xref{Predefined operations of ASDF}.
3048 See also @url{https://bugs.launchpad.net/asdf/+bug/479470}.
3052 @subsection ``How can I maintain non-Lisp (e.g. C) source files?''
3054 See @code{cffi}'s @code{cffi-grovel}.
3056 @anchor{report-bugs}
3059 @subsection ``I want to put my module's files at the top level. How do I do this?''
3061 By default, the files contained in an asdf module go
3062 in a subdirectory with the same name as the module.
3063 However, this can be overridden by adding a @code{:pathname ""} argument
3064 to the module description.
3065 For example, here is how it could be done
3066 in the spatial-trees ASDF system definition for ASDF 2:
3069 (asdf:defsystem :spatial-trees
3075 (:file "basedefs" :depends-on ("package"))
3076 (:file "rectangles" :depends-on ("package"))))
3082 (:file "greene-trees" :depends-on ("r-trees"))
3083 (:file "rstar-trees" :depends-on ("r-trees"))
3084 (:file "rplus-trees" :depends-on ("r-trees"))
3085 (:file "x-trees" :depends-on ("r-trees" "rstar-trees"))))
3090 ((:static-file "spatial-tree-viz.lisp")))
3095 ((:static-file "spatial-tree-test.lisp")))
3096 (:static-file "LICENCE")
3097 (:static-file "TODO")))
3100 All of the files in the @code{tree-impls} module are at the top level,
3101 instead of in a @file{tree-impls/} subdirectory.
3103 Note that the argument to @code{:pathname} can be either a pathname object or a string.
3104 A pathname object can be constructed with the @file{#p"foo/bar/"} syntax,
3105 but this is discouraged because the results of parsing a namestring are not portable.
3106 A pathname can only be portably constructed with such syntax as
3107 @code{#.(make-pathname :directory '(:relative "foo" "bar"))},
3108 and similarly the current directory can only be portably specified as
3109 @code{#.(make-pathname :directory '(:relative))}.
3110 However, as of ASDF 2, you can portably use a string to denote a pathname.
3111 The string will be parsed as a @code{/}-separated path from the current directory,
3112 such that the empty string @code{""} denotes the current directory, and
3113 @code{"foo/bar"} (no trailing @code{/} required in the case of modules)
3114 portably denotes the same subdirectory as above.
3115 When files are specified, the last @code{/}-separated component is interpreted
3116 either as the name component of a pathname
3117 (if the component class specifies a pathname type),
3118 or as a name component plus optional dot-separated type component
3119 (if the component class doesn't specifies a pathname type).
3122 @node TODO list, Inspiration, FAQ, Top
3123 @comment node-name, next, previous, up
3126 Here is an old list of things to do,
3127 in addition to the bugs that are now tracked on launchpad:
3128 @url{https://launchpad.net/asdf}.
3130 @section Outstanding spec questions, things to add
3132 ** packaging systems
3134 *** manual page component?
3136 ** style guide for .asd files
3138 You should either use keywords or be careful
3139 with the package that you evaluate defsystem forms in.
3140 Otherwise @code{(defsystem partition ...)}
3141 being read in the @code{cl-user} package
3142 will intern a @code{cl-user:partition} symbol,
3143 which will then collide with the @code{partition:partition} symbol.
3145 Actually there's a hairier packages problem to think about too.
3146 @code{in-order-to} is not a keyword:
3147 if you read @code{defsystem} forms in a package that doesn't use ASDF,
3148 odd things might happen.
3151 ** extending defsystem with new options
3153 You might not want to write a whole parser,
3154 but just to add options to the existing syntax.
3155 Reinstate @code{parse-option} or something akin.
3158 ** document all the error classes
3160 ** what to do with compile-file failure
3162 Should check the primary return value from compile-file and see if
3163 that gets us any closer to a sensible error handling strategy
3167 lift unix-dso stuff from db-sockets
3171 A ``dry run'' of an operation can be made with the following form:
3174 (traverse (make-instance '<operation-name>)
3175 (find-system <system-name>)
3179 This uses unexported symbols.
3180 What would be a nice interface for this functionality?
3182 @section Missing bits in implementation
3184 ** reuse the same scratch package whenever a system is reloaded from disk
3186 ** proclamations probably aren't
3188 ** when a system is reloaded with fewer components than it previously had, odd things happen
3190 We should do something inventive when processing a @code{defsystem} form,
3191 like take the list of kids and @code{setf} the slot to @code{nil},
3192 then transfer children from old to new list as they're found.
3194 ** (stuff that might happen later)
3196 *** Propagation of the @code{:force} option.
3200 @code{(asdf:compile-system :araneida :force t)}
3202 also forces compilation of every other system the @code{:araneida} system depends on.
3203 This is rarely useful to me;
3204 usually, when I want to force recompilation of something more than a single source file,
3205 I want to recompile only one system.
3206 So it would be more useful to have @code{make-sub-operation}
3207 refuse to propagate @code{:force t} to other systems, and
3208 propagate only something like @code{:force :recursively}.
3210 Ideally what we actually want is some kind of criterion that says
3211 to which systems (and which operations) a @code{:force} switch will propagate.
3213 The problem is perhaps that ``force'' is a pretty meaningless concept.
3214 How obvious is it that @code{load :force t} should force @emph{compilation}?
3215 But we don't really have the right dependency setup
3216 for the user to compile @code{:force t} and expect it to work
3217 (files will not be loaded after compilation, so the compile
3218 environment for subsequent files will be emptier than it needs to be)
3220 What does the user actually want to do when he forces?
3221 Usually, for me, update for use with a new version of the Lisp compiler.
3222 Perhaps for recovery when he suspects that something has gone wrong.
3223 Or else when he's changed compilation options or configuration
3224 in some way that's not reflected in the dependency graph.
3226 Other possible interface: have a ``revert'' function akin to @code{make clean}.
3229 (asdf:revert 'asdf:compile-op 'araneida)
3232 would delete any files produced by @code{(compile-system :araneida)}.
3233 Of course, it wouldn't be able to do much about stuff in the image itself.
3235 How would this work?
3239 There's a difference between a module's dependencies (peers)
3240 and its components (children).
3241 Perhaps there's a similar difference in operations?
3242 For example, @code{(load "use") depends-on (load "macros")} is a peer,
3243 whereas @code{(load "use") depends-on (compile "use")}
3244 is more of a ``subservient'' relationship.
3246 @node Inspiration, Concept Index, TODO list, Top
3247 @comment node-name, next, previous, up
3248 @chapter Inspiration
3250 @section mk-defsystem (defsystem-3.x)
3252 We aim to solve basically the same problems as @code{mk-defsystem} does.
3253 However, our architecture for extensibility
3254 better exploits CL language features (and is documented),
3255 and we intend to be portable rather than just widely-ported.
3256 No slight on the @code{mk-defsystem} authors and maintainers is intended here;
3257 that implementation has the unenviable task
3258 of supporting pre-ANSI implementations, which is no longer necessary.
3260 The surface defsystem syntax of asdf is more-or-less compatible with
3261 @code{mk-defsystem}, except that we do not support
3262 the @code{source-foo} and @code{binary-foo} prefixes
3263 for separating source and binary files, and
3264 we advise the removal of all options to specify pathnames.
3266 The @code{mk-defsystem} code for topologically sorting
3267 a module's dependency list was very useful.
3269 @section defsystem-4 proposal
3271 Marco and Peter's proposal for defsystem 4 served as the driver for
3272 many of the features in here. Notable differences are:
3276 We don't specify output files or output file extensions
3277 as part of the system.
3279 If you want to find out what files an operation would create,
3283 We don't deal with CL packages
3285 If you want to compile in a particular package, use an @code{in-package} form
3286 in that file (ilisp / SLIME will like you more if you do this anyway)
3289 There is no proposal here that @code{defsystem} does version control.
3291 A system has a given version which can be used to check dependencies,
3295 The defsystem 4 proposal tends to look more at the external features,
3296 whereas this one centres on a protocol for system introspection.
3298 @section kmp's ``The Description of Large Systems'', MIT AI Memu 801
3300 Available in updated-for-CL form on the web at
3301 @url{http://nhplace.com/kent/Papers/Large-Systems.html}
3303 In our implementation we borrow kmp's overall @code{PROCESS-OPTIONS}
3304 and concept to deal with creating component trees
3305 from @code{defsystem} surface syntax.
3306 [ this is not true right now, though it used to be and
3307 probably will be again soon ]
3310 @c -------------------
3313 @node Concept Index, Function and Class Index, Inspiration, Top
3314 @unnumbered Concept Index
3318 @node Function and Class Index, Variable Index, Concept Index, Top
3319 @unnumbered Function and Class Index
3323 @node Variable Index, , Function and Class Index, Top
3324 @unnumbered Variable Index