-README,v 1.39 2006/08/21 10:52:32 crhodes Exp -*- Text -*-
-
-The canonical documentation for asdf is in the file asdf.texinfo.
-The significant overlap between this file and that will one day be
-resolved by deleting text from this file; in the meantime, please look
-there before here.
-
-
-
-asdf: another system definition facility
+ASDF: another system definition facility
========================================
-* Getting the latest version
-
-0) Decide which version you want. HEAD is the newest version and
-usually OK, whereas RELEASE is for cautious people (e.g. who already
-have systems using asdf that they don't want broken), a slightly older
-version about which none of the HEAD users have complained.
-
-1) Check it out from sourceforge cCLan CVS:
-
-1a) cvs -d:pserver:anonymous@cvs.cclan.sourceforge.net:/cvsroot/cclan login
- (no password: just press Enter)
-
-1a.1) cvs -z3 -d:pserver:anonymous@cvs.cclan.sourceforge.net:/cvsroot/cclan
- co -r RELEASE asdf
-
-or for the bleeding edge, instead
-
-1a.2) cvs -z3 -d:pserver:anonymous@cvs.cclan.sourceforge.net:/cvsroot/cclan
- co -A asdf
-
-If you are tracking the bleeding edge, you may want to subscribe to
-the cclan-commits mailing list (see
-<URL:http://sourceforge.net/mail/?group_id=28536>) to receive commit
-messages and diffs whenever changes are made.
-
-For more CVS information, look at http://sourceforge.net/cvs/?group_id=28536
-
-
-* Getting started
-
-- The single file asdf.lisp is all you need to use asdf normally. For
-maximum convenience you want to have it loaded whenever you start your
-Lisp implementation, by loading it from the startup script, or dumping
-a custom core, or something.
-
-- The variable asdf:*central-registry* is a list of system directory
- designators. A system directory designator is a form which will be
- evaluated whenever a system is to be found, and must evaluate to a
- directory to look in. For example, you might have
-
- (*default-pathname-defaults* "/home/me/cl/systems/"
- "/usr/share/common-lisp/systems/")
-
- (When we say "directory" here, we mean "designator for a pathname
- with a supplied DIRECTORY component")
-
- It is possible to customize the system definition file search.
- That's considered advanced use, and covered later: search forward
- for *system-definition-search-functions*
-
-- To compile and load a system 'foo', you need to (1) ensure that
- foo.asd is in one of the directories in *central-registry* (a
- symlink to the real location of foo.asd is preferred), (2) execute
- ``(asdf:operate 'asdf:load-op 'foo)''
-
- $ cd /home/me/cl/systems/
- $ ln -s ~/src/foo/foo.asd .
- $ lisp
- * (asdf:operate 'asdf:load-op 'foo)
-
-- To write your own system definitions, look at the test systems in
- test/ , and read the rest of this. Ignore systems/ which is old
- and may go away when next I clean up
-
-- Syntax is similar to mk-defsystem 3 for straightforward systems, you
- may only need to remove the :source-pathname option (and replace it
- with :pathname if the asd file is not in the same place as the
- system sources)
-
-- Join cclan-list@lists.sf.net for discussion, bug reports, questions, etc
-
-- cclan.asd and the source files listed therein contain useful extensions
- for maintainers of systems in the cCLan. If this isn't you, you
- don't need them - although you may want to look at them anyway
-
-- For systems that do complicated things (e.g. compiling C files to
- load as foreign code), the packages in vn-cclan may provide some
- guidance. db-sockets, for example, is known to do outlandish things
- with preprocessors
-
- http://ww.telent.net/cliki/vn-cclan
-
-
-
-* Concepts
-
-This system definition utility talks in terms of 'components' and
-'operations'.
-
-Components form systems: a component represents a source file, or a
-collection of components. A system is therefore a component,
-recursively formed of a tree of subcomponents.
-
-Operations are instantiated then performed on the nodes of a tree to
-do things like
-
- - compile all its files
- - load the files into a running lisp environment
- - copy its source files somewhere else
-
-Operations can be invoked directly, or examined to see what their
-effects would be without performing them. There are a bunch of
-methods specialised on operation and component type which actually do
-the grunt work.
-
-asdf is extensible to new operations and to new component types. This
-allows the addition of behaviours: for example, a new component could
-be added for Java JAR archives, and methods specialised on compile-op
-added for it that would accomplish the relevant actions. Users
-defining their own operations and component types should inherit from
-the asdf base classes asdf:operation and asdf:component respectively.
-
-* Inspiration
-
-** mk-defsystem (defsystem-3.x)
-
-We aim to solve basically the same problems as mk-defsystem does.
-However, our architecture for extensibility better exploits CL
-language features (and is documented), and we intend to be portable
-rather than just widely-ported. No slight on the mk-defsystem authors
-and maintainers is intended here; that implementation has the
-unenviable task of supporting non-ANSI implementations, which I
-propose to ignore.
-
-The surface defsystem syntax of asdf is more-or-less compatible with
-mk-defsystem
-
-The mk-defsystem code for topologically sorting a module's dependency
-list was very useful.
-
-** defsystem-4 proposal
-
-Marco and Peter's proposal for defsystem 4 served as the driver for
-many of the features in here. Notable differences are
-
-- we don't specify output files or output file extensions as part of
- the system
-
- If you want to find out what files an operation would create, ask
- the operation
-
-- we don't deal with CL packages
-
- If you want to compile in a particular package, use an in-package
- form in that file (ilisp will like you more if you do this anyway)
-
-- there is no proposal here that defsystem does version control.
-
- A system has a given version which can be used to check
- dependencies, but that's all.
-
-The defsystem 4 proposal tends to look more at the external features,
-whereas this one centres on a protocol for system introspection.
-
-** kmp's "The Description of Large Systems", MIT AI Memu 801
-
-Available in updated-for-CL form on the web at
-http://world.std.com/~pitman/Papers/Large-Systems.html
-
-In our implementation we borrow kmp's overall PROCESS-OPTIONS and
-concept to deal with creating component trees from defsystem surface
-syntax. [ this is not true right now, though it used to be and
-probably will be again soon ]
-
-
-* The Objects
-
-** component
-
-*** Component Attributes
-
-**** A name (required)
-
-This is a string or a symbol. If a symbol, its name is taken and
-lowercased. The name must be a suitable value for the :name initarg
-to make-pathname in whatever filesystem the system is to be found.
-
-The lower-casing-symbols behaviour is unconventional, but was selected
-after some consideration. Observations suggest that the type of
-systems we want to support either have lowercase as customary case
-(Unix, Mac, windows) or silently convert lowercase to uppercase
-(lpns), so this makes more sense than attempting to use :case :common,
-which is reported not to work on some implementations
-
-**** a version identifier (optional)
-
-This is used by the test-system-version operation (see later).
-
-**** *features* required
-
-Traditionally defsystem users have used reader conditionals to include
-or exclude specific per-implementation files. This means that any
-single implementation cannot read the entire system, which becomes a
-problem if it doesn't wish to compile it, but instead for example to
-create an archive file containing all the sources, as it will omit to
-process the system-dependent sources for other systems.
-
-Each component in an asdf system may therefore specify features using
-the same syntax as #+ does, and it will (somehow) be ignored for
-certain operations unless the feature conditional matches
-
-**** dependencies on its siblings (optional but often necessary)
-
-There is an excitingly complicated relationship between the initarg
-and the method that you use to ask about dependencies
-
-Dependencies are between (operation component) pairs. In your
-initargs, you can say
-
-:in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
- (load-op (load-op "foo")))
-
-- before performing compile-op on this component, we must perform
-load-op on "a" and "b", and compile-op on c, - before performing
-load-op, we have to load "foo"
-
-The syntax is approximately
-
-(this-op {(other-op required-components)}+)
-
-required-components := component-name
- | (required-components required-components)
-
-component-name := string
- | (:version string minimum-version-object)
-
-[ This is on a par with what ACL defsystem does. mk-defsystem is less
-general: it has an implied dependency
-
- for all x, (load x) depends on (compile x)
-
-and using a :depends-on argument to say that b depends on a _actually_
-means that
-
- (compile b) depends on (load a)
-
-This is insufficient for e.g. the McCLIM system, which requires that
-all the files are loaded before any of them can be compiled ]
-
-In asdf, the dependency information for a given component and
-operation can be queried using (component-depends-on operation
-component), which returns a list
-
-((load-op "a") (load-op "b") (compile-op "c") ...)
-
-component-depends-on can be subclassed for more specific
-component/operation types: these need to (call-next-method) and append
-the answer to their dependency, unless they have a good reason for
-completely overriding the default dependencies
-
-(If it weren't for CLISP, we'd be using a LIST method combination to
-do this transparently. But, we need to support CLISP. If you have
-the time for some CLISP hacking, I'm sure they'd welcome your fixes)
-
-**** a pathname
-
-This is optional and if absent will be inferred from name, type (the
-subclass of source-file), and the location of parent.
-
-The rules for this inference are:
-
-(for source-files)
-- the host is taken from the parent
-- pathname type is (source-file-type component system)
-- the pathname case option is :local
-- the pathname is merged against the parent
-
-(for modules)
-- the host is taken from the parent
-- the name and type are NIL
-- the directory is (:relative component-name)
-- the pathname case option is :local
-- the pathname is merged against the parent
-
-Note that the DEFSYSTEM operator (used to create a "top-level" system)
-does additional processing to set the filesystem location of the
-top component in that system. This is detailed elsewhere
-
-The answer to the frequently asked question "how do I create a system
-definition where all the source files have a .cl extension" is thus
-
-(defmethod source-file-type ((c cl-source-file) (s (eql (find-system 'my-sys))))
- "cl")
-
-**** properties (optional)
-
-Packaging systems often require information about files or systems
-additional to that specified here. Programs that create vendor
-packages out of asdf systems therefore have to create "placeholder"
-information to satisfy these systems. Sometimes the creator of an
-asdf system may know the additional information and wish to provide it
-directly.
-
-(component-property component property-name) and associated setf method
-will allow the programmatic update of this information. Property
-names are compared as if by EQL, so use symbols or keywords or something
-
-** Subclasses of component
-
-*** 'source-file'
-
-A source file is any file that the system does not know how to
-generate from other components of the system.
-
-(Note that this is not necessarily the same thing as "a file
-containing data that is typically fed to a compiler". If a file is
-generated by some pre-processor stage (e.g. a ".h" file from ".h.in"
-by autoconf) then it is not, by this definition, a source file.
-Conversely, we might have a graphic file that cannot be automatically
-regenerated, or a proprietary shared library that we received as a
-binary: these do count as source files for our purposes. All
-suggestions for better terminology gratefully received)
-
-Subclasses of source-file exist for various languages.
-
-*** 'module', a collection of sub-components
-
-This has extra slots for
-
- :components - the components contained in this module
-
- :default-component-class - for child components which don't specify
- their class explicitly
-
- :if-component-dep-fails takes one of the values :fail, :try-next, :ignore
- (default value is :fail). The other values can be used for implementing
- conditional compilation based on implementation *features*, where
- it is not necessary for all files in a module to be compiled
-
-The default operation knows how to traverse a module, so most
-operations will not need to provide methods specialised on modules.
-
-The module may be subclassed to represent components such as
-foreign-language linked libraries or archive files.
-
-*** system, subclasses module
-
-A system is a module with a few extra attributes for documentation
-purposes. In behaviour, it's usually identical.
-
-Users can create new classes for their systems: the default defsystem
-macro takes a :classs keyword argument.
-
-
-** operation
-
-An operation is instantiated whenever the user asks that an operation
-be performed, inspected, or etc. The operation object contains
-whatever state is relevant to this purpose (perhaps a list of visited
-nodes, for example) but primarily is a nice thing to specialise
-operation methods on and easier than having them all be EQL methods.
-
-There are no differences between standard operations and user-defined
-operations, except that the user is respectfully requested to keep his
-(or more importantly, our) package namespace clean
-
-*** invoking operations
-
-(operate operation system &rest keywords-args)
-
-keyword-args are passed to the make-instance call when creating the
-operation: valid keywords depend on the initargs that the operation is
-defined to accept. Note that dependencies may cause the operation to
-invoke other operations on the system or its components: the new
-operation will be created with the same initargs as the original one.
-
-oos is accepted as a synonym for operate
-
-*** standard operations
-
-**** feature-dependent-op
-
-This is not intended to be instantiated directly, but other operations
-may inherit from it. An instance of feature-dependent-op will ignore
-any components which have a `features' attribute, unless the feature
-combination it designates is satisfied by *features*
-
-See the earlier explanation about the component features attribute for
-more information
-
-**** compile-op &key proclamations
-
-If proclamations are supplied, they will be proclaimed. This is a
-good place to specify optimization settings
-
-When creating a new component, you should provide methods for this.
-
-If you invoke compile-op as a user, component dependencies often mean
-you may get some parts of the system loaded. This may not necessarily
-be the whole thing, though; for your own sanity it is recommended that
-you use load-op if you want to load a system.
-
-**** load-op &key proclamations
-
-The default methods for load-op compile files before loading them.
-For parity, your own methods on new component types should probably do
-so too
-
-**** load-source-op
-
-This method will load the source for the files in a module even if the
-source files have been compiled. Systems sometimes have knotty
-dependencies which require that sources are loaded before they can be
-compiled. This is how you do that.
-
-If you are creating a component type, you need to implement this
-operation - at least, where meaningful.
-
-**** test-system-version &key minimum
-
-Asks the system whether it satisfies a version requirement.
-
-The default method accepts a string, which is expected to contain of a
-number of integers separated by #\. characters. The method is not
-recursive. The component satisfies the version dependency if it has
-the same major number as required and each of its sub-versions is
-greater than or equal to the sub-version number required.
-
-(defun version-satisfies (x y)
- (labels ((bigger (x y)
- (cond ((not y) t)
- ((not x) nil)
- ((> (car x) (car y)) t)
- ((= (car x) (car y))
- (bigger (cdr x) (cdr y))))))
- (and (= (car x) (car y))
- (or (not (cdr y)) (bigger (cdr x) (cdr y))))))
-
-If that doesn't work for your system, you can override it. I hope
-yoyu have as much fun writing the new method as #lisp did
-reimplementing this one.
-
-*** Creating new operations
-
-subclass operation, provide methods for source-file for
-
-- output-files
-- perform
- The perform method must call output-files to find out where to
- put its files, because the user is allowed to override output-files
- for local policy
-- explain
-- operation-done-p, if you don't like the default one
-
-* Writing system definitions
-
-** System designators
-
-System designators are strings or symbols and behave just like
-any other component names (including case conversion)
-
-** find-system
-
-Given a system designator, find-system finds an actual system - either
-in memory, or in a file on the disk. It funcalls each element in the
-*system-definition-search-functions* list, expecting a pathname to be
-returned.
-
-If a suitable file exists, it is loaded if
-
-- there is no system of that name in memory,
-- the file's last-modified time exceeds the last-modified time of the
- system in memory
-
-When system definitions are loaded from .asd files, a new scratch
-package is created for them to load into, so that different systems do
-not overwrite each others operations. The user may also wish to (and
-is recommended to) include defpackage and in-package forms in his
-system definition files, however, so that they can be loaded manually
-if need be. It is not recommended to use the CL-USER package for this
-purpose, as definitions made in this package will affect the parsing
-of asdf systems.
-
-For convenience in the normal case, and for backward compatibility
-with the spirit of mk-defsystem, the default contents of
-*system-definition-search-functions* is a function called
-sysdef-central-registry-search. This looks in each of the directories
-given by evaluating members of *central-registry*, for a file whose
-name is the name of the system and whose type is "asd". The first
-such file is returned, whether or not it turns out to actually define
-the appropriate system
-
-
-
-** Syntax
-
-Systems can always be constructed programmatically by instantiating
-components using make-instance. For most purposes, however, it is
-likely that people will want a static defystem form.
-
-asdf is based around the principle that components should not have to
-know defsystem syntax. That is, the initargs that a component accepts
-are not necessarily related to the defsystem form which creates it.
-
-A defsystem parser must implement a `defsystem' macro, which can
-be named for compatibility with whatever other system definition
-utility is being emulated. It should instantiate components in
-accordance with whatever language it accepts, and register the topmost
-component using REGISTER-SYSTEM
-
-*** Native syntax
-
-The native syntax is inspired by mk-defsystem, to the extent that it
-should be possible to take most straightforward mk- system definitions
-and run them with only light editing. For my convenience, this turns
-out to be basically the same as the initargs to the various
-components, with a few extensions for convenience
-
-system-definition := ( defsystem system-designator {option}* )
-
-option := :components component-list
- | :pathname pathname
- | :default-component-class
- | :perform method-form
- | :explain method-form
- | :output-files method-form
- | :operation-done-p method-form
- | :depends-on ( {simple-component-name}* )
- | :serial [ t | nil ]
- | :in-order-to ( {dependency}+ )
-
-component-list := ( {component-def}* )
-
-component-def := simple-component-name
- | ( component-type name {option}* )
-
-component-type := :module | :file | :system | other-component-type
-
-dependency := (dependent-op {requirement}+)
-requirement := (required-op {required-component}+)
- | (feature feature-name)
-dependent-op := operation-name
-required-op := operation-name | feature
-
-For example
-
-(defsystem "foo"
- :version "1.0"
- :components ((:module "foo" :components ((:file "bar") (:file"baz")
- (:file "quux"))
- :perform (compile-op :after (op c)
- (do-something c))
- :explain (compile-op :after (op c)
- (explain-something c)))
- (:file "blah")))
-
-
-The method-form tokens need explaining: esentially,
-
- :perform (compile-op :after (op c)
- (do-something c))
- :explain (compile-op :after (op c)
- (explain-something c)))
-has the effect of
-
-(defmethod perform :after ((op compile-op) (c (eql ...)))
- (do-something c))
-(defmethod explain :after ((op compile-op) (c (eql ...)))
- (explain-something c))
-
-where ... is the component in question; note that although this also
-supports :before methods, they may not do what you want them to - a
-:before method on perform ((op compile-op) (c (eql ...))) will run
-after all the dependencies and sub-components have been processed, but
-before the component in question has been compiled.
-
-**** Serial dependencies
-
-If the `:serial t' option is specified for a module, asdf will add
-dependencies for each each child component, on all the children
-textually preceding it. This is done as if by :depends-on
-
-:components ((:file "a") (:file "b") (:file "c"))
-:serial t
-
-is equivalent to
-:components ((:file "a")
- (:file "b" :depends-on ("a"))
- (:file "c" :depends-on ("a" "b")))
-
-
-
-have all the
-
-**** Source location
-
-The :pathname option is optional in all cases for native-syntax
-systems, and in the usual case the user is recommended not to supply
-it. If it is not supplied for the top-level form, defsystem will set
-it from
-
-- The host/device/directory parts of *load-truename*, if it is bound
-- *default-pathname-defaults*, otherwise
-
-If a system is being redefined, the top-level pathname will be
-
-- changed, if explicitly supplied or obtained from *load-truename*
-- changed if it had previously been set from *default-pathname-defaults*
-- left as before, if it had previously been set from *load-truename*
- and *load-truename* is not now bound
-
-These rules are designed so that (i) find-system will load a system
-from disk and have its pathname default to the right place, (ii)
-this pathname information will not be overwritten with
-*default-pathname-defaults* (which could be somewhere else altogether)
-if the user loads up the .asd file into his editor and
-interactively re-evaluates that form
-
- * Error handling
-
-It is an error to define a system incorrectly: an implementation may
-detect this and signal a generalised instance of
-SYSTEM-DEFINITION-ERROR.
-
-Operations may go wrong (for example when source files contain
-errors). These are signalled using generalised instances of
-OPERATION-ERROR, with condition readers ERROR-COMPONENT and
-ERROR-OPERATION for the component and operation which erred.
-
-* Compilation error and warning handling
-
-ASDF checks for warnings and errors when a file is compiled. The
-variables *compile-file-warnings-behaviour* and
-*compile-file-errors-behavior* controls the handling of any such
-events. The valid values for these variables are :error, :warn, and
-:ignore.
-
-----------------------------------------------------------
- TODO List
-----------------------------------------------------------
-
-* Outstanding spec questions, things to add
-
-** packaging systems
-
-*** manual page component?
-
-** style guide for .asd files
-
-You should either use keywords or be careful with the package that you
-evaluate defsystem forms in. Otherwise (defsystem partition ...)
-being read in the cl-user package will intern a cl-user:partition
-symbol, which will then collide with the partition:partition symbol.
-
-Actually there's a hairier packages problem to think about too.
-in-order-to is not a keyword: if you read defsystem forms in a package
-that doesn't use ASDF, odd things might happen
-
-** extending defsystem with new options
-
-You might not want to write a whole parser, but just to add options to
-the existing syntax. Reinstate parse-option or something akin
-
-** document all the error classes
-
-** what to do with compile-file failure
-
-Should check the primary return value from compile-file and see if
-that gets us any closer to a sensible error handling strategy
-
-** foreign files
-
-lift unix-dso stuff from db-sockets
-
-** Diagnostics
-
-A "dry run" of an operation can be made with the following form:
-
-(traverse (make-instance '<operation-name>)
- (find-system <system-name>)
- 'explain)
-
-This uses unexported symbols. What would be a nice interface for this
-functionality?
-
-** patches
-
-Sometimes one wants to
-
-
-* missing bits in implementation
-
-** all of the above
-** reuse the same scratch package whenever a system is reloaded from disk
-** rules for system pathname defaulting are not yet implemented properly
-** proclamations probably aren't
-** when a system is reloaded with fewer components than it previously
- had, odd things happen
-
-we should do something inventive when processing a defsystem form,
-like take the list of kids and setf the slot to nil, then transfer
-children from old to new list as they're found
-
-** traverse may become a normal function
-
-If you're defining methods on traverse, speak up.
-
-
-** a lot of load-op methods can be rewritten to use input-files
-
-so should be.
-
-
-** (stuff that might happen later)
-
-*** david lichteblau's patch for symlink resolution?
-
-*** Propagation of the :force option. ``I notice that
-
- (oos 'compile-op :araneida :force t)
-
-also forces compilation of every other system the :araneida system
-depends on. This is rarely useful to me; usually, when I want to force
-recompilation of something more than a single source file, I want to
-recompile only one system. So it would be more useful to have
-make-sub-operation refuse to propagate ":force t" to other systems, and
-propagate only something like ":force :recursively". ''
-
-Ideally what we actually want is some kind of criterion that says
-to which systems (and which operations) a :force switch will propagate.
-
-The problem is perhaps that 'force' is a pretty meaningless concept.
-How obvious is it that "load :force t" should force _compilation_?
-But we don't really have the right dependency setup for the user to
-compile :force t and expect it to work (files will not be loaded after
-compilation, so the compile environment for subsequent files will be
-emptier than it needs to be)
+If you want to use ASDF, read our manual:
-What does the user actually want to do when he forces? Usually, for
-me, update for use with a new version of the lisp compiler. Perhaps
-for recovery when he suspects that something has gone wrong. Or else
-when he's changed compilation options or configuration in some way
-that's not reflected in the dependency graph.
+ http://common-lisp.net/project/asdf/asdf.html
-Other possible interface: have a 'revert' function akin to 'make clean'
+The first few sections, Loading ASDF, Configuring ASDF and Using ASDF,
+will get you started as a simple user.
- (asdf:revert 'asdf:compile-op 'araneida)
+If you want to define your own systems, further read the section
+Defining systems with defsystem.
-would delete any files produced by 'compile-op 'araneida. Of course, it
-wouldn't be able to do much about stuff in the image itself.
+More information and additional links can be found on ASDF's
+home page at:
-How would this work?
+ http://common-lisp.net/project/asdf/
-traverse
-There's a difference between a module's dependencies (peers) and its
-components (children). Perhaps there's a similar difference in
-operations? For example, (load "use") depends-on (load "macros") is a
-peer, whereas (load "use") depends-on (compile "use") is more of a
-`subservient' relationship.
+last updated Wednesday; May 5, 2010
projects / sbcl.git / blobdiff