Skip to content
Snippets Groups Projects
asdf.texinfo 187 KiB
Newer Older
\input texinfo          @c -*- texinfo -*-
@c %**start of header
@settitle ASDF Manual
@c %**end of header

@c We use @&key, etc to escape & from TeX in lambda lists --
@c so we need to define them for info as well.
@c for install-info
@dircategory Software development
@direntry
* asdf: (asdf).           Another System Definition Facility (for Common Lisp)
This manual describes ASDF, a system definition facility
for Common Lisp programs and libraries.
You can find the latest version of this manual at
@url{http://common-lisp.net/project/asdf/asdf.html}.

ASDF Copyright @copyright{} 2001-2014 Daniel Barlow and contributors.
This manual Copyright @copyright{} 2001-2014 Daniel Barlow and contributors.
This manual revised @copyright{} 2009-2014 Robert P. Goldman and Francois-Rene Rideau.

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
``Software''), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

@end copying



@titlepage
@title ASDF: Another System Definition Facility
@c The following two commands start the copyright page.
@page
@vskip 0pt plus 1filll
@insertcopying
@end titlepage
@c Output the table of contents at the beginning.
@contents

@c -------------------

@ifnottex

@node Top, Introduction, (dir), (dir)
@top asdf: another system definition facility
* Introduction::
* Loading ASDF::
* Configuring ASDF::
* Using ASDF::
* Defining systems with defsystem::
* The object model of ASDF::
* Controlling where ASDF searches for systems::
* Controlling where ASDF saves compiled files::
* Error handling::
* Miscellaneous additional functionality::
* Getting the latest version::
* FAQ::
* TODO list::
* Inspiration::
* Concept Index::
* Function and Class Index::
* Variable Index::
@c @detailmenu
@c  --- The Detailed Node Listing ---
@c * The defsystem form::
@c * A more involved example::
@c * The defsystem grammar::
@c * Other code in .asd files::
@c * Predefined operations of ASDF::
@c * Creating new operations::
@c * Common attributes of components::
@c * Pre-defined subclasses of component::
@c * Creating new component types::
@c * Pre-defined subclasses of component::
@c * Creating new component types::
@end menu

@end ifnottex

@c -------------------

@node Introduction, Loading ASDF, Top, Top
@comment  node-name,  next,  previous,  up
@chapter Introduction
@cindex ASDF-related features
@vindex *features*
@cindex Testing for ASDF
@cindex ASDF versions
@cindex :asdf
@cindex :asdf2
ASDF is Another System Definition Facility:
a tool for specifying how systems of Common Lisp software
are comprised of components (sub-systems and files),
and how to operate on these components in the right order
so that they can be compiled, loaded, tested, etc.

ASDF presents three faces:
one for users of Common Lisp software who want to reuse other people's code,
one for writers of Common Lisp software who want to specify how to build their systems,
one for implementers of Common Lisp extensions who want to extend the build system.
Robert P. Goldman's avatar
Robert P. Goldman committed
@xref{Using ASDF,,Loading a system},
to learn how to use ASDF to load a system.
Robert P. Goldman's avatar
Robert P. Goldman committed
@xref{Defining systems with defsystem},
to learn how to define a system of your own.
@xref{The object model of ASDF}, for a description of
Robert P. Goldman's avatar
Robert P. Goldman committed
the ASDF internals and how to extend ASDF.
@emph{Nota Bene}:
We have released ASDF 2.000 on May 31st 2010,
and ASDF 3.0.0 on May 15th 2013.
Releases of ASDF 2 and later have since then been included
in all actively maintained CL implementations that used to bundle ASDF 1,
plus some implementations that didn't use to,
and has been made to work with all actively used CL implementations and a few more.
@xref{FAQ,,``What has changed between ASDF 1 and ASDF 2?''}.
Furthermore, it is possible to upgrade from ASDF 1 to ASDF 2 or ASDF 3 on the fly.
For this reason, we have stopped supporting ASDF 1 and ASDF 2.
If you are using ASDF 1 or ASDF 2 and are experiencing any kind of issues or limitations,
we recommend you upgrade to ASDF 3
--- and we explain how to do that. @xref{Loading ASDF}.
(In the context of compatibility requirements,
ASDF 2.27, released on Feb 1st 2013, and further 2.x releases up to 2.33,
count as pre-releases of ASDF 3, and define the :asdf3 feature;
still, please use the latest release).

Also note that ASDF is not to be confused with ASDF-Install.
ASDF-Install is not part of ASDF, but a separate piece of software.
ASDF-Install is also unmaintained and obsolete.
We recommend you use Quicklisp instead,
which works great and is being actively maintained.
If you want to download software from version control instead of tarballs,
so you may more easily modify it, we recommend clbuild.
@node Loading ASDF, Configuring ASDF, Introduction, Top
@comment  node-name,  next,  previous,  up
@vindex *central-registry*
@cindex link farm
@findex compile-system
@findex test-system
@cindex system directory designator
@findex operate
@findex oos
@section Loading a pre-installed ASDF

Most recent Lisp implementations include a copy of ASDF 3,
or at least ASDF 2.
You can usually load this copy using Common Lisp's @code{require} function:

@lisp
(require "asdf")
As of the writing of this manual,
the following implementations provide ASDF 3 this way:
abcl allegro ccl clisp cmucl ecl mkcl sbcl.
The following implementation only provide ASDF 2:
lispworks mkcl mocl xcl.
The following implementation doesn't provide it yet but will in an upcoming release:
The following implementations are obsolete, not actively maintained,
and most probably will never bundle it:
cormanlisp gcl genera mcl.
If the implementation you are using doesn't provide ASDF 2 or ASDF 3,
see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below.
If that implementation is still actively maintained,
you may also send a bug report to your Lisp vendor and complain
about their failing to provide ASDF.
NB: all implementations except clisp also accept
@code{(require "ASDF")}, @code{(require 'asdf)} and @code{(require :asdf)}.
For portability's sake, you probably want to use @code{(require "asdf")}.


@section Checking whether ASDF is loaded

To check whether ASDF is properly loaded in your current Lisp image,
you can run this form:

@lisp
(asdf:asdf-version)
@end lisp

If it returns a string,
that is the version of ASDF that is currently installed.

If it raises an error,
then either ASDF is not loaded, or
you are using an old version of ASDF.

You can check whether an old version is loaded
by checking if the ASDF package is present.
The form below will allow you to programmatically determine
whether a recent version is loaded, an old version is loaded,
or none at all:

@lisp
(when (find-package :asdf)
  (let ((ver (symbol-value (or (find-symbol (string :*asdf-version*) :asdf)
                               (find-symbol (string :*asdf-revision*) :asdf)))))
    (etypecase ver
      (string ver)
      (cons (with-output-to-string (s)
              (loop for (n . m) on ver do (princ n s) (when m (princ "." s)))))
If it returns @code{nil} then ASDF is not installed.
Otherwise it should return a string.
If it returns @code{"1.0"}, then it can actually be
any version before 1.77 or so, or some buggy variant of 1.x.
If you are experiencing problems with ASDF,
please try upgrading to the latest released version,
using the method below,
before you contact us and raise an issue.
@section Upgrading ASDF

If your implementation provides ASDF 3 or later,
you only need to @code{(require "asdf")}:
ASDF will automatically look whether an updated version of itself is available
amongst the regularly configured systems, before it compiles anything else.
See @pxref{Configuring ASDF} below.

If your implementation does provide ASDF 2 or later,
and you want to upgrade to a more recent version,
you need to install and configure your ASDF as above,
and additionally, you need to explicitly tell ASDF to load itself,
right after you require your implementation's old ASDF 2:
(require "asdf")
(asdf:load-system :asdf)
@end lisp

If on the other hand, your implementation only provides an old ASDF,
you will require a special configuration step and an old-style loading.
Take special attention to not omit the trailing directory separator
@code{/} at the end of your pathname:
(require "asdf")
(push #p"@var{/path/to/new/asdf/}" asdf:*central-registry*)
(asdf:oos 'asdf:load-op :asdf)
@end lisp

Note that ASDF 1 won't redirect its output files,
or at least won't do it according to your usual ASDF 2 configuration.
You therefore need write access on the directory
where you install the new ASDF,
and make sure you're not using it
for multiple mutually incompatible implementations.
At worst, you may have to have multiple copies of the new ASDF,
e.g. one per implementation installation, to avoid clashes.
Note that to our knowledge all implementations that provide ASDF
provide ASDF 2 in their latest release, so
you may want to upgrade your implementation rather than go through that hoop.
Finally, if you are using an unmaintained implementation
that does not provide ASDF at all,
see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below.

Note that there are some limitations to upgrading ASDF:
Previously loaded ASDF extensions become invalid, and will need to be reloaded.
This applies to e.g. CFFI-Grovel, or to hacks used by ironclad, etc.
Since it isn't possible to automatically detect what extensions are present
that need to be invalidated,
ASDF will actually invalidate all previously loaded systems
when it is loaded on top of a forward-incompatible ASDF version
(as per @code{asdf/upgrade::*oldest-forward-compatible-asdf-version*}
which is 2.33 at the time of this writing).
Furthermore, starting with ASDF 3 (2.27 or later),
this self-upgrade will be automatically attempted as the first step
to any system operation, to avoid any possibility of
a catastrophic attempt to self-upgrade in midflight.
For this an many other reasons,
it important reason to load, configure and upgrade ASDF (if needed)
as one of the very first things done by your build and startup scripts.
Until all implementations provide ASDF 3 or later,
it is safer if you upgrade ASDF and its extensions as a special step
at the very beginning of whatever script you are running,
before you start using ASDF to load anything else;
even afterwards, it is still a good idea, to avoid having to
load and reload code twice as it gets invalidated.

Until all implementations provide ASDF 3 or later,
it is unsafe to upgrade ASDF as part of loading a system
that depends on a more recent version of ASDF,
since the new one might shadow the old one while the old one is running,
and the running old one will be confused
when extensions are loaded into the new one.
In the meantime, we recommend that your systems should @emph{not} specify
@code{:depends-on (:asdf)}, or @code{:depends-on ((:version :asdf "2.010"))},
but instead that they check that a recent enough ASDF is installed,
with such code as:
@example
(unless (or #+asdf2 (asdf:version-satisfies
                     (asdf:asdf-version) *required-asdf-version*))
  (error "FOO requires ASDF ~A or later." *required-asdf-version*))
@end example
Until all implementations provide ASDF 3 or later,
it is unsafe for a system to transitively depend on ASDF
and not directly depend on ASDF;
if any of the system you use either depends-on asdf,
system-depends-on asdf, or transitively does,
you should also do as well.
@section Loading an otherwise installed ASDF

If your implementation doesn't include ASDF,
if for some reason the upgrade somehow fails,
does not or cannot apply to your case,
you will have to install the file @file{asdf.lisp}
somewhere and load it with:

@lisp
(load "/path/to/your/installed/asdf.lisp")
@end lisp
The single file @file{asdf.lisp} is all you normally need to use ASDF.
You can extract this file from latest release tarball on the
@url{http://common-lisp.net/project/asdf/,ASDF website}.
If you are daring and willing to report bugs, you can get
the latest and greatest version of ASDF from its git repository.
@xref{Getting the latest version}.

For maximum convenience you might want to have ASDF loaded
whenever you start your Lisp implementation,
for example by loading it from the startup script or dumping a custom core
--- check your Lisp implementation's manual for details.

@node Configuring ASDF, Using ASDF, Loading ASDF, Top
@comment  node-name,  next,  previous,  up

@chapter Configuring ASDF

@section Configuring ASDF to find your systems

So it may compile and load your systems, ASDF must be configured to find
the @file{.asd} files that contain system definitions.

Since ASDF 2, the preferred way to configure where ASDF finds your systems is
the @code{source-registry} facility,
fully described in its own chapter of this manual.
@xref{Controlling where ASDF searches for systems}.

The default location for a user to install Common Lisp software is under
@file{~/.local/share/common-lisp/source/}.
If you install software there (it can be a symlink),
you don't need further configuration.
If you're installing software yourself at a location that isn't standard,
you have to tell ASDF where you installed it. See below.
If you're using some tool to install software (e.g. Quicklisp),
the authors of that tool should already have configured ASDF.

The simplest way to add a path to your search path,
say @file{/home/luser/.asd-link-farm/}
is to create the directory
@file{~/.config/common-lisp/source-registry.conf.d/}
and there create a file with any name of your choice,
and with the type @file{conf},
for instance @file{42-asd-link-farm.conf}
@kbd{(:directory "/home/luser/.asd-link-farm/")}
If you want all the subdirectories under @file{/home/luser/lisp/}
to be recursively scanned for @file{.asd} files, instead use:


Note that your Operating System distribution or your system administrator
may already have configured system-managed libraries for you.

The required @file{.conf} extension allows you to have disabled files
or editor backups (ending in @file{~}), and works portably
(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
Excluded are files the name of which start with a @file{.} character.
It is customary to start the filename with two digits
that specify the order in which the directories will be scanned.

ASDF will automatically read your configuration
the first time you try to find a system.
You can reset the source-registry configuration with:
@lisp
(asdf:clear-source-registry)
@end lisp

And you probably should do so before you dump your Lisp image,
if the configuration may change
between the machine where you save it at the time you save it
and the machine you resume it at the time you resume it.
Actually, you should use @code{(asdf:clear-configuration)}
before you dump your Lisp image, which includes the above.
@section Configuring ASDF to find your systems --- old style

The old way to configure ASDF to find your systems is by
@code{push}ing directory pathnames onto the variable
@code{asdf:*central-registry*}.

You must configure this variable between the time you load ASDF
and the time you first try to use it.
Loading and configuring ASDF presumably happen
as part of some initialization script that builds or starts
your Common Lisp software system.
(For instance, some SBCL users used to put it in their @file{~/.sbclrc}.)

The @code{asdf:*central-registry*} is empty by default in ASDF 2 or ASDF 3,
but is still supported for compatibility with ASDF 1.
When used, it takes precedence over the above source-registry@footnote{
It is possible to further customize
the system definition file search.
That's considered advanced use, and covered later:
search forward for
@code{*system-definition-search-functions*}.
@xref{Defining systems with defsystem}.}.

For instance, if you wanted ASDF to find the @file{.asd} file
@file{/home/me/src/foo/foo.asd} your initialization script
could after it loads ASDF with @code{(require "asdf")}
configure it with:

@lisp
(push "/home/me/src/foo/" asdf:*central-registry*)
@end lisp

Note the trailing slash: when searching for a system,
ASDF will evaluate each entry of the central registry
and coerce the result to a pathname@footnote{
ASDF will indeed call @code{eval} on each entry.
It will also skip entries that evaluate to @code{nil}.

Strings and pathname objects are self-evaluating,
in which case the @code{eval} step does nothing;
but you may push arbitrary SEXP onto the central registry,
that will be evaluated to compute e.g. things that depend
on the value of shell variables or the identity of the user.

The variable @code{asdf:*central-registry*} is thus a list of
``system directory designators''.
A @dfn{system directory designator} is a form
which will be evaluated whenever a system is to be found,
and must evaluate to a directory to look in.
By ``directory'' here, we mean
``designator for a pathname with a supplied DIRECTORY component''.
}
at which point the presence of the trailing directory name separator
is necessary to tell Lisp that you're discussing a directory
rather than a file.

Typically, however, there are a lot of @file{.asd} files, and
a common idiom was to have to put
a bunch of @emph{symbolic links} to @file{.asd} files
in a common directory
and push @emph{that} directory (the ``link farm'')
to the
@code{asdf:*central-registry*}
instead of pushing each of the many involved directories
to the @code{asdf:*central-registry*}.
ASDF knows how to follow such @emph{symlinks}
to the actual file location when resolving the paths of system components
(on Windows, you can use Windows shortcuts instead of POSIX symlinks;
if you try aliases under MacOS, we are curious to hear about your experience).

For example, if @code{#p"/home/me/cl/systems/"} (note the trailing slash)
is a member of @code{*central-registry*}, you could set up the
system @var{foo} for loading with asdf with the following
commands at the shell:

@example
$ cd /home/me/cl/systems/
$ ln -s ~/src/foo/foo.asd .
@end example

This old style for configuring ASDF is not recommended for new users,
but it is supported for old users, and for users who want to programmatically
control what directories are added to the ASDF search path.

@section Configuring where ASDF stores object files
@findex clear-output-translations

ASDF lets you configure where object files will be stored.
Sensible defaults are provided and
you shouldn't normally have to worry about it.

This allows the same source code repository may be shared
between several versions of several Common Lisp implementations,
between several users using different compilation options
and without write privileges on shared source directories, etc.
This also allows to keep source directories uncluttered
by plenty of object files.

Starting with ASDF 2, the @code{asdf-output-translations} facility
was added to ASDF itself, that controls where object files will be stored.
This facility is fully described in a chapter of this manual,
@ref{Controlling where ASDF saves compiled files}.

The simplest way to add a translation to your search path,
say from @file{/foo/bar/baz/quux/}
to @file{/where/i/want/my/fasls/}
is to create the directory
@file{~/.config/common-lisp/asdf-output-translations.conf.d/}
and there create a file with any name of your choice and the type @file{conf},
for instance @file{42-bazquux.conf}
containing the line:

@kbd{("/foo/bar/baz/quux/" "/where/i/want/my/fasls/")}

To disable output translations for source under a given directory,
say @file{/toto/tata/}
you can create a file @file{40-disable-toto.conf}
with the line:

@kbd{("/toto/tata/")}

To wholly disable output translations for all directories,
you can create a file @file{00-disable.conf}
with the line:


Note that your Operating System distribution or your system administrator
may already have configured translations for you.
In absence of any configuration, the default is to redirect everything
under an implementation-dependent subdirectory of @file{~/.cache/common-lisp/}.
@xref{Controlling where ASDF searches for systems}, for full details.
The required @file{.conf} extension allows you to have disabled files
or editor backups (ending in @file{~}), and works portably
(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
Excluded are files the name of which start with a @file{.} character.
It is customary to start the filename with two digits
that specify the order in which the directories will be scanned.

ASDF will automatically read your configuration
the first time you try to find a system.
You can reset the source-registry configuration with:
(asdf:clear-output-translations)
And you probably should do so before you dump your Lisp image,
if the configuration may change
between the machine where you save it at the time you save it
and the machine you resume it at the time you resume it.
(Once again, you should use @code{(asdf:clear-configuration)}
before you dump your Lisp image, which includes the above.)
Finally note that before ASDF 2,
other ASDF add-ons offered the same functionality,
each in subtly different and incompatible ways:
ASDF-Binary-Locations, cl-launch, common-lisp-controller.
ASDF-Binary-Locations is now not needed anymore and should not be used.
cl-launch 3.000 and common-lisp-controller 7.2 have been updated
to just delegate this functionality to ASDF.
@node Using ASDF, Defining systems with defsystem, Configuring ASDF, Top
@comment  node-name,  next,  previous,  up

@section Resetting Configuration

When you dump and restore an image, or when you tweak your configuration,
you may want to reset the ASDF configuration.
For that you may use the following function:

@defun clear-configuration
   undoes any ASDF configuration,
   regarding source-registry or output-translations.
@end defun

If you use SBCL, CMUCL or SCL, you may use this snippet
so that the ASDF configuration be cleared automatically as you dump an image:

@example
#+(or cmu sbcl scl)
(pushnew 'clear-configuration
         #+(or cmu scl) ext:*before-save-initializations*
         #+sbcl sb-ext:*save-hooks*)
@end example

For compatibility with all Lisp implementations, however,
you might want instead your build script to explicitly call
@code{(asdf:clear-configuration)} at an appropriate moment before dumping.


@chapter Using ASDF

@section Loading a system

The system @var{foo} is loaded (and compiled, if necessary)
by evaluating the following Lisp form:
(asdf:load-system :@var{foo})
On some implementations (namely recent versions of
ABCL, Allegro CL, Clozure CL, CMUCL, ECL, GNU CLISP,
LispWorks, MKCL, SBCL and XCL),
Francois-Rene Rideau's avatar
Francois-Rene Rideau committed
ASDF hooks into the @code{CL:REQUIRE} facility
and you can just use:

@example
(require :@var{foo})
@end example

In older versions of ASDF, you needed to use
@code{(asdf:oos 'asdf:load-op :@var{foo})}.
Francois-Rene Rideau's avatar
Francois-Rene Rideau committed
If your ASDF is too old to provide @code{asdf:load-system} though
@xref{Loading ASDF,,Loading an otherwise installed ASDF}.
Note the name of a system is specified as a string or a symbol,
typically a keyword.
If a symbol (including a keyword), its name is taken and lowercased.
The name must be a suitable value for the @code{:name} initarg
to @code{make-pathname} in whatever filesystem the system is to be found.
The lower-casing-symbols behaviour is unconventional,
but was selected after some consideration.
Observations suggest that the type of systems we want to support
either have lowercase as customary case (unix, mac, windows)
or silently convert lowercase to uppercase (lpns),
so this makes more sense than attempting to use @code{:case :common},
which is reported not to work on some implementations

@section Other Operations
ASDF provides three commands for the most common system operations:
@code{load-system}, @code{compile-system} or @code{test-system}.
It also provides @code{require-system}, a version of @code{load-system}
that skips trying to update systems that are already loaded.
Because ASDF is an extensible system
for defining @emph{operations} on @emph{components},
it also provides a generic function @code{operate}
(which is usually abbreviated by @code{oos}).
You'll use @code{oos} whenever you want to do something beyond
compiling, loading and testing.
Output from ASDF and ASDF extensions are supposed to be sent
to the CL stream @code{*standard-output*},
and so rebinding that stream around calls to @code{asdf:operate}
should redirect all output from ASDF operations.
Reminder: before ASDF can operate on a system, however,
it must be able to find and load that system's definition.
@xref{Configuring ASDF,,Configuring ASDF to find your systems}.
@findex already-loaded-systems

For the advanced users, note that
@code{require-system} calls @code{load-system}
with keyword arguments @code{:force-not (already-loaded-systems)}.
@code{already-loaded-systems} returns a list of the names of loaded systems.
@code{load-system} applies @code{operate} with the operation from
@code{*load-system-operation*}, which by default is @code{load-op},
the system, and any provided keyword arguments.
Load ASDF itself into your Lisp image, either through
@code{(require "asdf")} or else through
@code{(load "/path/to/asdf.lisp")}.
Make sure ASDF can find system definitions
thanks to proper source-registry configuration.
Load a system with @code{(asdf:load-system :my-system)}
or use some other operation on some system of your choice.
That's all you need to know to use ASDF to load systems written by others.
The rest of this manual deals with writing system definitions
for Common Lisp software you write yourself,
including how to extend ASDF to define new operation and component types.

@node Defining systems with defsystem, The object model of ASDF, Using ASDF, Top
@comment  node-name,  next,  previous,  up
@chapter Defining systems with defsystem

This chapter describes how to use asdf to define systems and develop
software.


@menu
* The defsystem form::
* A more involved example::
* The defsystem grammar::
* Other code in .asd files::
@end menu

@node  The defsystem form, A more involved example, Defining systems with defsystem, Defining systems with defsystem
@comment  node-name,  next,  previous,  up
@section The defsystem form

Systems can be constructed programmatically
by instantiating components using @code{make-instance}.
Most of the time, however, it is much more practical to use
a static @code{defsystem} form.
This section begins with an example of a system definition,
then gives the full grammar of @code{defsystem}.
Let's look at a simple system.
This is a complete file that would
usually be saved as @file{hello-lisp.asd}:
(defsystem "hello-lisp"
  :description "hello-lisp: a sample Lisp system."
  :author "Joe User <joe@@example.com>"
  :licence "Public Domain"
  :components ((:file "packages")
               (:file "macros" :depends-on ("packages"))
               (:file "hello" :depends-on ("macros"))))
@end lisp

Some notes about this example:

@itemize
The file starts with an @code{in-package} form
to use package @code{asdf}.
You could instead start your definition by using
a qualified name @code{asdf:defsystem}.
If in addition to simply using @code{defsystem},
you are going to define functions,
create ASDF extension, globally bind symbols, etc.,
it is recommended that to avoid namespace pollution between systems,
you should create your own package for that purpose,
for instance replacing the above @code{(in-package :asdf)} with:

@lisp
(defpackage :foo-system
  (:use :cl :asdf))

(in-package :foo-system)
@end lisp
The @code{defsystem} form defines a system named @code{hello-lisp}
that contains three source files:
@file{packages}, @file{macros} and @file{hello}.
@item
The file @file{macros} depends on @file{packages}
(presumably because the package it's in is defined in @file{packages}),
and the file @file{hello} depends on @file{macros}
(and hence, transitively on @file{packages}).
This means that ASDF will compile and load @file{packages} and @file{macros}
before starting the compilation of file @file{hello}.
The files are located in the same directory
as the file with the system definition.
ASDF resolves symbolic links (or Windows shortcuts)
before loading the system definition file and
stores its location in the resulting system@footnote{
It is possible, though almost never necessary, to override this behaviour.}.
This is a good thing because the user can move the system sources
without having to edit the system definition.
@c FIXME: Should have cross-reference to "Version specifiers" in the
@c defsystem grammar, but the cross-referencing is so broken by
@c insufficient node breakdown that I have not put one in.
@item
Make sure you know how the @code{:version} numbers will be parsed!
They are parsed as period-separated lists of integers.
I.e., in the example, @code{0.2.1} is to be interpreted,
roughly speaking, as @code{(0 2 1)}.
In particular, version @code{0.2.1}
is interpreted the same as @code{0.0002.1} and
is strictly version-less-than version @code{0.20.1},
even though the two are the same when interpreted as decimal fractions.
Instead of a string representing the version,
the @code{:version} argument can be an expression that is resolved to
such a string using the following trivial domain-specific language:
in addition to being a literal string, it can be an expression of the form
@code{(:read-file-form <pathname-or-string> :at <access-at-specifier>)},
which will be resolved by reading a form in the specified pathname
(read as a subpathname of the current system if relative or a unix-namestring).
You may use a @code{uiop:access-at} specifier
with the (optional) @code{:at} keyword,
by default the specifier is @code{0}, meaning the first form is returned.
@end itemize

@node  A more involved example, The defsystem grammar, The defsystem form, Defining systems with defsystem
@comment  node-name,  next,  previous,  up
@section A more involved example

Let's illustrate some more involved uses of @code{defsystem} via a
slightly convoluted example:

@lisp
(defsystem "foo"
  :components ((:module "mod"
                            :components ((:file "bar")
                                                  (:file"baz")
                                                  (:file "quux"))
                            :perform (compile-op :after (op c)
                                                  (do-something c))
                            :explain (compile-op :after (op c)
                                            (explain-something c)))
                         (:file "blah")))
The @code{:module} component named @code{"mod"} is a collection of three files,
which will be located in a subdirectory of the main code directory named
@file{mod} (this location can be overridden; see the discussion of the
@code{:pathname} option in @ref{The defsystem grammar}).

The method-form tokens provide a shorthand for defining methods on
particular components.  This part
                :perform (compile-op :after (op c)
                          (do-something c))
                :explain (compile-op :after (op c)
                          (explain-something c))
@end lisp

has the effect of

@lisp
(defmethod perform :after ((op compile-op) (c (eql ...)))
(defmethod explain :after ((op compile-op) (c (eql ...)))
           (explain-something c))
where @code{...} is the component in question.
Francois-Rene Rideau's avatar
Francois-Rene Rideau committed
In this case @code{...} would expand to something like
@end lisp

For more details on the syntax of such forms, see @ref{The defsystem
grammar}.
For more details on what these methods do, @pxref{Operations} in
@ref{The object model of ASDF}.

@c The following plunge into the weeds is not appropriate in this
@c location. [2010/10/03:rpg]
@c note that although this also supports @code{:before} methods,
@c they may not do what you want them to ---
@c a @code{:before} method on perform @code{((op compile-op) (c (eql ...)))}
@c will run after all the dependencies and sub-components have been processed,
@c but before the component in question has been compiled.
@node  The defsystem grammar, Other code in .asd files, A more involved example, Defining systems with defsystem
@comment  node-name,  next,  previous,  up
@section The defsystem grammar

@c FIXME: @var typesetting not consistently used here.  We should either expand
@c its use to everywhere, or we should kill it everywhere.

system-definition := ( defsystem system-designator @var{system-option}* )

system-option := :defsystem-depends-on system-list
                 | :weakly-depends-on @var{system-list}
                 | :class class-name (see discussion below)
                 | module-option
                 | option

module-option := :components component-list
                 | :serial [ t | nil ]
Francois-Rene Rideau's avatar
Francois-Rene Rideau committed
option :=
        | :default-component-class class-name
        | :perform method-form
        | :explain method-form
        | :operation-done-p method-form
        | :if-feature feature-expression
        | :depends-on ( @var{dependency-def}* )
        | :in-order-to ( @var{dependency}+ )
system-list := ( @var{simple-component-name}* )

component-list := ( @var{component-def}* )
component-def  := ( component-type simple-component-name @var{option}* )
component-type := :module | :file | :static-file | other-component-type
other-component-type := symbol-by-name (@pxref{The defsystem grammar,,Component types})
# This is used in :depends-on, as opposed to ``dependency,''
# which is used in :in-order-to
dependency-def := simple-component-name
               | ( :feature @var{feature-expression} dependency-def )
               | ( :version simple-component-name version-specifier )
# ``dependency'' is used in :in-order-to, as opposed to
dependency := (dependent-op @var{requirement}+)
requirement := (required-op @var{required-component}+)
dependent-op := operation-name
required-op := operation-name

simple-component-name := string
                      |  symbol
pathname-specifier := pathname | string | symbol

method-form := (operation-name qual lambda-list @Arest body)
qual := method qualifier

component-dep-fail-option := :fail | :try-next | :ignore

feature-expression := keyword | (:and @var{feature-expression}*)
                      | (:or @var{feature-expression}*) | (:not @var{feature-expression})
Component names (@code{simple-component-name})
may be either strings or symbols.

Component type names, even if expressed as keywords, will be looked up
by name in the current package and in the asdf package, if not found in
the current package.  So a component type @code{my-component-type}, in
the current package @code{my-system-asd} can be specified as
@code{:my-component-type}, or @code{my-component-type}.

@code{system} and its subclasses are @emph{not}
allowed as component types for such children components.

A system class name will be looked up
in the same way as a Component type (see above),
except that only @code{system} and its subclasses are allowed.
Typically, one will not need to specify a system
class name, unless using a non-standard system class defined in some
ASDF extension, typically loaded through @code{DEFSYSTEM-DEPENDS-ON},
see below.  For such class names in the ASDF package, we recommend that
the @code{:class} option be specified using a keyword symbol, such as

@example
:class :MY-NEW-SYSTEM-SUBCLASS
@end example

This practice will ensure that package name conflicts are avoided.
Otherwise, the symbol @code{MY-NEW-SYSTEM-SUBCLASS} will be read into
the current package @emph{before} it has been exported from the ASDF
extension loaded by @code{:defsystem-depends-on}, causing a name
conflict in the current package.

@subsection Defsystem depends on

The @code{:defsystem-depends-on} option to @code{defsystem} allows the
programmer to specify another ASDF-defined system or set of systems that
must be loaded @emph{before} the system definition is processed.
Typically this is used to load an ASDF extension that is used in the
system definition.

@subsection Weakly depends on
We do @emph{NOT} recommend you use this feature.
If you are tempted to write a system @var{foo}
that weakly-depends-on a system @var{bar},
we recommend that you should instead
write system @var{foo} in a parametric way,
and offer some special variable and/or some hook to specialize its behavior;
then you should write a system @var{foo+bar}
that does the hooking of things together.

The (deprecated) @code{:weakly-depends-on} option to @code{defsystem}
allows the programmer to specify another ASDF-defined system or set of systems
that ASDF should @emph{try} to load,
but need not load in order to be successful.
Typically this is used if there are a number of systems
that, if present, could provide additional functionality,
but which are not necessary for basic function.
Currently, although it is specified to be an option only to @code{defsystem},
this option is accepted at any component, but it probably
only makes sense at the @code{defsystem} level.
Programmers are cautioned not
to use this component option except at the @code{defsystem} level, as
this anomalous behavior may be removed without warning.

Finally, you might look into the @code{asdf-system-connections} extension,
that will let you define additional code to be loaded
when two systems are simultaneously loaded.
It may or may not be considered good style, but at least it can be used
in a way that has deterministic behavior independent of load order,
unlike @code{weakly-depends-on}.


@cindex pathname specifiers
A pathname specifier (@code{pathname-specifier})
may be a pathname, a string or a symbol.
When no pathname specifier is given for a component,
which is the usual case, the component name itself is used.

If a string is given, which is the usual case,
the string will be interpreted as a Unix-style pathname
where @code{/} characters will be interpreted as directory separators.
Usually, Unix-style relative pathnames are used
(i.e. not starting with @code{/}, as opposed to absolute pathnames);
they are relative to the path of the parent component.
Finally, depending on the @code{component-type},
the pathname may be interpreted as either a file or a directory,
and if it's a file,
a file type may be added corresponding to the @code{component-type},
or else it will be extracted from the string itself (if applicable).

For instance, the @code{component-type} @code{:module}
wants a directory pathname, and so a string @code{"foo/bar"}
will be interpreted as the pathname @file{#p"foo/bar/"}.
On the other hand, the @code{component-type} @code{:file}
wants a file of type @code{lisp}, and so a string @code{"foo/bar"}
will be interpreted as the pathname @file{#p"foo/bar.lisp"},
and a string @code{"foo/bar.quux"}
will be interpreted as the pathname @file{#p"foo/bar.quux.lisp"}.
Finally, the @code{component-type} @code{:static-file}
wants a file without specifying a type, and so a string @code{"foo/bar"}
will be interpreted as the pathname @file{#p"foo/bar"},
and a string @code{"foo/bar.quux"}
will be interpreted as the pathname @file{#p"foo/bar.quux"}.

ASDF does not interpret the string @code{".."} to designate the parent
directory.  This string will be passed through to the underlying
operating system for interpretation.  We @emph{believe} that this will
work on all platforms where ASDF is deployed, but do not guarantee this
behavior.  A pathname object with a relative directory component of
@code{:up} or @code{:back} is the only guaranteed way to specify a
parent directory.

If a symbol is given, it will be translated into a string,
and downcased in the process.
The downcasing of symbols is unconventional,
but was selected after some consideration.
Observations suggest that the type of systems we want to support
either have lowercase as customary case (Unix, Mac, windows)
or silently convert lowercase to uppercase (lpns),
so this makes more sense than attempting to use @code{:case :common}
as argument to @code{make-pathname},
which is reported not to work on some implementations.

Pathname objects may be given to override the path for a component.
Such objects are typically specified using reader macros such as @code{#p}
or @code{#.(make-pathname ...)}.
Note however, that @code{#p...} is
a shorthand for @code{#.(parse-namestring ...)}
and that the behavior of @code{parse-namestring} is completely non-portable,
unless you are using Common Lisp @code{logical-pathname}s,
which themselves involve other non-portable behavior
(@pxref{The defsystem grammar,,Using logical pathnames}, below).
Pathnames made with @code{#.(make-pathname ...)}
can usually be done more easily with the string syntax above.
The only case that you really need a pathname object is to override
the component-type default file type for a given component.
Therefore, pathname objects should only rarely be used.
Unhappily, ASDF 1 didn't properly support
parsing component names as strings specifying paths with directories,
and the cumbersome @code{#.(make-pathname ...)} syntax had to be used.
An alternative to @code{#.} read-time evaluation is to use
@code{(eval `(defsystem ... ,pathname ...))}.
Francois-Rene Rideau's avatar
Francois-Rene Rideau committed
Note that when specifying pathname objects,
ASDF does not do any special interpretation of the pathname
influenced by the component type, unlike the procedure for
pathname-specifying strings.
On the one hand, you have to be careful to provide a pathname that correctly
fulfills whatever constraints are required from that component type
(e.g. naming a directory or a file with appropriate type);
on the other hand, you can circumvent the file type that would otherwise
be forced upon you if you were specifying a string.
@subsection Version specifiers
@cindex version specifiers
@cindex :version

Version specifiers are strings to be parsed as period-separated lists of integers.
I.e., in the example, @code{"0.2.1"} is to be interpreted,
roughly speaking, as @code{(0 2 1)}.
In particular, version @code{"0.2.1"} is interpreted the same as @code{"0.0002.1"},
though the latter is not canonical and may lead to a warning being issued.
Also, @code{"1.3"} and @code{"1.4"} are both strictly @code{uiop:version<} to @code{"1.30"},
quite unlike what would have happened
had the version strings been interpreted as decimal fractions.

System definers are encouraged to use version identifiers of the form
@var{x}.@var{y}.@var{z} for
major version, minor version and patch level,
where significant API incompatibilities are signaled by an increased major number.
@subsection Using logical pathnames
Francois-Rene Rideau's avatar
Francois-Rene Rideau committed
@cindex logical pathnames
We do not generally recommend the use of logical pathnames,
especially not so to newcomers to Common Lisp.
However, we do support the use of logical pathnames by old timers,
when such is their preference.
To use logical pathnames,
you will have to provide a pathname object as a @code{:pathname} specifier
to components that use it, using such syntax as
@code{#p"LOGICAL-HOST:absolute;path;to;component.lisp"}.

You only have to specify such logical pathname
for your system or some top-level component.
Sub-components' relative pathnames,
specified using the string syntax for names,
will be properly merged with the pathnames of their parents.
The specification of a logical pathname host however is @emph{not}
otherwise directly supported in the ASDF syntax
for pathname specifiers as strings.
The @code{asdf-output-translation} layer will
avoid trying to resolve and translate logical pathnames.
The advantage of this is that
you can define yourself what translations you want to use
with the logical pathname facility.
The disadvantage is that if you do not define such translations,
any system that uses logical pathnames will behave differently under
asdf-output-translations than other systems you use.
If you wish to use logical pathnames you will have to configure the
translations yourself before they may be used.
ASDF currently provides no specific support
for defining logical pathname translations.
Note that the reasons we do not recommend logical pathnames are that
(1) there is no portable way to set up logical pathnames before they are used,
(2) logical pathnames are limited to only portably use
a single character case, digits and hyphens.
While you can solve the first issue on your own,
describing how to do it on each of fifteen implementations supported by ASDF
is more than we can document.
As for the second issue, mind that the limitation is notably enforced on SBCL,
and that you therefore can't portably violate the limitations
but must instead define some encoding of your own and add individual mappings
to name physical pathnames that do not fit the restrictions.
This can notably be a problem when your Lisp files are part of a larger project
in which it is common to name files or directories in a way that
includes the version numbers of supported protocols,
or in which files are shared with software written
in different programming languages where conventions include the use of
underscores, dots or CamelCase in pathnames.

@subsection Serial dependencies
@cindex serial dependencies
If the @code{:serial t} option is specified for a module,
ASDF will add dependencies for each child component,
on all the children textually preceding it.
This is done as if by @code{:depends-on}.
:components ((:file "a") (:file "b") (:file "c"))
:components ((:file "a")
             (:file "b" :depends-on ("a"))
             (:file "c" :depends-on ("a" "b")))
@end lisp


@subsection Source location

The @code{:pathname} option is optional in all cases for systems
defined via @code{defsystem},
and in the usual case the user is recommended not to supply it.
Instead, ASDF follows a hairy set of rules that are designed so that
Francois-Rene Rideau's avatar
Francois-Rene Rideau committed
@code{find-system}
will load a system from disk
and have its pathname default to the right place.
@item
This pathname information will not be overwritten with
@code{*default-pathname-defaults*}
(which could be somewhere else altogether)
if the user loads up the @file{.asd} file into his editor
and interactively re-evaluates that form.
If a system is being loaded for the first time,
its top-level pathname will be set to:
@item
The host/device/directory parts of @code{*load-truename*},
if it is bound.
@item
@code{*default-pathname-defaults*}, otherwise.
@end itemize

If a system is being redefined, the top-level pathname will be

@itemize
@item
changed, if explicitly supplied or obtained from @code{*load-truename*}
(so that an updated source location is reflected in the system definition)
changed if it had previously been set from @code{*default-pathname-defaults*}
left as before, if it had previously been set from @code{*load-truename*}
and @code{*load-truename*} is currently unbound
(so that a developer can evaluate a @code{defsystem} form
from within an editor without clobbering its source location)
@subsection if-feature option
@cindex :if-feature component option
@anchor{if-feature-option}      @c redo if this ever becomes a node in
@c its own right...

This option allows you to specify a feature expression to be evaluated
as if by @code{#+} to conditionally include a component in your build.
If the expression is false, the component is dropped
as well as any dependency pointing to it.
As compared to using @code{#+} which is expanded at read-time,
this allows you to have an object in your component hierarchy
that can be used for manipulations beside building your project, and
that is accessible to outside code that wishes to reason about system
structure.

This option was added in ASDF 3.  For more information,
@xref{required-features, Required features}.
@subsection if-component-dep-fails option
Its semantics was limited in purpose and dubious to explain,
and its implementation was breaking a hole into the ASDF object model.
Please use the @code{if-feature} option instead.
@subsection feature requirement
This requirement was removed in ASDF 3.1.
It used to ensure that a chain of component dependencies will raise an error,
which in conjunction with if-component-dep-fails would offer
a roundabout way to express conditional compilation.


@node Other code in .asd files, The package-system extension, The defsystem grammar, Defining systems with defsystem
Files containing @code{defsystem} forms
are regular Lisp files that are executed by @code{load}.
Consequently, you can put whatever Lisp code you like into these files.
However, it is recommended to keep such forms to a minimal,
and to instead define @code{defsystem} extensions
that you use with @code{:defsystem-depends-on}.

If however, you might insist on including code in the @code{.asd} file itself,
e.g., to examine and adjust the compile-time environment,
possibly adding appropriate features to @code{*features*}.
If so, here are some conventions we recommend you follow,
so that users can control certain details of execution
of the Lisp in @file{.asd} files:
Any informative output
(other than warnings and errors,
which are the condition system's to dispose of)
should be sent to the standard CL stream @code{*standard-output*},
so that users can easily control the disposition
of output from ASDF operations.
@node The package-system extension,  , Other code in .asd files, Defining systems with defsystem
@section The package-system extension

Starting with ASDF 3.0.3,
ASDF supports a one-package-per-file style of programming,
whereby each file is its own system,
and dependencies are deduced from the @code{defpackage} form.

In this style, packages referring to a same-named system (downcased);
and if a system is defined with @code{:class package-system},
then system names that start with that name
(using the slash @code{/} separator)
refer to files under the filesystem hierarchy where the system is defined.
For instance, if system @code{my-lib} is defined in
@file{/foo/bar/my-lib/my-lib.asd}, then system @code{my-lib/src/utility}
will be found in file @file{/foo/bar/my-lib/src/utility.lisp}.

This style was made popular by @code{faslpath} and @code{quick-build} before,
and at the cost of a stricter package discipline,
seems to make for more maintainable code.
It is used by ASDF itself (starting with ASDF 3) and by @code{lisp-interface-library}.

To use this style, choose a toplevel system name, e.g. @code{my-lib},
and create a file @file{my-lib.asd}
with the @code{:class :package-system} option in its @code{defsystem}.
For instance:
@example
#-asdf (error "my-lib requires ASDF 3")
(defsystem my-lib
  :class :package-system
  :defsystem-depends-on (:asdf-package-system)
  :depends-on (:my-lib/src/all)
  :in-order-to ((test-op (load-op :my-lib/test/all)))
  :perform (test-op (o c) (symbol-call :my-lib/test/all :test-suite)))
(register-system-packages :closer-mop
 '(:c2mop :closer-common-lisp :c2cl :closer-common-lisp-user :c2cl-user))
@end example

In the code above, the
@code{:defsystem-depends-on (:asdf-package-system)} is
for compatibility with older versions of ASDF 3 (ASDF 2 not supported),
and requires the @code{asdf-package-system} library to be present
(it is implicitly provided by ASDF starting with ASDF 3.0.3).

The function @code{register-system-packages} has to be called to register
packages used or provided by your system and its components
where the name of the system that provides the package
is not the downcase of the package name.

File @file{my-lib/src/utility.lisp} might start with:

@example
(defpackage :my-lib/src/utility
  (:use :closer-common-lisp :my-lib/src/macros :my-lib/src/variables)
  (:import-from :cl-ppcre #:register-groups-bind)
  (:export #:foo #:bar))
@end example

And from the @code{:use} and @code{:import-from} clauses,
ASDF could tell that you depend on systems @code{closer-mop} (registered above),
@code{cl-ppcre} (package and system names match), and
@code{my-lib/src/macros} and @code{my-lib/src/variables}
(package and system names match, and they will be looked up hierarchically).

The form @code{uiop:define-package} is supported as well as @code{defpackage},
and has many options that prove useful in this context,
such as @code{:use-reexport} and @code{:mix-reexport}
that allow for ``inheritance'' of symbols being exported.

@node The object model of ASDF, Controlling where ASDF searches for systems, Defining systems with defsystem, Top
@comment  node-name,  next,  previous,  up
@chapter The object model of ASDF

ASDF is designed in an object-oriented way from the ground up.
Both a system's structure and the operations that can be performed on systems
follow a extensible protocol.
This allows the addition of behaviours:
for example, @code{cffi} adds support of special FFI description files
to interface with C libraries and of wrapper files to embed C code in Lisp;
@code{abcl-jar} supports creating Java JAR archives in ABCL;
and @code{poiu} supports for compiling code in parallel using background processes.

This chapter deals with @code{component}s and @code{operation}s.
A @code{component} represents an individual source file or a group of source files,
and the things that get transformed into.
A @code{system} is a component at the top level of the component hierarchy.
A @code{source-file} is a component representing a single source-file
and the successive output files into which it is transformed.
A @code{module} is an intermediate component itself grouping several other components,
themselves source-files or further modules.

An @code{Operation} represents a transformation that can be performed on a component,
turning them from source files to intermediate results to final outputs.

A pair of an @code{operation} and a @code{component} is called an @code{action}.
An @code{action} represents a particular build step to be @code{perform}ed,
after all its dependencies have been fulfilled.
In the ASDF model, actions depend on other actions.
The term @emph{action} itself was used by Kent Pitman in his old article,
but was only used by ASDF hackers starting with the ASDF 2;
but the concept is ubiquitous since the very beginning of ASDF 1,
though previously implicit.

Then, there are many @emph{functions} available
to users, extenders and implementers of ASDF
to use, define or implement the activities
that are part of building your software.
most of these functions do not take as an argument
a reified pair (a @code{cons} cell) of an operation and a component;
instead, they usually take two separate arguments,
which allows to take advantage of the power CLOS-style multiple dispatch
for fun and profit.
There are many @emph{hooks} in which to add functionality,
by customizing the behavior of existing @emph{functions}.

Last but not least is the notion of @emph{dependency} between two actions.
The structure of dependencies between actions is
a directed @emph{dependency graph}.
ASDF is invoked by being told to @emph{operate}
with some @emph{operation} on some toplevel @emph{system};
it will then @emph{traverse} the graph and build a @emph{plan}
that follows its structure.
To be successfully buildable, this graph of actions but be acyclic.
If, as a user, extender or implementer of ASDF, you fail
to keep the dependency graph without cycles,
ASDF will fail loudly as it eventually finds one.
To clearly distinguish the direction of dependencies,
ASDF 3 uses the words @emph{requiring} and @emph{required}
as applied to an action depending on the other:
the requiring action @code{depends-on} the completion of all required actions
before it may itself be @code{perform}ed.
Using the @code{defsystem} syntax, users may easily express
direct dependencies along the graph of the object hierarchy:
between a component and its parent, its children, and its siblings.
By defining custom CLOS methods, you can express more elaborate dependencies as you wish.
Most common operations, such as @code{load-op}, @code{compile-op} or @code{load-source-op}
are automatically propagate ``downward'' the component hierarchy and are ``covariant'' with it:
to act the operation on the parent module, you must first act it on all the children components,
with the action on the parent being parent of the action on each child.
Other operations, such as @code{prepare-op} and @code{prepare-source-op}
(introduced in ASDF 3) are automatically propagated ``upward'' the component hierarchy
and are ``contravariant'' with it:
to perform the operation of preparing for compilation of a child component,
you must perform the operation of preparing for compilation of its parent component, and so on,
ensuring that all the parent's dependencies are (compiled and) loaded
before the child component may be compiled and loaded.
Yet other operations, such as @code{test-op} or @code{load-fasl-op}
remain at the system level, and are not propagated along the hierarchy,
but instead do something global on the system.
* Operations::
* Components::
@node  Operations, Components, The object model of ASDF, The object model of ASDF
@comment  node-name,  next,  previous,  up
@section Operations
@cindex operation

An @dfn{operation} object of the appropriate type is instantiated
whenever the user wants to do something with a system like

@itemize
@item compile all its files
@item load the files into a running lisp environment
@item copy its source files somewhere else
@end itemize

Operations can be invoked directly, or examined
to see what their effects would be without performing them.
There are a bunch of methods specialised on operation and component type
that actually do the grunt work.

The operation object contains whatever state is relevant for this purpose
(perhaps a list of visited nodes, for example)
but primarily is a nice thing to specialise operation methods on
and easier than having them all be @code{EQL} methods.

Operations are invoked on systems via @code{operate}.
@deffn {Generic function} @code{operate} @var{operation} @var{system} @Arest @var{initargs} @Akey @code{force} @code{force-not} @code{verbose} @AallowOtherKeys
@deffnx {Generic function} @code{oos} @var{operation} @var{system} @Arest @var{initargs} @Akey @AallowOtherKeys
@code{operate} invokes @var{operation} on @var{system}.
@code{oos} is a synonym for @code{operate}.

@var{operation} is a symbol that is passed, along with the supplied
@var{initargs}, to @code{make-instance} to create the operation object.
@var{system} is a system designator.

The @var{initargs} are passed to the @code{make-instance} call
when creating the operation object.
Note that dependencies may cause the operation
to invoke other operations on the system or its components:
the new operations will be created
with the same @var{initargs} as the original one.
If @var{force} is @code{:all}, then all systems
are forced to be recompiled even if not modified since last compilation.
If @var{force} is @code{t}, then only the system being loaded
is forced to be recompiled even if not modified since last compilation,
but other systems are not affected.
If @var{force} is a list, then it specifies a list of systems that
are forced to be recompiled even if not modified since last compilation.
If @var{force-not} is @code{:all}, then all systems
are forced not to be recompiled even if modified since last compilation.
If @var{force-not} is @code{t}, then only the system being loaded
is forced not to be recompiled even if modified since last compilation,
but other systems are not affected.
If @var{force-not} is a list, then it specifies a list of systems that
are forced not to be recompiled even if modified since last compilation.
@var{force} takes precedences over @var{force-not};
both of them apply to systems that are dependencies and were already compiled.

To see what @code{operate} would do, you can use:
@example
(asdf:traverse operation-class system-name)
@end example

* Predefined operations of ASDF::
* Creating new operations::
@node Predefined operations of ASDF, Creating new operations, Operations, Operations
@comment  node-name,  next,  previous,  up
@subsection Predefined operations of ASDF
All the operations described in this section are in the @code{asdf} package.
They are invoked via the @code{operate} generic function.
(asdf:operate 'asdf:@var{operation-name} :@var{system-name} @{@var{operation-options ...}@})
This operation compiles the specified component.
A @code{cl-source-file} will be @code{compile-file}'d.
All the children and dependencies of a system or module
will be recursively compiled by @code{compile-op}.

@code{compile-op} depends on @code{prepare-op} which
itself depends on a @code{load-op} of all of a component's dependencies,
as well as of its parent's dependencies.
When @code{operate} is called on @code{compile-op},
all these dependencies will be loaded as well as compiled;
yet, some parts of the system main remain unloaded,
because nothing depends on them.
Use @code{load-op} to load a system.
This operation loads the compiled code for a specified component.
A @code{cl-source-file} will have its compiled fasl @code{load}ed,
which fasl is the output of @code{compile-op} that @code{load-op} depends on.
All the children and dependencies of a system or module
will be recursively loaded by @code{load-op}.
@code{load-op} depends on @code{prepare-op} which
itself depends on a @code{load-op} of all of a component's dependencies,
as well as of its parent's dependencies.
This operation ensures that the dependencies of a component
and its recursive parents are loaded (as per @code{load-op}),
as a prerequisite before @code{compile-op} and @code{load-op} operations
may be performed on a given component.
@deffn Operation @code{load-source-op}, @code{prepare-source-op}

@code{load-source-op} will load the source for the files in a module
rather than they compiled fasl output.
It has a @code{prepare-source-op} analog to @code{prepare-op},
that ensures the dependencies are themselves loaded via @code{load-source-op}.

There is no provision in ASDF for ensuring that
some components are always loaded as source, while others are always compiled.
While this idea often comes up in discussions,
it actually doesn't play well with either the linking model of ECL
or with various bundle operations (see below), and is eventually not workable;
also the dependency model of ASDF would have to be modified incompatibly
to allow for such trick.
If your code doesn't compile cleanly, fix it.
If compilation makes it slow, use @code{declaim} or @code{eval-when}
to adjust your compiler settings,
or eschew compilation by @code{eval}uating a quoted source form at load-time.
@deffn Operation @code{test-op}

This operation will perform some tests on the module.
The default method will do nothing.
The default dependency is to require
@code{load-op} to be performed on the module first.
The default @code{operation-done-p} is that the operation is @emph{never} done
---
we assume that if you invoke the @code{test-op},
you want to test the system, even if you have already done so.

The results of this operation are not defined by ASDF.
It has proven difficult to define how the test operation
should signal its results to the user
in a way that is compatible with all of the various test libraries
and test techniques in use in the community.

People typically define @code{test-op} methods like thus:
@example
(defmethod perform ((o asdf:test-op)
                    (s (eql (asdf:find-system @var{:my-system}))))
  (asdf:load-system @var{:my-system-test})
  (funcall (read-from-string "my-system-test:test-suite")))
@end example

Using @code{load-system} in the perform method
rather than an @code{:in-order-to} dependency,
is sometimes necessary for backward compatibility with ASDF 2 and older,
to avoid circular dependencies that could arise
because of the way these old versions propagate dependencies.

If you don't care for compatibility with ASDF 2,
you could use the following options in your @code{defsystem} form:
@example
  :in-order-to ((test-op (load-op :my-system-test)))
  :perform (test-op (o c) (symbol-call :my-system-test :test-suite))
@end example
Robert P. Goldman's avatar
Robert P. Goldman committed
@end deffn

@deffn Operation @code{fasl-op}, @code{monolithic-fasl-op}, @code{load-fasl-op}, @code{binary-op}, @code{monolithic-binary-op}, @code{lib-op}, @code{monolithic-lib-op}, @code{dll-op}, @code{monolithic-dll-op}, @code{program-op}

These are ``bundle'' operations, that can create a single-file ``bundle''
for all the contents of each system in an application,
or for the entire application.

@code{fasl-op} will create a single fasl file for each of the systems needed,
grouping all its many fasls in one,
so you can deliver each system as a single fasl.
@code{monolithic-fasl-op} will create a single fasl file for target system
and all its dependencies,
so you can deliver your entire application as a single fasl.
@code{load-fasl-op} will load the output of @code{fasl-op}
(though if it the output is not up-to-date,
it will load the intermediate fasls indeed as part of building it);
this matters a lot on ECL, where the dynamic linking involved in loading
tens of individual fasls can be noticeably more expensive
than loading a single one.

Once you have created a fasl with @code{fasl-op},
you can use @code{precompiled-system} to deliver it in a way
that is compatible with clients having dependencies on your system,
whether it is distributed as source or as a single binary;
the @file{.asd} file to be delivered with the fasl will look like this:
@example
(defsystem :mysystem :class :precompiled-system
    :fasl (some expression that will evaluate to a pathname))
@end example
Or you can use @code{binary-op} to let ASDF create such a system for you
as well as the @code{fasl-op} output, or @code{monolithic-binary-op}.
This allows you to deliver code for your systems or applications
as a single file.
Of course, if you want to test the result in the current image,
@emph{before} you try to use any newly created @file{.asd} files,
you should not forget to @code{(asdf:clear-configuration)}
or at least @code{(asdf:clear-source-registry)},
so it re-populates the source-registry from the filesystem.

The @code{program-op} operation will create an executable program
from the specified system and its dependencies.
You can use UIOP for its pre-image-dump hooks, its post-image-restore hooks,
and its access to command-line arguments.
And you can specify an entry point @code{my-app:main}
by specifying in your @code{defsystem}
the option @code{:entry-point "my-app:main"}.
Depending on your implementation,
running @code{(asdf:operate 'asdf:program-op :my-app)}
may quit the current Lisp image upon completion.
See the example in
@file{test/hello-world-example.asd} and @file{test/hello.lisp},
as built and tested by
@file{test/test-program.script} and @file{test/make-hello-world.lisp}.

There is also @code{lib-op}
for building a linkable @file{.a} file (Windows: @file{.lib})
from all linkable object dependencies (FFI files, and on ECL, Lisp files too),
and its monolithic equivalent @code{monolithic-lib-op}.
And there is also @code{dll-op}
(respectively its monolithic equivalent @code{monolithic-lib-op})
for building a linkable @file{.so} file
(Windows: @file{.dll}, MacOS X: @file{.dynlib})
to create a single dynamic library
for all the extra FFI code to be linked into each of your systems
(respectively your entire application).

All these ``bundle'' operations are available since ASDF 3
on all actively supported Lisp implementations,
but may be unavailable on unmaintained legacy implementations.
This functionality was previously available for select implementations,
as part of a separate system @code{asdf-bundle},
itself descended from the ECL-only @code{asdf-ecl}.

The pathname of the output of bundle operations
is subject to output-translation as usual,
unless the operation is equal to
the @code{:build-operation} argument to @code{defsystem}.
This behavior is not very satisfactory and may change in the future.
Maybe you have suggestions on how to better configure it?
@deffn Operation @code{concatenate-source-op}, @code{monolithic-concatenate-source-op}, @code{load-concatenated-source-op}, @code{compile-concatenated-source-op}, @code{load-compiled-concatenated-source-op}, @code{monolithic-load-concatenated-source-op}, @code{monolithic-compile-concatenated-source-op}, @code{monolithic-load-compiled-concatenated-source-op}

These operation, as their respective names indicate,
consist in concatenating all @code{cl-source-file} source files in a system
(or in a system and all its dependencies, if monolithic),
in the order defined by dependencies,
then loading the result, or compiling then loading the result.

These operations are useful to deliver a system or application
as a single source file,
and for testing that said file loads properly, or compiles then loads properly.

ASDF itself is notably delivered as a single source file this way
using @code{monolithic-concatenate-source-op},
transcluding a prelude and the @code{uiop} library
before the @code{asdf/defsystem} system itself.
@end deffn

@node  Creating new operations,  , Predefined operations of ASDF, Operations
@comment  node-name,  next,  previous,  up
@subsection Creating new operations

ASDF was designed to be extensible in an object-oriented fashion.
To teach ASDF new tricks, a programmer can implement the behaviour he wants
by creating a subclass of @code{operation}.
ASDF's pre-defined operations are in no way ``privileged'',
but it is requested that developers never use the @code{asdf} package
for operations they develop themselves.
The rationale for this rule is that we don't want to establish a
``global asdf operation name registry'',
but also want to avoid name clashes.
Your operation @emph{must} usually provide methods
for one or more of the following generic functions:
Unless your operation, like @code{prepare-op},
is for dependency propagation only,
the most important function for which to define a method
is usually @code{perform},
which will be called to perform the operation on a specified component,
after all dependencies have been performed.

The @code{perform} method must call @code{output-files} (see below)
to find out where to put its files,
because the user is allowed to override the method
or tweak the output-translation mechanism.
Perform should only use the primary value returned by @code{output-files}.
If one and only one output file is expected,
it can call @code{output-file} that checks that this is the case
and returns the first and only list element.
@item @code{output-files}
If your perform method has any output,
you must define a method for this function.
for ASDF to determine where the outputs of performing operation lie.

Your method may return two values, a list of pathnames, and a boolean.
If the boolean is @code{nil} (or you fail to return multiple values),
then enclosing @code{:around} methods may translate these pathnames,
e.g. to ensure object files are somehow stored
in some implementation-dependent cache.
If the boolean is @code{t} then the pathnames are marked
not be translated by the enclosing @code{:around} method.
@item @code{component-depends-on}
If the action of performing the operation on a component has dependencies,
you must define a method on @code{component-depends-on}.

Your method will take as specialized arguments
an operation and a component which together identify an action,
and return a list of entries describing actions that this action depends on.
The format of entries is described below.

It is @emph{strongly} advised that
you should always append the results of @code{(call-next-method)}
to the results of your method,
or ``interesting'' failures will likely occur,
unless you're a true specialist of ASDF internals.
It is unhappily too late to compatibly use the @code{append} method combination,
but conceptually that's the protocol that is being manually implemented.

Each entry returned by @code{component-depends-on} is itself a list.

The first element of an entry is an operation designator:
either an operation object designating itself, or
a symbol that names an operation class
(that ASDF will instantiate using @code{make-operation}).
For instance, @code{load-op}, @code{compile-op} and @code{prepare-op}
are common such names, denoting the respective operations.

The rest of each entry is a list of component designators:
either a component object designating itself,
or an identifier to be used with @code{find-component}.
@code{find-component} will be called with the current component's parent as parent,
and the identifier as second argument.
The identifier is typically a string,
a symbol (to be downcased as per @code{coerce-name}),
or a list of strings or symbols.
In particular, the empty list @code{nil} denotes the parent itself.

@end itemize

An operation @emph{may} provide methods for the following generic functions:

@itemize

@item @code{input-files}
A method for this function is often not needed,
since ASDF has a pretty clever default @code{input-files} mechanism.
You only need create a method if there are multiple ultimate input files,
and/or the bottom one doesn't depend
on the @code{component-pathname} of the component.

@item @code{operation-done-p}
You only need to define a method on that function
if you can detect conditions that invalidate previous runs of the operation,
even though no filesystem timestamp has changed,
in which case you return @code{nil} (the default is @code{t}).
For instance, the method for @code{test-op} always returns @code{nil},
so that tests are always run afresh.
Of course, the @code{test-op} for your system could depend
on a deterministically repeatable @code{test-report-op},
and just read the results from the report files,
in which case you could have this method return @code{t}.
Operations that print output should send that output to the standard
CL stream @code{*standard-output*}, as the Lisp compiler and loader do.

@node Components, Functions, Operations, The object model of ASDF
@comment  node-name,  next,  previous,  up
@section Components
@cindex component
@cindex system
@cindex system designator
@vindex *system-definition-search-functions*

A @dfn{component} represents a source file or
(recursively) a collection of components.
A @dfn{system} is (roughly speaking) a top-level component
that can be found via @code{find-system}.
A @dfn{system designator} is a system itself,
or a string or symbol that behaves just like any other component name
(including with regard to the case conversion rules for component names).
A @dfn{component designator}, relative to a base component,
is either a component itself,
or a string or symbol,
or a list of designators.
@defun find-system system-designator @Aoptional (error-p t)
Given a system designator, @code{find-system} finds and returns a system.
If no system is found, an error of type
@code{missing-component} is thrown,
or @code{nil} is returned if @code{error-p} is false.

To find and update systems, @code{find-system} funcalls each element
in the @code{*system-definition-search-functions*} list,
expecting a pathname to be returned, or a system object,
from which a pathname may be extracted, and that will be registered.
The resulting pathname (if any) is loaded
if one of the following conditions is true:
@item
there is no system of that name in memory
@item
the pathname is different from that which was previously loaded
@item
the file's @code{last-modified} time exceeds the @code{last-modified} time
of the system in memory
When system definitions are loaded from @file{.asd} files,
a new scratch package is created for them to load into,
so that different systems do not overwrite each others operations.
The user may also wish to (and is recommended to)
include @code{defpackage} and @code{in-package} forms
in his system definition files, however,
so that they can be loaded manually if need be.

The default value of @code{*system-definition-search-functions*}
is a list of two functions.
The first function looks in each of the directories given
by evaluating members of @code{*central-registry*}
for a file whose name is the name of the system and whose type is @file{asd}.
The first such file is returned,
whether or not it turns out to actually define the appropriate system.
The second function does something similar,
for the directories specified in the @code{source-registry}.
Hence, it is strongly advised to define a system
@var{foo} in the corresponding file @var{foo.asd}.
@end defun

@defun find-component base path

Given a @var{base} component (or designator for such),
and a @var{path}, find the component designated by the @var{path}
starting from the @var{base}.

If @var{path} is a component object, it designates itself,
independently from the base.

If @var{path} is a string, or symbol denoting a string via @code{coerce-name},
then @var{base} is resolved to a component object,
which must be a system or module,
and the designated component is the child named by the @var{path}.

If @var{path} is a @code{cons} cell,
@code{find-component} with the base and the @code{car} of the @var{path},
and the resulting object is used as the base for a tail call
to @code{find-component} with the @code{car} of the @var{path}.

If @var{base} is a component object, it designates itself.

If @var{base} is null, then @var{path} is used as the base, with @code{nil} as the path.

If @var{base} is a string, or symbol denoting a string via @code{coerce-name},
it designates a system as per @code{find-system}.

If @var{base} is a @code{cons} cell, it designates the component found by
@code{find-component} with its @code{car} as base and @code{cdr} as path.
@end defun

* Common attributes of components::
* Pre-defined subclasses of component::
* Creating new component types::
@end menu

@node  Common attributes of components, Pre-defined subclasses of component, Components, Components
@comment  node-name,  next,  previous,  up
@subsection Common attributes of components

All components, regardless of type, have the following attributes.
All attributes except @code{name} are optional.

@subsubsection Name

A component name is a string or a symbol.
If a symbol, its name is taken and lowercased.
Unless overridden by a @code{:pathname} attribute,
the name will be interpreted as a pathname specifier according
to a Unix-style syntax.
@xref{The defsystem grammar,,Pathname specifiers}.

@subsubsection Version identifier
@findex version-satisfies
@cindex :version
This optional attribute specifies a version for the current component.
The version should typically be a string of integers separated by dots,
for example @samp{1.0.11}.
For more information on version specifiers, see @ref{The defsystem grammar}.
A version may then be queried by the generic function @code{version-satisfies},
to see if @code{:version} dependencies are satisfied,
and when specifying dependencies, a constraint of minimal version to satisfy
can be specified using e.g. @code{(:version "mydepname" "1.0.11")}.
Note that in the wild, we typically see version numbering
only on components of type @code{system}.
Presumably it is much less useful within a given system,
wherein the library author is responsible to keep the various files in synch.
@subsubsection Required features
Traditionally defsystem users have used @code{#+} reader conditionals
to include or exclude specific per-implementation files.
For example, CFFI, the portable C foreign function interface contained
lines like:
@lisp
     #+sbcl       (:file "cffi-sbcl")
@end lisp
An unfortunate side effect of this approach is that no single
implementation can read the entire system.
This causes problems if, for example, one wished to design an @code{archive-op}
that would create an archive file containing all the sources, since
for example the file @code{cffi-sbcl.lisp} above would be invisible when
running the @code{archive-op} on any implementation other than SBCL.

Starting with ASDF 3,
components may therefore have an @code{:if-feature} option.
The value of this option should be
a feature expression using the same syntax as @code{#+} does.
If that feature expression evaluates to false, any reference to the component will be ignored
during compilation, loading and/or linking.
Since the expression is read by the normal reader,
you must explicitly prefix your symbols with @code{:} so they be read as keywords;
this is as contrasted with the @code{#+} syntax
that implicitly reads symbols in the keyword package by default.

For instance, @code{:if-feature (:and :x86 (:or :sbcl :cmu :scl))} specifies that
the given component is only to be compiled and loaded
when the implementation is SBCL, CMUCL or Scieneer CL on an x86 machine.
You cannot write it as @code{:if-feature (and x86 (or sbcl cmu scl))}
since the symbols would not be read as keywords.

@xref{if-feature-option}.

@subsubsection Dependencies

This attribute specifies dependencies of the component on its siblings.
It is optional but often necessary.

There is an excitingly complicated relationship between the initarg
and the method that you use to ask about dependencies

Dependencies are between (operation component) pairs.
In your initargs for the component, you can say

@lisp
:in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
              (load-op (load-op "foo")))
@end lisp

This means the following things:
@itemize
@item
before performing compile-op on this component, we must perform
load-op on @var{a} and @var{b}, and compile-op on @var{c},
@item
before performing @code{load-op}, we have to load @var{foo}
@end itemize

The syntax is approximately

@verbatim
(this-op @{(other-op required-components)@}+)
simple-component-name := string
                      |  symbol

required-components := simple-component-name
                     | (required-components required-components)

component-name := simple-component-name
                | (:version simple-component-name minimum-version-object)
This is on a par with what ACL defsystem does.
mk-defsystem is less general: it has an implied dependency
  for all source file x, (load x) depends on (compile x)
@end verbatim

and using a @code{:depends-on} argument to say that @var{b} depends on
@var{a} @emph{actually} means that

@verbatim
  (compile b) depends on (load a)
@end verbatim

This is insufficient for e.g. the McCLIM system, which requires that
all the files are loaded before any of them can be compiled ]

End side note
Loading
Loading full blame...