Skip to content
Snippets Groups Projects
asdf.texinfo 114 KiB
Newer Older
@chapter The object model of ASDF

ASDF is designed in an object-oriented way from the ground up.
Both a system's structure and the operations that can be performed on systems
follow a protocol.
ASDF is extensible to new operations and to new component types.
This allows the addition of behaviours:
for example, a new component could be added for Java JAR archives,
and methods specialised on @code{compile-op} added for it
that would accomplish the relevant actions.

This chapter deals with @emph{components}, the building blocks of a system,
and @emph{operations}, the actions that can be performed on a system.
* Operations::
* Components::
@node  Operations, Components, The object model of ASDF, The object model of ASDF
@comment  node-name,  next,  previous,  up
@section Operations
@cindex operation

An @dfn{operation} object of the appropriate type is instantiated
whenever the user wants to do something with a system like

@itemize
@item compile all its files
@item load the files into a running lisp environment
@item copy its source files somewhere else
@end itemize

Operations can be invoked directly, or examined
to see what their effects would be without performing them.
@emph{FIXME: document how!}
There are a bunch of methods specialised on operation and component type
that actually do the grunt work.

The operation object contains whatever state is relevant for this purpose
(perhaps a list of visited nodes, for example)
but primarily is a nice thing to specialise operation methods on
and easier than having them all be @code{EQL} methods.

Operations are invoked on systems via @code{operate}.
@deffn {Generic function} @code{operate} @var{operation} @var{system} @&rest @var{initargs}
@deffnx {Generic function} @code{oos} @var{operation} @var{system} @&rest @var{initargs}
@code{operate} invokes @var{operation} on @var{system}.
@code{oos} is a synonym for @code{operate}.

@var{operation} is a symbol that is passed, along with the supplied
@var{initargs}, to @code{make-instance} to create the operation object.
@var{system} is a system designator.

The @var{initargs} are passed to the @code{make-instance} call
when creating the operation object.
Note that dependencies may cause the operation
to invoke other operations on the system or its components:
the new operations will be created
with the same @var{initargs} as the original one.
* Predefined operations of ASDF::
* Creating new operations::
@node Predefined operations of ASDF, Creating new operations, Operations, Operations
@comment  node-name,  next,  previous,  up
@subsection Predefined operations of ASDF
All the operations described in this section are in the @code{asdf} package.
They are invoked via the @code{operate} generic function.
(asdf:operate 'asdf:@var{operation-name} :@var{system-name} @{@var{operation-options ...}@})
@deffn Operation @code{compile-op} @&key @code{proclamations}
This operation compiles the specified component.
If proclamations are supplied, they will be proclaimed.
This is a good place to specify optimization settings.
When creating a new component type,
you should provide methods for @code{compile-op}.
When @code{compile-op} is invoked,
component dependencies often cause some parts of the system
to be loaded as well as compiled.
Invoking @code{compile-op}
does not necessarily load all the parts of the system, though;
use @code{load-op} to load a system.
@deffn Operation @code{load-op} @&key @code{proclamations}

This operation loads a system.

The default methods for @code{load-op} compile files before loading them.
For parity, your own methods on new component types should probably do so too.
@deffn Operation @code{load-source-op}
This operation will load the source for the files in a module
even if the source files have been compiled.
Systems sometimes have knotty dependencies
which require that sources are loaded
before they can be compiled.
This is how you do that.
If you are creating a component type,
you need to implement this operation --- at least, where meaningful.
@deffn Operation @code{test-op}

This operation will perform some tests on the module.
The default method will do nothing.
The default dependency is to require
@code{load-op} to be performed on the module first.
The default @code{operation-done-p} is that the operation is @emph{never} done
---
we assume that if you invoke the @code{test-op},
you want to test the system, even if you have already done so.

The results of this operation are not defined by ASDF.
It has proven difficult to define how the test operation
should signal its results to the user
in a way that is compatible with all of the various test libraries
and test techniques in use in the community.
Robert P. Goldman's avatar
Robert P. Goldman committed
@end deffn

Francois-Rene Rideau's avatar
Francois-Rene Rideau committed
@c @deffn Operation test-system-version @&key minimum

@c Asks the system whether it satisfies a version requirement.

@c The default method accepts a string, which is expected to contain of a
@c number of integers separated by #\. characters.  The method is not
@c recursive.  The component satisfies the version dependency if it has
@c the same major number as required and each of its sub-versions is
@c greater than or equal to the sub-version number required.

@c @lisp
@c (defun version-satisfies (x y)
@c   (labels ((bigger (x y)
@c           (cond ((not y) t)
@c                 ((not x) nil)
@c                 ((> (car x) (car y)) t)
@c                 ((= (car x) (car y))
@c                  (bigger (cdr x) (cdr y))))))
@c     (and (= (car x) (car y))
@c       (or (not (cdr y)) (bigger (cdr x) (cdr y))))))
@c @end lisp

@c If that doesn't work for your system, you can override it.  I hope
@c you have as much fun writing the new method as @verb{|#lisp|} did
@c reimplementing this one.
@c @end deffn

@c @deffn Operation feature-dependent-op

@c An instance of @code{feature-dependent-op} will ignore any components
@c which have a @code{features} attribute, unless the feature combination
@c it designates is satisfied by @code{*features*}.  This operation is
@c not intended to be instantiated directly, but other operations may
@c inherit from it.

@c @end deffn
@node  Creating new operations,  , Predefined operations of ASDF, Operations
@comment  node-name,  next,  previous,  up
@subsection Creating new operations

ASDF was designed to be extensible in an object-oriented fashion.
To teach ASDF new tricks, a programmer can implement the behaviour he wants
by creating a subclass of @code{operation}.
ASDF's pre-defined operations are in no way ``privileged'',
but it is requested that developers never use the @code{asdf} package
for operations they develop themselves.
The rationale for this rule is that we don't want to establish a
``global asdf operation name registry'',
but also want to avoid name clashes.

An operation must provide methods for the following generic functions
when invoked with an object of type @code{source-file}:
@emph{FIXME describe this better}

@itemize

@item @code{output-files}
The @code{output-files} method determines where the method will put its files.
It returns two values, a list of pathnames, and a boolean.
If the boolean is @code{T} then the pathnames are marked
not be translated by enclosing @code{:around} methods.
If the boolean is @code{NIL} then enclosing @code{:around} methods
may translate these pathnames, e.g. to ensure object files
are somehow stored in some implementation-dependent cache.
@item @code{perform}
The @code{perform} method must call @code{output-files}
to find out where to put its files,
because the user is allowed to override.
@item @code{output-files}
for local policy @code{explain}
@item @code{operation-done-p},
if you don't like the default one
Operations that print output should send that output to the standard
CL stream @code{*standard-output*}, as the Lisp compiler and loader do.

@node Components,  , Operations, The object model of ASDF
@comment  node-name,  next,  previous,  up
@section Components
@cindex component
@cindex system
@cindex system designator
@vindex *system-definition-search-functions*

A @dfn{component} represents a source file or
(recursively) a collection of components.
A @dfn{system} is (roughly speaking) a top-level component
that can be found via @code{find-system}.
A @dfn{system designator} is a string or symbol
and behaves just like any other component name
(including with regard to the case conversion rules for component names).


@defun find-system system-designator &optional (error-p t)

Given a system designator, @code{find-system} finds and returns a system.
If no system is found, an error of type
@code{missing-component} is thrown,
or @code{nil} is returned if @code{error-p} is false.

To find and update systems, @code{find-system} funcalls each element
in the @code{*system-definition-search-functions*} list,
expecting a pathname to be returned.
The resulting pathname is loaded if either of the following conditions is true:
@item
there is no system of that name in memory
@item
the file's @code{last-modified} time exceeds the @code{last-modified} time
of the system in memory
When system definitions are loaded from @file{.asd} files,
a new scratch package is created for them to load into,
so that different systems do not overwrite each others operations.
The user may also wish to (and is recommended to)
include @code{defpackage} and @code{in-package} forms
in his system definition files, however,
so that they can be loaded manually if need be.

The default value of @code{*system-definition-search-functions*}
is a list of two functions.
The first function looks in each of the directories given
by evaluating members of @code{*central-registry*}
for a file whose name is the name of the system and whose type is @file{asd}.
The first such file is returned,
whether or not it turns out to actually define the appropriate system.
The second function does something similar,
for the directories specified in the @code{source-registry}.
Hence, it is strongly advised to define a system
@var{foo} in the corresponding file @var{foo.asd}.
@end defun


@menu
* Common attributes of components::
* Pre-defined subclasses of component::
* Creating new component types::
@end menu

@node  Common attributes of components, Pre-defined subclasses of component, Components, Components
@comment  node-name,  next,  previous,  up
@subsection Common attributes of components

All components, regardless of type, have the following attributes.
All attributes except @code{name} are optional.

@subsubsection Name

A component name is a string or a symbol.
If a symbol, its name is taken and lowercased.
Unless overridden by a @code{:pathname} attribute,
the name will be interpreted as a pathname specifier according
to a Unix-style syntax.
@xref{The defsystem grammar,,Pathname specifiers}.

@subsubsection Version identifier

This optional attribute is used by the @code{test-system-version} operation.
@xref{Predefined operations of ASDF}.
For the default method of @code{test-system-version},
the version should be a string of integers separated by dots,
for example @samp{1.0.11}.
@emph{Nota Bene}:
This operation, planned for ASDF 1,
is still not implement yet as of ASDF 2.
Don't hold your breath.


@subsubsection Required features

@emph{FIXME: This subsection seems to contradict the
@code{defsystem} grammar subsection,
which doesn't provide any obvious way to specify required features.
Furthermore, in 2009, discussions on the
@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
suggested that the specification of required features may be broken,
and that no one may have been using them for a while.
Please contact the
@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
if you are interested in getting this features feature fixed.}

Traditionally defsystem users have used reader conditionals
to include or exclude specific per-implementation files.
This means that any single implementation cannot read the entire system,
which becomes a problem if it doesn't wish to compile it,
but instead for example to create an archive file containing all the sources,
as it will omit to process the system-dependent sources for other systems.

Each component in an asdf system may therefore specify features using
the same syntax as @code{#+} does, and it will (somehow) be ignored for
certain operations unless the feature conditional is a member of
@code{*features*}.


@subsubsection Dependencies

This attribute specifies dependencies of the component on its siblings.
It is optional but often necessary.

There is an excitingly complicated relationship between the initarg
and the method that you use to ask about dependencies

Dependencies are between (operation component) pairs.
In your initargs for the component, you can say

@lisp
:in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
              (load-op (load-op "foo")))
@end lisp

This means the following things:
@itemize
@item
before performing compile-op on this component, we must perform
load-op on @var{a} and @var{b}, and compile-op on @var{c},
@item
before performing @code{load-op}, we have to load @var{foo}
@end itemize

The syntax is approximately

@verbatim
(this-op {(other-op required-components)}+)

required-components := component-name
                     | (required-components required-components)

component-name := string
                | (:version string minimum-version-object)
@end verbatim

Side note:

This is on a par with what ACL defsystem does.
mk-defsystem is less general: it has an implied dependency

@verbatim
  for all x, (load x) depends on (compile x)
@end verbatim

and using a @code{:depends-on} argument to say that @var{b} depends on
@var{a} @emph{actually} means that

@verbatim
  (compile b) depends on (load a)
@end verbatim

This is insufficient for e.g. the McCLIM system, which requires that
all the files are loaded before any of them can be compiled ]

End side note

In ASDF, the dependency information for a given component and operation
can be queried using @code{(component-depends-on operation component)},
which returns a list

@lisp
((load-op "a") (load-op "b") (compile-op "c") ...)
@end lisp

@code{component-depends-on} can be subclassed for more specific
component/operation types: these need to @code{(call-next-method)}
and append the answer to their dependency, unless
they have a good reason for completely overriding the default dependencies.
Robert P. Goldman's avatar
Robert P. Goldman committed
If it weren't for CLISP, we'd be using @code{LIST} method
combination to do this transparently.
But, we need to support CLISP.
If you have the time for some CLISP hacking,
I'm sure they'd welcome your fixes.
@c Doesn't CLISP now support LIST method combination?
This attribute is optional and if absent (which is the usual case),
the component name will be used.
@xref{The defsystem grammar,,Pathname specifiers},
for an explanation of how this attribute is interpreted.
Note that the @code{defsystem} macro (used to create a ``top-level'' system)
does additional processing to set the filesystem location of
the top component in that system.
This is detailed elsewhere. @xref{Defining systems with defsystem}.
The answer to the frequently asked question
``how do I create a system definition
where all the source files have a @file{.cl} extension''
is thus

@lisp
(defmethod source-file-type ((c cl-source-file) (s (eql (find-system 'my-sys))))
@end lisp

@subsubsection properties

This attribute is optional.

Packaging systems often require information about files or systems
in addition to that specified by ASDF's pre-defined component attributes.
Programs that create vendor packages out of ASDF systems therefore
have to create ``placeholder'' information to satisfy these systems.
Sometimes the creator of an ASDF system may know the additional
information and wish to provide it directly.

@code{(component-property component property-name)} and
associated @code{setf} method will allow
the programmatic update of this information.
Property names are compared as if by @code{EQL},
so use symbols or keywords or something.
* Pre-defined subclasses of component::
* Creating new component types::
@end menu

@node Pre-defined subclasses of component, Creating new component types, Common attributes of components, Components
@comment  node-name,  next,  previous,  up
@subsection Pre-defined subclasses of component

@deffn Component source-file

A source file is any file that the system does not know how to
generate from other components of the system.
Note that this is not necessarily the same thing as
``a file containing data that is typically fed to a compiler''.
If a file is generated by some pre-processor stage
(e.g. a @file{.h} file from @file{.h.in} by autoconf)
then it is not, by this definition, a source file.
Conversely, we might have a graphic file
that cannot be automatically regenerated,
or a proprietary shared library that we received as a binary:
these do count as source files for our purposes.

Subclasses of source-file exist for various languages.
@emph{FIXME: describe these.}
@end deffn

@deffn Component module

A module is a collection of sub-components.

A module component has the following extra initargs:

@itemize
@item
@code{:components} the components contained in this module

@item
@code{:default-component-class}
All children components which don't specify their class explicitly
are inferred to be of this type.
@code{:if-component-dep-fails}
This attribute takes one of the values
@code{:fail}, @code{:try-next}, @code{:ignore},
its default value is @code{:fail}.
The other values can be used for implementing conditional compilation
based on implementation @code{*features*},
for the case where it is not necessary for all files in a module to be
@emph{FIXME: such conditional compilation has been reported
to be broken in 2009.}
@code{:serial} When this attribute is set,
each subcomponent of this component is assumed to depend on all subcomponents
before it in the list given to @code{:components}, i.e.
all of them are loaded before a compile or load operation is performed on it.
The default operation knows how to traverse a module, so
most operations will not need to provide methods specialised on modules.

@code{module} may be subclassed to represent components such as
foreign-language linked libraries or archive files.
@end deffn

@deffn Component system

@code{system} is a subclass of @code{module}.

A system is a module with a few extra attributes for documentation
purposes; these are given elsewhere.
@xref{The defsystem grammar}.
Users can create new classes for their systems:
the default @code{defsystem} macro takes a @code{:class} keyword argument.
@end deffn

@node  Creating new component types,  , Pre-defined subclasses of component, Components
@comment  node-name,  next,  previous,  up
@subsection Creating new component types

New component types are defined by subclassing one of the existing
component classes and specializing methods on the new component class.

@emph{FIXME: this should perhaps be explained more throughly,
not only by example ...}

As an example, suppose we have some implementation-dependent
functionality that we want to isolate
in one subdirectory per Lisp implementation our system supports.
We create a subclass of
@code{cl-source-file}:

@lisp
(defclass unportable-cl-source-file (cl-source-file)
A hypothetical function @code{system-dependent-dirname}
gives us the name of the subdirectory.
All that's left is to define how to calculate the pathname
of an @code{unportable-cl-source-file}.

@lisp
(defmethod component-pathname ((component unportable-cl-source-file))
  (let ((pathname (call-next-method))
        (name (string-downcase (system-dependent-dirname))))
     (make-pathname :directory (list :relative name))
     pathname)))
@end lisp

The new component type is used in a @code{defsystem} form in this way:

@lisp
(defsystem :foo
    :components
    ((:file "packages")
     ...
     (:unportable-cl-source-file "threads"
      :depends-on ("packages" ...))
     ...
    )
@end lisp

@node Controlling where ASDF searches for systems, Controlling where ASDF saves compiled files, The object model of ASDF, Top
@comment  node-name,  next,  previous,  up
@chapter Controlling where ASDF searches for systems

@section Configurations

Configurations specify paths where to find system files.

@enumerate

@item
The search registry may use some hardcoded wrapping registry specification.
This allows some implementations (notably SBCL) to specify where to find
some special implementation-provided systems that
need to precisely match the version of the implementation itself.
@item
An application may explicitly initialize the source-registry configuration
using the configuration API
(@pxref{Controlling where ASDF searches for systems,Configuration API,Configuration API}, below)
in which case this takes precedence.
It may itself compute this configuration from the command-line,
from a script, from its own configuration file, etc.
@item
The source registry will be configured from
the environment variable @code{CL_SOURCE_REGISTRY} if it exists.
@item
The source registry will be configured from
user configuration file
@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf}
(which defaults to
@file{~/.config/common-lisp/source-registry.conf})
if it exists.
@item
The source registry will be configured from
user configuration directory
@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf.d/}
(which defaults to
@file{~/.config/common-lisp/source-registry.conf.d/})
if it exists.
@item
The source registry will be configured from
system configuration file
@file{/etc/common-lisp/source-registry.conf}
if it exists/
@item
The source registry will be configured from
system configuration directory
@file{/etc/common-lisp/source-registry.conf.d/}
if it exists.

Robert P. Goldman's avatar
Robert P. Goldman committed
@item
The source registry will be configured from a default configuration.
This configuration may allow for implementation-specific systems
to be found, for systems to be found the current directory
(at the time that the configuration is initialized) as well as
@code{:directory} entries for @file{$XDG_DATA_DIRS/common-lisp/systems/} and
@code{:tree} entries for @file{$XDG_DATA_DIRS/common-lisp/source/}.

@end enumerate

Each of these configuration is specified as a SEXP
in a trival domain-specific language (defined below).
Additionally, a more shell-friendly syntax is available
for the environment variable (defined yet below).

Each of these configurations is only used if the previous
configuration explicitly or implicitly specifies that it
includes its inherited configuration.

Additionally, some implementation-specific directories
may be automatically prepended to whatever directories are specified
in configuration files, no matter if the last one inherits or not.

@section XDG base directory

Note that we purport to respect the XDG base directory specification
as to where configuration files are located,
where data files are located,
where output file caches are located.
Mentions of XDG variables refer to that document.

@url{http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html}

This specification allows the user to specify some environment variables
to customize how applications behave to his preferences.

On Windows platforms, when not using Cygwin,
instead of the XDG base directory specification,
we try to use folder configuration from the registry regarding
@code{Common AppData} and similar directories.
However, support querying the Windows registry is limited as of ASDF 2,
and on many implementations, we may fall back to always using the defaults
without consulting the registry.
Patches welcome.
For backward compatibility as well as for a practical backdoor for hackers,
ASDF will first search for @code{.asd} files in the directories specified in
@code{asdf:*central-registry*}
before it searches in the source registry above.
Robert P. Goldman's avatar
Robert P. Goldman committed
@xref{Configuring ASDF,,Configuring ASDF to find your systems -- old style}.
By default, @code{asdf:*central-registry*} will be empty.

This old mechanism will therefore not affect you if you don't use it,
but will take precedence over the new mechanism if you do use it.

@section Configuration DSL

Here is the grammar of the SEXP DSL for source-registry configuration:

@example
;; A configuration is single SEXP starting with keyword :source-registry
;; followed by a list of directives.
CONFIGURATION := (:source-registry DIRECTIVE ...)

;; A directive is one of the following:
DIRECTIVE :=
    ;; add a single directory to be scanned (no recursion)
    (:directory DIRECTORY-PATHNAME-DESIGNATOR) |

    ;; add a directory hierarchy, recursing but excluding specified patterns
    (:tree DIRECTORY-PATHNAME-DESIGNATOR) |

    ;; override the defaults for exclusion patterns
    ;; augment the defaults for exclusion patterns
    (:also-exclude PATTERN ...) |

    ;; splice the parsed contents of another config file
    (:include REGULAR-FILE-PATHNAME-DESIGNATOR) |

    ;; Your configuration expression MUST contain
    ;; exactly one of either of these:
    :inherit-configuration | ; splices contents of inherited configuration
    :ignore-inherited-configuration | ; drop contents of inherited configuration

    ;; This directive specifies that some default must be spliced.
    :default-registry

PATTERN := a string without wildcards, that will be matched exactly
	against the name of a any subdirectory in the directory component
Francois-Rene Rideau's avatar
Francois-Rene Rideau committed
        of a path. e.g. @code{"_darcs"} will match @file{#p"/foo/bar/_darcs/src/bar.asd"}
@end example


@section Configuration Directories

Configuration directories consist in files each contains
a list of directives without any enclosing @code{(:source-registry ...)} form.
Robert P. Goldman's avatar
Robert P. Goldman committed
The files will be sorted by namestring as if by @code{string<} and
the lists of directives of these files with be concatenated in order.
An implicit @code{:inherit-configuration} will be included
at the end of the list.

This allows for packaging software that has file granularity
(e.g. Debian's @code{dpkg} or some future version of @code{clbuild})
to easily include configuration information about distributed software.

Francois-Rene Rideau's avatar
Francois-Rene Rideau committed
The convention is that, for sorting purposes,
the names of files in such a directory begin with two digits
that determine the order in which these entries will be read.
Also, the type of these files is conventionally @code{"conf"}
Francois-Rene Rideau's avatar
Francois-Rene Rideau committed
and as a limitation to some implementations (e.g. GNU clisp),
the type cannot be @code{NIL}.
Directories may be included by specifying a directory pathname
Francois-Rene Rideau's avatar
Francois-Rene Rideau committed
or namestring in an @code{:include} directive, e.g.:

@example
Francois-Rene Rideau's avatar
Francois-Rene Rideau committed
@end example


@section Shell-friendly syntax for configuration

When considering environment variable @code{CL_SOURCE_REGISTRY}
ASDF will skip to next configuration if it's an empty string.
It will @code{READ} the string as a SEXP in the DSL
if it begins with a paren @code{(}
and it will be interpreted much like @code{TEXINPUTS}
list of paths, where

  * paths are separated
   by a @code{:} (colon) on Unix platforms (including cygwin),
   by a @code{;} (semicolon) on other platforms (mainly, Windows).

  * each entry is a directory to add to the search path.

  * if the entry ends with a double slash @code{//}
    then it instead indicates a tree in the subdirectories
    of which to recurse.

  * if the entry is the empty string (which may only appear once),
    then it indicates that the inherited configuration should be
    spliced there.


@section Search Algorithm

In case that isn't clear, the semantics of the configuration is that
when searching for a system of a given name,
directives are processed in order.

When looking in a directory, if the system is found, the search succeeds,
otherwise it continues.

When looking in a tree, if one system is found, the search succeeds.
If multiple systems are found, the consequences are unspecified:
the search may succeed with any of the found systems,
or an error may be raised.
ASDF currently returns the first system found,
XCVB currently raised an error.
If none is found, the search continues.

Exclude statements specify patterns of subdirectories
the systems from which to ignore.
Typically you don't want to use copies of files kept by such
version control systems as Darcs.
Exclude statements are not propagated to further included or inherited
configuration files or expressions;
instead the defaults are reset around every configuration statement
to the default defaults from @code{asdf::*default-source-registry-exclusions*}.

Include statements cause the search to recurse with the path specifications
from the file specified.

An inherit-configuration statement cause the search to recurse with the path
specifications from the next configuration
(@pxref{Controlling where ASDF searches for systems,,Configurations} above).


@section Caching Results

The implementation is allowed to either eagerly compute the information
from the configurations and file system, or to lazily re-compute it
every time, or to cache any part of it as it goes.
To explicitly flush any information cached by the system, use the API below.


@section Configuration API

The specified functions are exported from your build system's package.
Thus for ASDF the corresponding functions are in package ASDF,
and for XCVB the corresponding functions are in package XCVB.

@defun initialize-source-registry @&optional PARAMETER
   will read the configuration and initialize all internal variables.
Francois-Rene Rideau's avatar
Francois-Rene Rideau committed
   You may extend or override configuration
   from the environment and configuration files
   with the given @var{PARAMETER}, which can be
   @code{NIL} (no configuration override),
   or a SEXP (in the SEXP DSL),
   a string (as in the string DSL),
   a pathname (of a file or directory with configuration),
   or a symbol (fbound to function that when called returns one of the above).
@end defun

@defun clear-source-registry
   undoes any source registry configuration
   and clears any cache for the search algorithm.
   You might want to call that before you
   dump an image that would be resumed with a different configuration,
   and return an empty configuration.
   Note that this does not include clearing information about
   systems defined in the current image, only about
   where to look for systems not yet defined.
@end defun

@defun ensure-source-registry @&optional PARAMETER
   checks whether a source registry has been initialized.
Francois-Rene Rideau's avatar
Francois-Rene Rideau committed
   If not, initialize it with the given @var{PARAMETER}.
@section Future

If this mechanism is successful, in the future, we may declare
@code{asdf:*central-registry*} obsolete and eventually remove it.
Any hook into implementation-specific search mechanisms will by then
have been integrated in the @code{:default-configuration} which everyone
should either explicitly use or implicit inherit. Some shell syntax
for it should probably be added somehow.

But we're not there yet. For now, let's see how practical this new
source-registry is.


@section Rejected ideas

Alternatives I considered and rejected included:

@enumerate
@item Keep @code{asdf:*central-registry*} as the master with its current semantics,
   and somehow the configuration parser expands the new configuration
   language into a expanded series of directories of subdirectories to
   lookup, pre-recursing through specified hierarchies. This is kludgy,
   and leaves little space of future cleanups and extensions.

@item Keep @code{asdf:*central-registry*} remains the master but extend its semantics
   in completely new ways, so that new kinds of entries may be implemented
   as a recursive search, etc. This seems somewhat backwards.

@item Completely remove @code{asdf:*central-registry*}
   and break backwards compatibility.
   Hopefully this will happen in a few years after everyone migrate to
   a better ASDF and/or to XCVB, but it would be very bad to do it now.

@item Replace @code{asdf:*central-registry*} by a symbol-macro with appropriate magic
   when you dereference it or setf it. Only the new variable with new
   semantics is handled by the new search procedure.
   Complex and still introduces subtle semantic issues.
@end enumerate


I've been suggested the below features, but have rejected them,
for the sake of keeping ASDF no more complex than strictly necessary.

@itemize
@item
  More syntactic sugar: synonyms for the configuration directives, such as
  @code{(:add-directory X)} for @code{(:directory X)}, or @code{(:add-directory-hierarchy X)}
  or @code{(:add-directory X :recurse t)} for @code{(:tree X)}.

@item
   The possibility to register individual files instead of directories.

@item
  Integrate Xach Beane's tilde expander into the parser,
  or something similar that is shell-friendly or shell-compatible.
  I'd rather keep ASDF minimal. But maybe this precisely keeps it
  minimal by removing the need for evaluated entries that ASDF has?
  i.e. uses of @code{USER-HOMEDIR-PATHNAME} and @code{$SBCL_HOME}
  Hopefully, these are already superseded by the @code{:default-registry}

@item
  Using the shell-unfriendly syntax @code{/**} instead of @code{//} to specify recursion
  down a filesystem tree in the environment variable.
  It isn't that Lisp friendly either.
@end itemize

@section TODO

@itemize
@item Add examples
@end itemize


@section Credits for the source-registry

Thanks a lot to Stelian Ionescu for the initial idea.

Thanks to Rommel Martinez for the initial implementation attempt.

All bad design ideas and implementation bugs are to mine, not theirs.
But so are good design ideas and elegant implementation tricks.

 --- Francois-Rene Rideau @email{fare@@tunes.org}, Mon, 22 Feb 2010 00:07:33 -0500



@node Controlling where ASDF saves compiled files, Error handling, Controlling where ASDF searches for systems, Top
@comment  node-name,  next,  previous,  up
@chapter Controlling where ASDF saves compiled files
@cindex asdf-output-translations
@vindex ASDF_OUTPUT_TRANSLATIONS
Each Common Lisp implementation has its own format
for compiled files (fasls for short, short for ``fast loading'').
If you use multiple implementations
(or multiple versions of the same implementation),
you'll soon find your source directories
littered with various @file{fasl}s, @file{dfsl}s, @file{cfsl}s and so on.
Worse yet, some implementations use the same file extension
while changing formats from version to version (or platform to platform)
which means that you'll have to recompile binaries
as you switch from one implementation to the next.
ASDF 2 includes the @code{asdf-output-translations} facility
@section Configurations
Configurations specify mappings from input locations to output locations.
Once again we rely on the XDG base directory specification for configuration.
@xref{Controlling where ASDF searches for systems,,XDG base directory}.
@item
Some hardcoded wrapping output translations configuration may be used.
This allows special output translations (or usually, invariant directories)
to be specified corresponding to the similar special entries in the source registry.

@item
An application may explicitly initialize the output-translations
configuration using the Configuration API
in which case this takes precedence.
(@pxref{Controlling where ASDF saves compiled files,,Configuration API}.)
It may itself compute this configuration from the command-line,
from a script, from its own configuration file, etc.