Newer
Older
@item
Using the shell-unfriendly syntax @code{/**} instead of @code{//} to specify recursion
down a filesystem tree in the environment variable.
It isn't that Lisp friendly either.
@end itemize
@section TODO
@itemize
@item Add examples
@end itemize
Thanks a lot to Stelian Ionescu for the initial idea.
Thanks to Rommel Martinez for the initial implementation attempt.
All bad design ideas and implementation bugs are to mine, not theirs.
But so are good design ideas and elegant implementation tricks.
--- Francois-Rene Rideau @email{fare@@tunes.org}, Mon, 22 Feb 2010 00:07:33 -0500
@node Controlling where ASDF saves compiled files, Error handling, Controlling where ASDF searches for systems, Top
Robert P. Goldman
committed
@comment node-name, next, previous, up
@chapter Controlling where ASDF saves compiled files
@cindex asdf-output-translations
@vindex ASDF_OUTPUT_TRANSLATIONS
Robert P. Goldman
committed
Each Common Lisp implementation has its own format
for compiled files (fasls for short, short for ``fast loading'').
If you use multiple implementations
(or multiple versions of the same implementation),
you'll soon find your source directories
littered with various @file{fasl}s, @file{dfsl}s, @file{cfsl}s and so on.
Robert P. Goldman
committed
Worse yet, some implementations use the same file extension
while changing formats from version to version (or platform to platform)
which means that you'll have to recompile binaries
as you switch from one implementation to the next.
Robert P. Goldman
committed
ASDF 2 includes the @code{asdf-output-translations} facility
to mitigate the problem.
Robert P. Goldman
committed
@section Configurations
Robert P. Goldman
committed
Configurations specify mappings from input locations to output locations.
Once again we rely on the XDG base directory specification for configuration.
@xref{Controlling where ASDF searches for systems,,XDG base directory}.
Robert P. Goldman
committed
Robert P. Goldman
committed
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
@item
Some hardcoded wrapping output translations configuration may be used.
This allows special output translations (or usually, invariant directories)
to be specified corresponding to the similar special entries in the source registry.
@item
An application may explicitly initialize the output-translations
configuration using the Configuration API
in which case this takes precedence.
(@pxref{Controlling where ASDF saves compiled files,,Configuration API}.)
It may itself compute this configuration from the command-line,
from a script, from its own configuration file, etc.
@item
The source registry will be configured from
the environment variable @code{ASDF_OUTPUT_TRANSLATIONS} if it exists.
@item
The source registry will be configured from
user configuration file
@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf}
(which defaults to
@file{~/.config/common-lisp/asdf-output-translations.conf})
if it exists.
@item
The source registry will be configured from
user configuration directory
@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf.d/}
(which defaults to
@file{~/.config/common-lisp/asdf-output-translations.conf.d/})
if it exists.
@item
The source registry will be configured from
system configuration file
@file{/etc/common-lisp/asdf-output-translations.conf}
if it exists.
@item
The source registry will be configured from
system configuration directory
@file{/etc/common-lisp/asdf-output-translations.conf.d/}
if it exists.
Robert P. Goldman
committed
Each of these configurations is specified as a SEXP
in a trival domain-specific language (defined below).
Additionally, a more shell-friendly syntax is available
for the environment variable (defined yet below).
Each of these configurations is only used if the previous
configuration explicitly or implicitly specifies that it
includes its inherited configuration.
Note that by default, a per-user cache is used for output files.
This allows the seamless use of shared installations of software
between several users, and takes files out of the way of the developers
when they browse source code,
at the expense of taking a small toll when developers have to clean up
output files and find they need to get familiar with output-translations first.
@section Backward Compatibility
@c FIXME -- I think we should provide an easy way
@c to get behavior equivalent to A-B-L and
@c I will propose a technique for doing this.
We purposefully do NOT provide backward compatibility with earlier versions of
@code{ASDF-Binary-Locations} (8 Sept 2009),
@code{common-lisp-controller} (7.0) or
@code{cl-launch} (2.35),
each of which had similar general capabilities.
The previous APIs of these programs were not designed
for configuration by the end-user
in an easy way with configuration files.
Recent versions of same packages use
the new @code{asdf-output-translations} API as defined below:
Francois-Rene Rideau
committed
@code{common-lisp-controller} (7.2) and @code{cl-launch} (3.000).
@code{ASDF-Binary-Locations} is fully superseded and not to be used anymore.
This incompatibility shouldn't inconvenience many people.
Indeed, few people use and customize these packages;
these few people are experts who can trivially adapt to the new configuration.
Most people are not experts, could not properly configure these features
(except inasmuch as the default configuration of
@code{common-lisp-controller} and/or @code{cl-launch}
might have been doing the right thing for some users),
and yet will experience software that ``just works'',
as configured by the system distributor, or by default.
Nevertheless, if you are a fan of @code{ASDF-Binary-Locations},
we provide a limited emulation mode:
@defun asdf:enable-asdf-binary-locations-compatibility @&key centralize-lisp-binaries default-toplevel-directory include-per-user-information map-all-source-files source-to-target-mappings
This function will initialize the new @code{asdf-output-translations} facility in a way
that emulates the behavior of the old @code{ASDF-Binary-Locations} facility.
Where you would previously set global variables
@var{*centralize-lisp-binaries*},
@var{*default-toplevel-directory*},
@var{*include-per-user-information*},
@var{*map-all-source-files*} or @var{*source-to-target-mappings*}
you will now have to pass the same values as keyword arguments to this function.
Note however that as an extension the @code{:source-to-target-mappings} keyword argument
will accept any valid pathname designator for @code{asdf-output-translations}
instead of just strings and pathnames.
@end defun
If you insist, you can also keep using the old @code{ASDF-Binary-Locations}
(the one available as an extension to load of top of ASDF,
not the one built into a few old versions of ASDF),
but first you must disable @code{asdf-output-translations}
with @code{(asdf:disable-output-translations)},
or you might experience ``interesting'' issues.
Also, note that output translation is enabled by default.
To disable it, use @code{(asdf:disable-output-translations)}.
@section Configuration DSL
Here is the grammar of the SEXP DSL
for @code{asdf-output-translations} configuration:
@verbatim
;; A configuration is single SEXP starting with keyword :source-registry
;; followed by a list of directives.
CONFIGURATION := (:output-translations DIRECTIVE ...)
;; A directive is one of the following:
DIRECTIVE :=
;; INHERITANCE DIRECTIVE:
;; Your configuration expression MUST contain
;; exactly one of either of these:
:inherit-configuration | ; splices inherited configuration (often specified last)
:ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
;; include a configuration file or directory
(:include PATHNAME-DESIGNATOR) |
;; enable global cache in ~/.common-lisp/cache/sbcl-1.0.35-x86-64/ or something.
:enable-user-cache |
;; Disable global cache. Map / to /
:disable-cache |
;; add a single directory to be scanned (no recursion)
(DIRECTORY-DESIGNATOR DIRECTORY-DESIGNATOR)
Francois-Rene Rideau
committed
;; use a function to return the translation of a directory designator
(DIRECTORY-DESIGNATOR (:function TRANSLATION-FUNCTION))
DIRECTORY-DESIGNATOR :=
Francois-Rene Rideau
committed
T | ;; as source matches anything, as destination leaves pathname unmapped.
ABSOLUTE-COMPONENT-DESIGNATOR |
(ABSOLUTE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...)
ABSOLUTE-COMPONENT-DESIGNATOR :=
Francois-Rene Rideau
committed
NULL | ;; As source: skip this entry. As destination: same as source
Francois-Rene Rideau
committed
:ROOT | ;; magic: paths that are relative to the root of the source host and device
STRING | ;; namestring (directory is assumed, better be absolute or bust, ``/**/*.*'' added)
PATHNAME | ;; pathname (better be an absolute directory or bust)
:HOME | ;; designates the user-homedir-pathname ~/
:USER-CACHE | ;; designates the default location for the user cache
Francois-Rene Rideau
committed
:SYSTEM-CACHE ;; designates the default location for the system cache
RELATIVE-COMPONENT-DESIGNATOR :=
STRING | ;; namestring, directory is assumed. If the last component, /**/*.* is added
PATHNAME | ;; pathname unless last component, directory is assumed.
:IMPLEMENTATION | ;; a directory based on implementation, e.g. sbcl-1.0.32.30-linux-x86-64
:IMPLEMENTATION-TYPE | ;; a directory based on lisp-implementation-type only, e.g. sbcl
:UID | ;; current UID -- not available on Windows
:USER ;; current USER name -- NOT IMPLEMENTED(!)
Francois-Rene Rideau
committed
TRANSLATION-FUNCTION :=
SYMBOL | ;; symbol of a function that takes two arguments,
;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
LAMBDA ;; A form which evalutates to a function taking two arguments consisting of
;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
@end verbatim
Relative components better be either relative
or subdirectories of the path before them, or bust.
The last component, if not a pathname, is notionally completed by @file{/**/*.*}.
You can specify more fine-grained patterns
by using a pathname object as the last component
e.g. @file{#p"some/path/**/foo*/bar-*.fasl"}
You may use @code{#+features} to customize the configuration file.
The second designator of a mapping may be @code{NIL}, indicating that files are not mapped
to anything but themselves (same as if the second designator was the same as the first).
When the first designator is @code{t},
the mapping always matches.
When the first designator starts with @code{:root},
the mapping matches any host and device.
In either of these cases, if the second designator
isn't @code{t} and doesn't start with @code{:root},
then strings indicating the host and pathname are somehow copied
in the beginning of the directory component of the source pathname
before it is translated.
When the second designator is @code{t}, the mapping is the identity.
When the second designator starts with @code{root},
the mapping preserves the host and device of the original pathname.
@code{:include} statements cause the search to recurse with the path specifications
from the file specified.
Francois-Rene Rideau
committed
If the @code{translate-pathname} mechanism cannot achieve a desired
translation, the user may provide a function which provides the
required algorithim. Such a translation function is specified by
supplying a list as the second @code{directory-designator}
the first element of which is the keyword @code{:function},
and the second element of which is
Francois-Rene Rideau
committed
either a symbol which designates a function or a lambda expression.
The function designated by the second argument must take two arguments,
the first being the pathname of the source file,
the second being the wildcard that was matched.
The result of the function invocation should be the translated pathname.
Francois-Rene Rideau
committed
An @code{:inherit-configuration} statement cause the search to recurse with the path
specifications from the next configuration.
@xref{Controlling where ASDF saves compiled files,,Configurations}, above.
Francois-Rene Rideau
committed
@code{:enable-user-cache} is the same as @code{(t :user-cache)}.
Francois-Rene Rideau
committed
@code{:disable-cache} is the same as @code{(t t)}.
@item
@code{:user-cache} uses the contents of variable @code{asdf::*user-cache*}
which by default is the same as using
@code{(:home ".cache" "common-lisp" :implementation)}.
@item
@code{:system-cache} uses the contents of variable @code{asdf::*system-cache*}
which by default is the same as using
Francois-Rene Rideau
committed
@code{("/var/cache/common-lisp" :uid :implementation-type)}
(on Unix and cygwin), or something semi-sensible on Windows.
Robert P. Goldman
committed
@end itemize
Robert P. Goldman
committed
@section Configuration Directories
Configuration directories consist in files each contains
a list of directives without any enclosing
Francois-Rene Rideau
committed
@code{(:output-translations ...)} form.
The files will be sorted by namestring as if by @code{string<} and
the lists of directives of these files with be concatenated in order.
An implicit @code{:inherit-configuration} will be included
at the @emph{end} of the list.
This allows for packaging software that has file granularity
(e.g. Debian's @command{dpkg} or some future version of @command{clbuild})
to easily include configuration information about software being distributed.
The convention is that, for sorting purposes,
the names of files in such a directory begin with two digits
that determine the order in which these entries will be read.
Also, the type of these files is conventionally @code{"conf"}
and as a limitation of some implementations, the type cannot be @code{NIL}.
Directories may be included by specifying a directory pathname
or namestring in an @code{:include} directive, e.g.:
@verbatim
(:include "/foo/bar/")
@end verbatim
@section Shell-friendly syntax for configuration
When considering environment variable @code{ASDF_OUTPUT_TRANSLATIONS}
ASDF will skip to next configuration if it's an empty string.
It will @code{READ} the string as an SEXP in the DSL
if it begins with a paren @code{(}
Francois-Rene Rideau
committed
and it will be interpreted as a list of directories.
Directories should come by pairs, indicating a mapping directive.
Francois-Rene Rideau
committed
Entries are separated
by a @code{:} (colon) on Unix platforms (including cygwin),
by a @code{;} (semicolon) on other platforms (mainly, Windows).
The magic empty entry,
if it comes in what would otherwise be the first entry in a pair,
indicates the splicing of inherited configuration.
If it comes as the second entry in a pair,
it indicates that the directory specified first is to be left untranslated
(which has the same effect as if the directory had been repeated).
@section Semantics of Output Translations
From the specified configuration,
a list of mappings is extracted in a straightforward way:
2352
2353
2354
2355
2356
2357
2358
2359
2360
2361
2362
2363
2364
2365
2366
2367
2368
2369
2370
2371
2372
2373
2374
2375
2376
2377
2378
2379
2380
2381
2382
2383
2384
2385
2386
2387
2388
mappings are collected in order, recursing through
included or inherited configuration as specified.
To this list is prepended some implementation-specific mappings,
and is appended a global default.
The list is then compiled to a mapping table as follows:
for each entry, in order, resolve the first designated directory
into an actual directory pathname for source locations.
If no mapping was specified yet for that location,
resolve the second designated directory to an output location directory
add a mapping to the table mapping the source location to the output location,
and add another mapping from the output location to itself
(unless a mapping already exists for the output location).
Based on the table, a mapping function is defined,
mapping source pathnames to output pathnames:
given a source pathname, locate the longest matching prefix
in the source column of the mapping table.
Replace that prefix by the corresponding output column
in the same row of the table, and return the result.
If no match is found, return the source pathname.
(A global default mapping the filesystem root to itself
may ensure that there will always be a match,
with same fall-through semantics).
@section Caching Results
The implementation is allowed to either eagerly compute the information
from the configurations and file system, or to lazily re-compute it
every time, or to cache any part of it as it goes.
To explicitly flush any information cached by the system, use the API below.
@section Output location API
The specified functions are exported from package ASDF.
Francois-Rene Rideau
committed
@defun initialize-output-translations @&optional PARAMETER
will read the configuration and initialize all internal variables.
You may extend or override configuration
from the environment and configuration files
with the given @var{PARAMETER}, which can be
@code{NIL} (no configuration override),
or a SEXP (in the SEXP DSL),
a string (as in the string DSL),
a pathname (of a file or directory with configuration),
or a symbol (fbound to function that when called returns one of the above).
@defun disable-output-translations
will initialize output translations in a way
that maps every pathname to itself,
effectively disabling the output translation facility.
@end defun
@defun clear-output-translations
undoes any output translation configuration
and clears any cache for the mapping algorithm.
Francois-Rene Rideau
committed
You might want to call this function
(or better, @code{clear-configuration})
before you dump an image that would be resumed
with a different configuration,
and return an empty configuration.
Note that this does not include clearing information about
systems defined in the current image, only about
where to look for systems not yet defined.
Francois-Rene Rideau
committed
@defun ensure-output-translations @&optional PARAMETER
checks whether output translations have been initialized.
If not, initialize them with the given @var{PARAMETER}.
This function will be called before any attempt to operate on a system.
@defun apply-output-translations PATHNAME
Applies the configured output location translations to @var{PATHNAME}
(calls @code{ensure-output-translations} for the translations).
Francois-Rene Rideau
committed
Every time you use ASDF's @code{output-files}, or
anything that uses it (that may compile, such as @code{operate}, @code{perform}, etc.),
@code{ensure-output-translations} is called with parameter NIL,
which the first time around causes your configuration to be read.
If you change a configuration file,
you need to explicitly @code{initialize-output-translations} again,
or maybe @code{clear-output-translations} (or @code{clear-configuration}),
which will cause the initialization to happen next time around.
Thanks a lot to Bjorn Lindberg and Gary King for @code{ASDF-Binary-Locations},
and to Peter van Eynde for @code{Common Lisp Controller}.
All bad design ideas and implementation bugs are to mine, not theirs.
But so are good design ideas and elegant implementation tricks.
--- Francois-Rene Rideau @email{fare@@tunes.org}
2450
2451
2452
2453
2454
2455
2456
2457
2458
2459
2460
2461
2462
2463
2464
2465
2466
2467
2468
2469
2470
2471
2472
2473
2474
2475
2476
@c @section Default locations
@c @findex output-files-for-system-and-operation
@c The default binary location for each Lisp implementation
@c is a subdirectory of each source directory.
@c To account for different Lisps, Operating Systems, Implementation versions,
@c and so on, ASDF borrows code from SLIME
@c to create reasonable custom directory names.
@c Here are some examples:
@c @itemize
@c @item
@c SBCL, version 1.0 on Mac OS X for intel: @code{sbcl-1.0-darwin-x86}
@c @item
@c Franz Allegro, version 8.0, ANSI Common Lisp: @code{allegro-8.0a-macosx-x86}
@c @item
@c Franz Allegro, version 8.1, Modern (case sensitive) Common Lisp: @code{allegro-8.1m-macosx-x86}
@c @end itemize
@c By default, all output file pathnames will be relocated
@c to some thus-named subdirectory of @file{~/.cache/common-lisp/}.
@c See the document @file{README.asdf-output-translations}
@c for a full specification on how to configure @code{asdf-output-translations}.
Robert P. Goldman
committed
@node Error handling, Miscellaneous additional functionality, Controlling where ASDF saves compiled files, Top
@comment node-name, next, previous, up
@chapter Error handling
@findex SYSTEM-DEFINITION-ERROR
@findex OPERATION-ERROR
@section ASDF errors
Robert P. Goldman
committed
If ASDF detects an incorrect system definition, it will signal a generalised instance of
@code{SYSTEM-DEFINITION-ERROR}.
Operations may go wrong (for example when source files contain errors).
These are signalled using generalised instances of
@code{OPERATION-ERROR}.
@section Compilation error and warning handling
@vindex *compile-file-warnings-behaviour*
@vindex *compile-file-errors-behavior*
ASDF checks for warnings and errors when a file is compiled.
The variables @var{*compile-file-warnings-behaviour*} and
@var{*compile-file-errors-behavior*}
control the handling of any such events.
The valid values for these variables are
@code{:error}, @code{:warn}, and @code{:ignore}.
@node Miscellaneous additional functionality, Getting the latest version, Error handling, Top
@comment node-name, next, previous, up
@chapter Miscellaneous additional functionality
@emph{FIXME: Add discussion of @code{run-shell-command}? Others?}
ASDF includes several additional features that are generally
useful for system definition and development. These include:
Robert P. Goldman
committed
@defun system-relative-pathname system name @&key type
It's often handy to locate a file relative to some system.
The @code{system-relative-pathname} function meets this need.
It takes two arguments: the name of a system and a relative pathname.
It returns a pathname built from the location of the system's source file
and the relative pathname. For example
@lisp
> (asdf:system-relative-pathname 'cl-ppcre #p"regex.data")
#P"/repository/other/cl-ppcre/regex.data"
@end lisp
Instead of a pathname, you can provide a symbol or a string,
and optionally a keyword argument @code{type}.
The arguments will then be interpreted in the same way
as pathname specifiers for components.
@xref{The defsystem grammar,,Pathname specifiers}.
Robert P. Goldman
committed
@end defun
Robert P. Goldman
committed
@defun system-source-directory system-designator
ASDF does not provide a turnkey solution for locating
data (or other miscellaneous) files
that are distributed together with the source code of a system.
Programmers can use @code{system-source-directory} to find such files.
Returns a pathname object.
The @var{system-designator} may be a string, symbol, or ASDF system object.
Robert P. Goldman
committed
@end defun
2543
2544
2545
2546
2547
2548
2549
2550
2551
2552
2553
2554
2555
2556
2557
2558
2559
2560
2561
2562
2563
2564
2565
2566
2567
2568
2569
2570
2571
2572
2573
@defun clear-system system-designator
It is sometimes useful to force recompilation of a previously loaded system.
In these cases, it may be useful to @code{(asdf:clear-system :foo)}
to remove the system from the table of currently loaded systems;
the next time the system @code{foo} or one that depends on it is re-loaded,
@code{foo} will then be loaded again.
Alternatively, you could touch @code{foo.asd} or
remove the corresponding fasls from the output file cache.
(It was once conceived that one should provide
a list of systems the recompilation of which to force
as the @code{:force} keyword argument to @code{load-system};
but this has never worked, and though the feature was fixed in ASDF 2.000,
it remains @code{cerror}'ed out as nobody ever used it.)
Note that this does not and cannot by itself undo the previous loading
of the system. Common Lisp has no provision for such an operation,
and its reliance on irreversible side-effects to global datastructures
makes such a thing impossible in the general case.
If the software being re-loaded is not conceived with hot upgrade in mind,
this re-loading may cause many errors, warnings or subtle silent problems,
as packages, generic function signatures, structures, types, macros, constants, etc.
are being redefined incompatibly.
It is up to the user to make sure that reloading is possible and has the desired effect.
In some cases, extreme measures such as recursively deleting packages,
unregistering symbols, defining methods on @code{update-instance-for-redefined-class}
and much more are necessary for reloading to happen smoothly.
ASDF itself goes through notable pains to make such a hot upgrade possible
with respect to its own code, and what it does is ridiculously complex;
look at the beginning of @file{asdf.lisp} to see what it does.
@end defun
@node Getting the latest version, FAQ, Miscellaneous additional functionality, Top
@comment node-name, next, previous, up
@chapter Getting the latest version
Decide which version you want.
HEAD is the newest version and usually OK, whereas
RELEASE is for cautious people
(e.g. who already have systems using ASDF that they don't want broken),
a slightly older version about which none of the HEAD users have complained.
There is also a STABLE version, which is earlier than release.
You may get the ASDF source repository using git:
@kbd{git clone git://common-lisp.net/projects/asdf/asdf.git}
You will find the above referenced tags in this repository.
You can also browse the repository on
@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf.git}.
Discussion of ASDF development is conducted on the
mailing list
@kbd{asdf-devel@@common-lisp.net}.
@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel}
@node FAQ, TODO list, Getting the latest version, Top
@comment node-name, next, previous, up
ASDF bugs are tracked on launchpad: @url{https://launchpad.net/asdf}.
If you're unsure about whether something is a bug, or for general discussion,
use the @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
@section ``What has changed between ASDF 1 and ASDF 2?''
@subsection What are ASDF 1 and ASDF 2?
On May 31st 2010, we have released ASDF 2.
ASDF 2 refers to release 2.000 and later.
(Releases between 1.656 and 1.728 were development releases for ASDF 2.)
ASDF 1 to any release earlier than 1.369 or so.
If your ASDF doesn't sport a version, it's an old ASDF 1.
ASDF 2 and its release candidates push
@code{:asdf2} onto @code{*features*} so that if you are writing
ASDF-dependent code you may check for this feature
to see if the new API is present.
@emph{All} versions of ASDF should have the @code{:asdf} feature.
If you are experiencing problems or limitations of any sort with ASDF 1,
we recommend that you should upgrade to ASDF 2,
or whatever is the latest release.
@subsection ASDF can portably name files in subdirectories
Common Lisp namestrings are not portable,
except maybe for logical pathnamestrings,
that themselves have various limitations and require a lot of setup
that is itself ultimately non-portable.
In ASDF 1, the only portable ways to refer to pathnames inside systems and components
were very awkward, using @code{#.(make-pathname ...)} and
@code{#.(merge-pathnames ...)}.
Even the above were themselves were inadequate in the general case
due to host and device issues, unless horribly complex patterns were used.
Plenty of simple cases that looked portable actually weren't,
leading to much confusion and greavance.
ASDF 2 implements its own portable syntax for strings as pathname specifiers.
Naming files within a system definition becomes easy and portable again.
Robert P. Goldman
committed
@xref{Miscellaneous additional functionality,asdf:system-relative-pathname},
@code{asdf-utilities:merge-pathnames*},
Francois-Rene Rideau
committed
On the other hand, there are places where systems used to accept namestrings
where you must now use an explicit pathname object:
@code{(defsystem ... :pathname "LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
must now be written with the @code{#p} syntax:
@code{(defsystem ... :pathname #p"LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
@xref{The defsystem grammar,,Pathname specifiers}.
A popular feature added to ASDF was output pathname translation:
@code{asdf-binary-locations}, @code{common-lisp-controller},
@code{cl-launch} and other hacks were all implementing it in ways
both mutually incompatible and difficult to configure.
Output pathname translation is essential to share
source directories of portable systems across multiple implementations
or variants thereof,
or source directories of shared installations of systems across multiple users,
or combinations of the above.
In ASDF 2, a standard mechanism is provided for that,
@code{asdf-output-translations},
with sensible defaults, adequate configuration languages,
a coherent set of configuration files and hooks,
and support for non-Unix platforms.
@xref{Controlling where ASDF saves compiled files}.
@subsection Source Registry Configuration
Configuring ASDF used to require special magic
to be applied just at the right moment,
between the moment ASDF is loaded and the moment it is used,
in a way that is specific to the user,
the implementation he is using and the application he is building.
This made for awkward configuration files and startup scripts
that could not be shared between users, managed by administrators
or packaged by distributions.
ASDF 2 provides a well-documented way to configure ASDF,
with sensible defaults, adequate configuration languages,
and a coherent set of configuration files and hooks.
We believe it's a vast improvement because it decouples
application distribution from library distribution.
The application writer can avoid thinking where the libraries are,
and the library distributor (dpkg, clbuild, advanced user, etc.)
can configure them once and for every application.
Yet settings can be easily overridden where needed,
so whoever needs control has exactly as much as required.
At the same time, ASDF 2 remains compatible
with the old magic you may have in your build scripts
(using @code{*central-registry*} and
@code{*system-definition-search-functions*})
to tailor the ASDF configuration to your build automation needs,
and also allows for new magic, simpler and more powerful magic.
@xref{Controlling where ASDF searches for systems}.
@subsection Usual operations are made easier to the user
In ASDF 1, you had to use the awkward syntax
@code{(asdf:oos 'asdf:load-op :foo)}
to load a system,
and similarly for @code{compile-op}, @code{test-op}.
In ASDF 2, you can use shortcuts for the usual operations:
@code{(asdf:load-system :foo)}, and
similarly for @code{compile-system}, @code{test-system}.
@subsection Many bugs have been fixed
The following issues and many others have been fixed:
The infamous TRAVERSE function has been revamped significantly,
with many bugs squashed.
In particular, dependencies were not correctly propagated
across submodules within a system but now are.
The :version and :feature features and
the :force (system1 .. systemN) feature have been fixed.
Francois-Rene Rideau
committed
Performance has been notably improved for large systems
(say with thousands of components) by using
hash-tables instead of linear search,
and linear-time list accumulation
instead of quadratic-time recursive appends.
@item
Many features used to not be portable,
especially where pathnames were involved.
Windows support was notably quirky because of such non-portability.
@item
The internal test suite used to massively fail on many implementations.
While still incomplete, it now fully passes
on all implementations supported by the test suite,
except for GCL (due to GCL bugs).
@item
Support was lacking for some implementations.
ABCL and GCL were notably wholly broken.
ECL extensions were not integrated in the ASDF release.
The documentation was grossly out of date.
@subsection ASDF itself is versioned
Between new features, old bugs fixed, and new bugs introduced,
there were various releases of ASDF in the wild,
and no simple way to check which release had which feature set.
People using or writing systems had to either make worst-case assumptions
as to what features were available and worked,
or take great pains to have the correct version of ASDF installed.
With ASDF 2, we provide a new stable set of working features
that everyone can rely on from now on.
Use @code{#+asdf2} to detect presence of ASDF 2,
@code{(asdf:version-satisfies (asdf:asdf-version) "2.000")}
to check the availability of a version no earlier than required.
@subsection ASDF can be upgraded
When an old version of ASDF was loaded,
it was very hard to upgrade ASDF in your current image
without breaking everything.
Instead you have to exit the Lisp process and
somehow arrange to start a new one from a simpler image.
Something that can't be done from within Lisp,
making automation of it difficult,
which compounded with difficulty in configuration,
made the task quite hard.
Yet as we saw before, the task would have been required
to not have to live with the worst case or non-portable
subset of ASDF features.
With ASDF 2, it is easy to upgrade
from ASDF 2 to later versions from within Lisp,
and not too hard to upgrade from ASDF 1 to ASDF 2 from within Lisp.
We support hot upgrade of ASDF and any breakage is a bug
that we will do our best to fix.
There are still limitations on upgrade, though,
most notably the fact that after you upgrade ASDF,
you must also reload or upgrade all ASDF extensions.
@subsection Decoupled release cycle
When vendors were releasing their Lisp implementations with ASDF,
they had to basically never change version
because neither upgrade nor downgrade was possible
without breaking something for someone,
and no obvious upgrade path was visible and recommendable.
With ASDF 2, upgrade is possible, easy and can be recommended.
This means that vendors can safely ship a recent version of ASDF,
confident that if a user isn't fully satisfied,
he can easily upgrade ASDF and deal
with a supported recent version of it.
This means that release cycles will be causally decoupled,
the practical consequence of which will mean faster convergence
towards the latest version for everyone.
Francois-Rene Rideau
committed
@subsection Pitfalls of the transition to ASDF 2
2830
2831
2832
2833
2834
2835
2836
2837
2838
2839
2840
2841
2842
2843
2844
2845
2846
2847
2848
2849
2850
2851
2852
2853
2854
2855
The main pitfalls in upgrading to ASDF 2 seem to be related
to the output translation mechanism.
@itemize
@item
Output translations is enabled by default. This may surprise some users,
most of them in pleasant way (we hope), a few of them in an unpleasant way.
It is trivial to disable output translations.
@xref{FAQ,,``How can I wholly disable the compiler output cache?''}.
@item
Some systems in the large have been known not to play well with output translations.
They were relatively easy to fix.
Once again, it is also easy to disable output translations,
or to override its configuration.
@item
The new ASDF output translations are incompatible with ASDF-Binary-Locations.
They replace A-B-L, and there is compatibility mode to emulate
your previous A-B-L configuration.
See @code{asdf:enable-asdf-binary-locations-compatibility} in
@pxref{Controlling where ASDF saves compiled files,,Backward Compatibility}.
But thou shall not load ABL on top of ASDF 2.
Francois-Rene Rideau
committed
@end itemize
Other issues include the following:
@itemize
Francois-Rene Rideau
committed
@item
ASDF pathname designators are now specified in places where they were unspecified,
and a few small adjustments have to be made to some non-portable defsystems.
Notably, in the @code{:pathname} argument to a @code{defsystem} and its components,
a logical pathname (or implementation-dependent hierarchical pathname)
must now be specified with @code{#p} syntax
where the namestring might have previously sufficed;
moreover when evaluation is desired @code{#.} must be used,
where it wasn't necessary in the toplevel @code{:pathname} argument.
@item
There is a slight performance bug, notably on SBCL,
when initially searching for @file{asd} files,
the implicit @code{(directory "/configured/path/**/*.asd")}
for every configured path @code{(:tree "/configured/path/")}
in your @code{source-registry} configuration can cause a slight pause.
Try to @code{(time (asdf:initialize-source-registry))}
to see how bad it is or isn't on your system.
If you insist on not having this pause,
you can avoid the pause by overriding the default source-registry configuration
and not use any deep @code{:tree} entry but only @code{:directory} entries
or shallow @code{:tree} entries.
Or you can fix your implementation to not be quite that slow
when recursing through directories.
@item
On Windows, only LispWorks supports proper default configuration pathnames
based on the Windows registry.
Francois-Rene Rideau
committed
Other implementations make do with environment variables.
Windows support is somewhat less tested than Unix support.
Please help report and fix bugs.
@item
The mechanism by which one customizes a system so that Lisp files
may use a different extension from the default @file{.lisp} has changed.
Previously, the pathname for a component was lazily computed when operating on a system,
and you would
@code{(defmethod source-file-type ((component cl-source-file) (system (eql (find-system 'foo))))
(declare (ignorable component system)) "cl")}.
Now, the pathname for a component is eagerly computed when defining the system,
and instead you will @code{(defclass my-cl-source-file (cl-source-file) ((type :iniform "cl")))}
and use @code{:default-component-class my-cl-source-file} as argument to @code{defsystem},
as detailed in a @pxref{FAQ,How do I create a system definition where all the source files have a .cl extension?} below.
@findex source-file-type
@section Issues with installing the proper version of ASDF
@subsection ``My Common Lisp implementation comes with an outdated version of ASDF. What to do?''
We recommend you upgrade ASDF.
@xref{Loading ASDF,,Upgrading ASDF}.
If this does not work, it is a bug, and you should report it.
@xref{FAQ, report-bugs, Where do I report a bug}.
In the meantime, you can load @file{asdf.lisp} directly.
@xref{Loading ASDF,Loading an otherwise installed ASDF}.
@subsection ``I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?''
Starting with current candidate releases of ASDF 2,
it should always be a good time to upgrade to a recent ASDF.
You may consult with the maintainer for which specific version they recommend,
but the latest RELEASE should be correct.
We trust you to thoroughly test it with your implementation before you release it.
If there are any issues with the current release,
it's a bug that you should report upstream and that we will fix ASAP.
Francois-Rene Rideau
committed
As to how to include ASDF, we recommend the following:
Francois-Rene Rideau
committed
@itemize
@item
If ASDF isn't loaded yet, then @code{(require :asdf)}
Francois-Rene Rideau
committed
should load the version of ASDF that is bundled with your system.
You may have it load some other version configured by the user,
if you allow such configuration.
@item
If your system provides a mechanism to hook into @code{CL:REQUIRE},
then it would be nice to add ASDF to this hook the same way that
ABCL, CCL, CLISP, CMUCL, ECL, SBCL and SCL do it.
Francois-Rene Rideau
committed
@item
You may, like SBCL, have ASDF be implicitly used to require systems
that are bundled with your Lisp distribution.
If you do have a few magic systems that come with your implementation
in a precompiled way such that one should only use the binary version
that goes with your distribution, like SBCL does,
then you should add them in the beginning of @code{wrapping-source-registry}.
@item
If you have magic systems as above, like SBCL does,
then we explicitly ask you to @emph{NOT} distribute
@file{asdf.asd} as part of those magic systems.
You should still include the file @file{asdf.lisp} in your source distribution
and precompile it in your binary distribution,
but @file{asdf.asd} if included at all,
should be secluded from the magic systems,
in a separate file hierarchy;
alternatively, you may provide the system
after renaming it and its @file{.asd} file to e.g.
Francois-Rene Rideau
committed
@code{asdf-ecl} and @file{asdf-ecl.asd}, or
@code{sb-asdf} and @file{sb-asdf.asd}.
Indeed, if you made @file{asdf.asd} a magic system,
then users would no longer be able to upgrade ASDF using ASDF itself
to some version of their preference that
they maintain independently from your Lisp distribution.
@item
If you do not have any such magic systems, or have other non-magic systems
that you want to bundle with your implementation,
then you may add them to the @code{default-source-registry},
and you are welcome to include @file{asdf.asd} amongst them.
Francois-Rene Rideau
committed
@item
Please send us upstream any patches you make to ASDF itself,
so we can merge them back in for the benefit of your users
when they upgrade to the upstream version.
Francois-Rene Rideau
committed
@end itemize
@section Issues with configuring ASDF
@subsection ``How can I customize where fasl files are stored?''
@xref{Controlling where ASDF saves compiled files}.
Note that in the past there was an add-on to ASDF called
@code{ASDF-binary-locations}, developed by Gary King.
That add-on has been merged into ASDF proper,
Francois-Rene Rideau
committed
then superseded by the @code{asdf-output-translations} facility.