Newer
Older
Francois-Rene Rideau
committed
comments will be skipped just the same, strings with be read and printed
with slightly different lengths, symbol names will be accordingly longer,
but none of it should matter.
But a few systems that actually depend on unicode characters
may fail to work properly, or may work in a subtly different way.
We invite you to embrace UTF-8
as the encoding for non-ASCII characters starting today,
even without any explicit specification in your @code{.asd} files.
Indeed, on some implementations and configurations,
UTF-8 is already the @code{:default},
and loading your code may cause errors if it is encoded in anything but UTF-8.
Therefore, even with the legacy behavior,
non-UTF-8 is guaranteed to break for some users,
whereas UTF-8 is pretty much guaranteed not to break anywhere
(provided you do @emph{not} use a BOM),
although it might be read incorrectly on some implementations.
In the future, we intend to make @code{:utf-8}
the default value of @code{*default-encoding*},
to be enforced everywhere, so at least the code is guaranteed
to be read correctly everywhere it can be.
Francois-Rene Rideau
committed
If you need non-standard character encodings for your source code,
use the extension system @code{asdf-encodings}, by specifying
@code{:defsystem-depends-on (:asdf-encodings)} in your @code{defsystem}.
This extension system will register support for more encodings using the
@code{*encoding-external-format-hook*} facility,
so you can explicitly specify @code{:encoding :latin1}
in your @code{.asd} file.
Using the @code{*encoding-detection-hook*} it will also
eventually implement some autodetection of a file's encoding
from an emacs-style @code{-*- mode: lisp ; coding: latin1 -*-} declaration,
or otherwise based on an analysis of octet patterns in the file.
At this point, asdf-encoding only supports the encodings
that are supported as part of your implementation.
Since the list varies depending on implementations,
we once again recommend you use @code{:utf-8} everywhere,
which is the most portable (next is @code{:latin1}).
If you're not using a version of Quicklisp that has it,
you may get the source for @code{asdf-encodings} using git:
@kbd{git clone git://common-lisp.net/projects/asdf/asdf-encodings.git}
or
@kbd{git clone ssh://common-lisp.net/project/asdf/git/asdf-encodings.git}.
You can also browse the repository on
@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf-encodings.git}.
Francois-Rene Rideau
committed
In the future, we intend to change the default @code{*default-encoding*}
to @code{:utf-8}, which is already the de facto standard
for most libraries that use non-ASCII characters:
utf-8 works everywhere and was backhandedly enforced by
a lot of people using SBCL and utf-8 and sending reports to authors
so they make their packages compatible.
A survey showed only about a handful few libraries
are incompatible with non-UTF-8, and then, only in comments,
and we believe that authors will adopt UTF-8 when prompted.
See the April 2012 discussion on the asdf-devel mailing-list.
Francois-Rene Rideau
committed
For backwards compatibility with users who insist on a non-UTF-8 encoding,
but cannot immediately transition to using @code{asdf-encodings}
(maybe because it isn't ready), it will still be possible to use
Francois-Rene Rideau
committed
the @code{:encoding :default} option in your @code{defsystem} form
to restore the behavior of ASDF 2.20 and earlier.
Francois-Rene Rideau
committed
This shouldn't be required in libraries,
because user pressure as mentioned above will already have pushed
library authors towards using UTF-8;
but authors of end-user programs might care.
When you use @code{asdf-encodings}, any @code{.asd} file
loaded
will use the autodetection algorithm to determine its encoding.
If you depend on this detection happening,
you should explicitly load @code{asdf-encodings} early in your build.
Note that @code{:defsystem-depends-on} cannot be used here: by the time
the @code{:defsystem-depends-on} is loaded, the enclosing
@code{defsystem} form has already been read.
In practice, this means that the @code{*default-encoding*}
is usually used for @code{.asd} files.
Currently, this defaults to @code{:default} for backwards compatibility,
and that means that you shouldn't rely on non-ASCII characters in a .asd file.
Since component (path)names are the only real data in these files,
and non-ASCII characters are not very portable for file names,
this isn't too much of an issue.
We still encourage you to use either plain ASCII or UTF-8
in @code{.asd} files,
as we intend to make @code{:utf-8} the default encoding in the future.
This might matter, for instance, in meta-data about author's names.
Francois-Rene Rideau
committed
These functions are exported by ASDF for your convenience.
@anchor{system-relative-pathname}
@defun system-relative-pathname system name @Akey@ type
It's often handy to locate a file relative to some system.
The @code{system-relative-pathname} function meets this need.
Francois-Rene Rideau
committed
It takes two mandatory arguments @var{system} and @var{name}
and a keyword argument @var{type}:
@var{system} is name of a system, whereas @var{name} and optionally @var{type}
specify a relative pathname, interpreted like a component pathname specifier
by @code{coerce-pathname}. @xref{The defsystem grammar,,Pathname specifiers}.
Francois-Rene Rideau
committed
It returns a pathname built from the location of the system's
source directory and the relative pathname. For example:
@lisp
Francois-Rene Rideau
committed
> (asdf:system-relative-pathname 'cl-ppcre "regex.data")
#P"/repository/other/cl-ppcre/regex.data"
@end lisp
Robert P. Goldman
committed
@end defun
Robert P. Goldman
committed
@defun system-source-directory system-designator
ASDF does not provide a turnkey solution for locating
data (or other miscellaneous) files
that are distributed together with the source code of a system.
Programmers can use @code{system-source-directory} to find such files.
Returns a pathname object.
The @var{system-designator} may be a string, symbol, or ASDF system object.
Robert P. Goldman
committed
@end defun
@defun clear-system system-designator
It is sometimes useful to force recompilation of a previously loaded system.
For these cases, @code{(asdf:clear-system :foo)}
will remove the system from the table of currently loaded systems:
the next time the system @code{foo} or one that depends on it is re-loaded,
@code{foo} will be loaded again.\footnote{Alternatively, you could touch @code{foo.asd} or
remove the corresponding fasls from the output file cache.}
@c FIXME: Is the following correct?
@c (It was once conceived that one should provide
@c a list of systems the recompilation of which to force
@c as the @code{:force} keyword argument to @code{load-system};
@c but this has never worked, and though the feature was fixed in ASDF 2.000,
@c it remains @code{cerror}'ed out as nobody ever used it.)
Note that this does not and cannot undo the previous loading
of the system. Common Lisp has no provision for such an operation,
and its reliance on irreversible side-effects to global data structures
makes such a thing impossible in the general case.
If the software being re-loaded is not conceived with hot upgrade in mind,
re-loading may cause many errors, warnings or subtle silent problems,
as packages, generic function signatures, structures, types, macros, constants, etc.
are being redefined incompatibly.
It is up to the user to make sure that reloading is possible and has the desired effect.
In some cases, extreme measures such as recursively deleting packages,
unregistering symbols, defining methods on @code{update-instance-for-redefined-class}
and much more are necessary for reloading to happen smoothly.
ASDF itself goes to extensive effort to make A hot upgrade possible
with respect to its own code, and what it does is ridiculously complex;
look at the beginning of @file{asdf.lisp} to see what it does.
@end defun
@defun register-preloaded-system name @Arest@ keys
A system with name @var{name},
created by @code{make-instance} with extra keys @var{keys}
(e.g. @code{:version}),
is registered as @emph{preloaded}.
That is, its code has already been loaded into the current image,
and if at some point some other system @code{:depends-on} it yet no source code is found,
it is considered as already provided,
and ASDF will not raise a @code{missing-component} error.
This function is particularly useful if you distribute your code
as fasls with either @code{fasl-op} or @code{monolithic-fasl-op},
and want to register systems so that dependencies will work uniformly
whether you're using your software from source or from fasl.
@end defun
@defun run-shell-command control-string @Arest@ args
This function is obsolete and present only for the sake of backwards-compatibility:
``If it's not backwards, it's not compatible''. We @emph{strongly} discourage its use.
Its current behavior is only well-defined on Unix platforms
(which include MacOS X and cygwin). On Windows, anything goes.
The following documentation is only for the purpose of your migrating away from it
in a way that preserves semantics.
Instead we recommend the use @code{run-program}, described in the next section, and
available as part of ASDF since ASDF 3.
@code{run-shell-command} takes as arguments a format @code{control-string}
and arguments to be passed to @code{format} after this control-string
to produce a string.
This string is a command that will be evaluated with a POSIX shell if possible;
yet, on Windows, some implementations will use CMD.EXE,
while others (like SBCL) will make an attempt at invoking a POSIX shell
(and fail if it is not present).
@end defun
@node Some Utility Functions, , Controlling source file character encoding, Miscellaneous additional functionality
@section Some Utility Functions
The below functions are not exported by ASDF itself, but by UIOP, available since ASDF 3.
Some of them have precursors in ASDF 2, but we recommend
you rely on ASDF 3 for active developments.
UIOP provides many, many more utility functions, and we recommend
you read its README and sources for more information.
@defun parse-unix-namestring name @Akey@ type defaults dot-dot ensure-directory @AallowOtherKeys
Coerce NAME into a PATHNAME using standard Unix syntax.
Unix syntax is used whether or not the underlying system is Unix;
on non-Unix systems it is only usable for relative pathnames.
In order to manipulate relative pathnames portably, it is crucial
to possess a portable pathname syntax independent of the underlying OS.
This is what @code{parse-unix-namestring} provides, and why we use it in ASDF.
When given a @code{pathname} object, just return it untouched.
When given @code{nil}, just return @code{nil}.
When given a non-null @code{symbol}, first downcase its name and treat it as a string.
When given a @code{string}, portably decompose it into a pathname as below.
@code{#\/} separates directory components.
The last @code{#\/}-separated substring is interpreted as follows:
1- If @var{type} is @code{:directory} or @var{ensure-directory} is true,
the string is made the last directory component, and its @code{name} and @code{type} are @code{nil}.
if the string is empty, it's the empty pathname with all slots @code{nil}.
2- If @var{type} is @code{nil}, the substring is a file-namestring,
and its @code{name} and @code{type} are separated by @code{split-name-type}.
3- If @var{type} is a string, it is the given @code{type}, and the whole string is the @code{name}.
Directory components with an empty name the name @code{.} are removed.
Any directory named @code{..} is read as @var{dot-dot},
which must be one of @code{:back} or @code{:up} and defaults to @code{:back}.
@code{host}, @code{device} and @code{version} components are taken from @var{defaults},
which itself defaults to @code{*nil-pathname*}.
@code{*nil-pathname*} is also used if @var{defaults} is @code{nil}.
No host or device can be specified in the string itself,
which makes it unsuitable for absolute pathnames outside Unix.
For relative pathnames, these components (and hence the defaults) won't matter
if you use @code{merge-pathnames*} but will matter if you use @code{merge-pathnames},
which is an important reason to always use @code{merge-pathnames*}.
Arbitrary keys are accepted, and the parse result is passed to @code{ensure-pathname}
with those keys, removing @var{type}, @var{defaults} and @var{dot-dot}.
When you're manipulating pathnames that are supposed to make sense portably
even though the OS may not be Unixish, we recommend you use @code{:want-relative t}
so that @code{parse-unix-namestring} will throw an error if the pathname is absolute.
@defun merge-pathnames* specified @Aoptional@ defaults
This function is a replacement for @code{merge-pathnames} that uses the host and device
from the @var{defaults} rather than the @var{specified} pathname when the latter
is a relative pathname. This allows ASDF and its users to create and use relative pathnames
without having to know beforehand what are the host and device
of the absolute pathnames they are relative to.
@end defun
@defun subpathname pathname subpath @Akey@ type
This function takes a @var{pathname} and a @var{subpath} and a @var{type}.
If @var{subpath} is already a @code{pathname} object (not namestring),
and is an absolute pathname at that, it is returned unchanged;
otherwise, @var{subpath} is turned into a relative pathname with given @var{type}
as per @code{parse-unix-namestring} with @code{:want-relative t :type }@var{type},
then it is merged with the @code{pathname-directory-pathname} of @var{pathname},
as per @code{merge-pathnames*}.
We strongly encourage the use of this function
for portably resolving relative pathnames in your code base.
@end defun
@defun subpathname* pathname subpath @Akey@ type
This function returns @code{nil} if the base @var{pathname} is @code{nil},
otherwise acts like @code{subpathname}.
@end defun
@defun run-program command @Akey@ ignore-error-status force-shell input output @
error-output if-input-does-not-exist if-output-exists if-error-output-exists @
element-type external-format @AallowOtherKeys
@code{run-program} takes a @var{command} argument that is either
a list of a program name or path and its arguments,
or a string to be executed by a shell.
It spawns the command, waits for it to return,
verifies that it exited cleanly (unless told not too below),
and optionally captures and processes its output.
It accepts many keyword arguments to configure its behavior.
@code{run-program} returns three values: the first for the output,
the second for the error-output, and the third for the return value.
(Beware that before ASDF 3.0.2.11, it didn't handle input or error-output,
and returned only one value,
the one for the output if any handler was specified, or else the exit code;
please upgrade ASDF, or at least UIOP, to rely on the new enhanced behavior.)
@var{output} is its most important argument;
it specifies how the output is captured and processed.
4304
4305
4306
4307
4308
4309
4310
4311
4312
4313
4314
4315
4316
4317
4318
4319
4320
4321
4322
4323
4324
4325
4326
4327
4328
4329
4330
4331
4332
4333
4334
4335
If it is @code{nil}, then the output is redirected to the null device,
that will discard it.
If it is @code{:interactive}, then it is inherited from the current process
(beware: this may be different from your @var{*standard-output*},
and under SLIME will be on your @code{*inferior-lisp*} buffer).
If it is @code{t}, output goes to your current @var{*standard-output*} stream.
Otherwise, @var{output} should be a value that is a suitable first argument to
@code{slurp-input-stream} (see below), or
a list of such a value and keyword arguments.
In this case, @code{run-program} will
create a temporary stream for the program output;
the program output, in that stream,
will be processed by a call to @code{slurp-input-stream},
using @var{output} as the first argument
(or if it's a list the first element of @var{output} and the rest as keywords).
The primary value resulting from that call
(or @code{nil} if no call was needed)
will be the first value returned by @code{run-program}.
E.g., using @code{:output :string}
will have it return the entire output stream as a string.
And using @code{:output '(:string :stripped t)}
will have it return the same string stripped of any ending newline.
@var{error-output} is similar to @var{output}, except that
the resulting value is returned as the second value of @code{run-program}.
@code{t} designates the @var{*error-output*}.
Also @code{:output} means redirecting the error output to the output stream,
in which case @code{nil} is returned.
@var{input} is similar to @var{output}, except that
@code{vomit-output-stream} is used, no value is returned,
and @code{t} designates the @var{*standard-input*}.
@code{element-type} and @code{external-format} are passed on
to your Lisp implementation, when applicable, for creation of the output stream.
One and only one of the stream slurping or vomiting may or may not happen
in parallel in parallel with the subprocess,
depending on options and implementation,
and with priority being given to output processing.
Other streams are completely produced or consumed
before or after the subprocess is spawned, using temporary files.
@code{force-shell} forces evaluation of the command through a shell,
even if it was passed as a list rather than a string.
If a shell is used, it is @file{/bin/sh} on Unix or @file{CMD.EXE} on Windows,
except on implementations that (erroneously, IMNSHO)
insist on consulting @code{$SHELL} like clisp.
@code{ignore-error-status} causes @code{run-program}
to not raise an error if the spawned program exits in error.
Following POSIX convention, an error is anything but
a normal exit with status code zero.
By default, an error of type @code{subprocess-error} is raised in this case.
@code{run-program} works on all platforms supported by ASDF, except Genera.
See the source code for more documentation.
@end defun
@defun slurp-input-stream processor input-stream @Akey@
@code{slurp-input-stream} is a generic function of two arguments, a target object and an input stream,
and accepting keyword arguments.
Predefined methods based on the target object are as follows:
If the object is a function, the function is called with the stream as argument.
@item If the object is a cons, its first element is applied to its rest appended by
a list of the input stream.
@item If the object is an output stream, the contents of the input stream are copied to it.
If the @var{linewise} keyword argument is provided, copying happens line by line,
and an optional @var{prefix} is printed before each line.
Otherwise, copying happen based on a buffer of size @var{buffer-size},
using the specified @var{element-type}.
@item If the object is @code{'string} or @code{:string}, the content is captured into a string.
Accepted keywords include the @var{element-type} and a flag @var{stripped},
which when true causes any single line ending to be removed as per @code{uiop:stripln}.
@item If the object is @code{:lines}, the content is captured as a list of strings,
one per line, without line ending. If the @var{count} keyword argument is provided,
it is a maximum count of lines to be read.
@item If the object is @code{:line}, the content is captured as with @code{:lines} above,
and then its sub-object is extracted with the @var{at} argument,
which defaults to @code{0}, extracting the first line.
A number will extract the corresponding line.
See the documentation for @code{uiop:access-at}.
@item If the object is @code{:forms}, the content is captured as a list of S-expressions,
as read by the Lisp reader.
If the @var{count} argument is provided,
it is a maximum count of lines to be read.
We recommend you control the syntax with such macro as
@code{uiop:with-safe-io-syntax}.
@item If the object is @code{:form}, the content is captured as with @code{:forms} above,
and then its sub-object is extracted with the @var{at} argument,
which defaults to @code{0}, extracting the first form.
A number will extract the corresponding form.
See the documentation for @code{uiop:access-at}.
We recommend you control the syntax with such macro as
@code{uiop:with-safe-io-syntax}.
@end defun
@node Getting the latest version, FAQ, Miscellaneous additional functionality, Top
@comment node-name, next, previous, up
@chapter Getting the latest version
Decide which version you want.
The @code{master} branch is where development happens;
its @code{HEAD} is usually OK, including the latest fixes and portability tweaks,
but an occasional regression may happen despite our (limited) test suite.
The @code{release} branch is what cautious people should be using;
it has usually been tested more, and releases are cut at a point
where there isn't any known unresolved issue.
You may get the ASDF source repository using git:
@kbd{git clone git://common-lisp.net/projects/asdf/asdf.git}
You will find the above referenced tags in this repository.
You can also browse the repository on
@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf.git}.
Discussion of ASDF development is conducted on the
mailing list
@kbd{asdf-devel@@common-lisp.net}.
@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel}
Francois-Rene Rideau
committed
@node FAQ, Ongoing Work, Getting the latest version, Top
@comment node-name, next, previous, up
@menu
* Where do I report a bug?::
* What has changed between ASDF 1 and ASDF 2?::
* Issues with installing the proper version of ASDF::
* Issues with configuring ASDF::
* Issues with using and extending ASDF to define systems::
* ASDF development FAQs::
@end menu
@node Where do I report a bug?, What has changed between ASDF 1 and ASDF 2?, FAQ, FAQ
ASDF bugs are tracked on launchpad: @url{https://launchpad.net/asdf}.
If you're unsure about whether something is a bug, or for general discussion,
use the @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
@node What has changed between ASDF 1 and ASDF 2?, Issues with installing the proper version of ASDF, Where do I report a bug?, FAQ
4464
4465
4466
4467
4468
4469
4470
4471
4472
4473
4474
4475
4476
4477
4478
4479
4480
4481
4482
4483
4484
4485
4486
@section ``What has changed between ASDF 1, ASDF 2 and ASDF 3?''
We released ASDF 2.000 on May 31st 2010,
and ASDF 3.0.0 on May 15th 2013.
Releases of ASDF 2 and later have since then been included
in all actively maintained CL implementations that used to bundle ASDF 1,
plus some implementations that previously did not.
ASDF has been made to work with all actively maintained CL
implementations and even a few implementations that are @emph{not}
actively maintained.
@xref{FAQ,,``What has changed between ASDF 1 and ASDF 2?''}.
Furthermore, it is possible to upgrade from ASDF 1 to ASDF 2 or ASDF 3 on the fly
(though we recommend instead upgrading your implementation or its ASDF module).
For this reason, we have stopped supporting ASDF 1 and ASDF 2.
If you are using ASDF 1 or ASDF 2 and are experiencing any kind of issues or limitations,
we recommend you upgrade to ASDF 3
--- and we explain how to do that. @xref{Loading ASDF}.
(In the context of compatibility requirements,
ASDF 2.27, released on Feb 1st 2013, and further 2.x releases up to 2.33,
count as pre-releases of ASDF 3, and define the @code{:asdf3} feature;
still, please use the latest release).
Release ASDF 3.1.1 and later also define the @code{:asdf3.1} feature.
@menu
* What are ASDF 1 2 3?::
* How do I detect the ASDF version?::
* ASDF can portably name files in subdirectories::
* Output translations::
* Source Registry Configuration::
* Usual operations are made easier to the user::
* Many bugs have been fixed::
* ASDF itself is versioned::
* ASDF can be upgraded::
* Decoupled release cycle::
* Pitfalls of the transition to ASDF 2::
@end menu
@node What are ASDF 1 2 3?, How do I detect the ASDF version?, What has changed between ASDF 1 and ASDF 2?, What has changed between ASDF 1 and ASDF 2?
@subsection What are ASDF 1, ASDF 2, and ASDF 3?
Francois-Rene Rideau
committed
ASDF 1 refers to any release earlier than 1.369 or so (from August 2001 to October 2009),
and to any development revision earlier than 2.000 (May 2010).
If your copy of ASDF doesn't even contain version information, it's an old ASDF 1.
Revisions between 1.656 and 1.728 may count as development releases for ASDF 2.
ASDF 2 refers to releases from 2.000 (May 31st 2010) to 2.26 (Oct 30 2012),
and any development revision newer than ASDF 1 and older than 2.27 (Feb 1 2013).
Francois-Rene Rideau
committed
ASDF 3 refers to releases from 2.27 (Feb 1 2013) to 2.33 and 3.0.0 onward (May 15 2013).
2.27 to 2.33 count as pre-releases to ASDF 3.
@node How do I detect the ASDF version?, ASDF can portably name files in subdirectories, What are ASDF 1 2 3?, What has changed between ASDF 1 and ASDF 2?
@subsection How do I detect the ASDF version?
@findex asdf-version
@cindex *features*
Francois-Rene Rideau
committed
All releases of ASDF
push @code{:asdf} onto @code{*features*}.
Releases starting with ASDF 2
push @code{:asdf2} onto @code{*features*}.
Releases starting with ASDF 3 (including 2.27 and later pre-releases)
push @code{:asdf3} onto @code{*features*}.
Furthermore, releases starting with ASDF 3.1 (March 2014),
though they count as ASDF 3, includes enough progress that they
push @code{:asdf3.1} onto @code{*features*}.
You may depend on the presence or absence of these features
to write code that takes advantage of recent ASDF functionality
but still works on older versions, or at least detects the old version and signals an error.
Francois-Rene Rideau
committed
Additionally, all releases starting with ASDF 2
define a function @code{(asdf:asdf-version)} you may use to query the version.
All releases starting with 2.013 display the version number prominently
on the second line of the @file{asdf.lisp} source file.
Francois-Rene Rideau
committed
If you are experiencing problems or limitations of any sort with ASDF 1 or ASDF 2,
we recommend that you should upgrade to the latest release, be it ASDF 3 or other.
@node ASDF can portably name files in subdirectories, Output translations, How do I detect the ASDF version?, What has changed between ASDF 1 and ASDF 2?
@subsection ASDF can portably name files in subdirectories
Common Lisp namestrings are not portable,
except maybe for logical pathnamestrings,
that themselves have various limitations and require a lot of setup
that is itself ultimately non-portable.
In ASDF 1, the only portable ways to refer to pathnames inside systems and components
were very awkward, using @code{#.(make-pathname ...)} and
@code{#.(merge-pathnames ...)}.
Even the above were themselves were inadequate in the general case
due to host and device issues, unless horribly complex patterns were used.
Plenty of simple cases that looked portable actually weren't,
leading to much confusion and greavance.
ASDF 2 implements its own portable syntax for strings as pathname specifiers.
Naming files within a system definition becomes easy and portable again.
@xref{Miscellaneous additional functionality,system-relative-pathname},
@code{merge-pathnames*},
@code{coerce-pathname}.
Francois-Rene Rideau
committed
On the other hand, there are places where systems used to accept namestrings
where you must now use an explicit pathname object:
@code{(defsystem ... :pathname "LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
must now be written with the @code{#p} syntax:
@code{(defsystem ... :pathname #p"LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
@xref{The defsystem grammar,,Pathname specifiers}.
@node Output translations, Source Registry Configuration, ASDF can portably name files in subdirectories, What has changed between ASDF 1 and ASDF 2?
A popular feature added to ASDF was output pathname translation:
@code{asdf-binary-locations}, @code{common-lisp-controller},
@code{cl-launch} and other hacks were all implementing it in ways
both mutually incompatible and difficult to configure.
Output pathname translation is essential to share
source directories of portable systems across multiple implementations
or variants thereof,
or source directories of shared installations of systems across multiple users,
or combinations of the above.
In ASDF 2, a standard mechanism is provided for that,
@code{asdf-output-translations},
with sensible defaults, adequate configuration languages,
a coherent set of configuration files and hooks,
and support for non-Unix platforms.
@xref{Controlling where ASDF saves compiled files}.
@node Source Registry Configuration, Usual operations are made easier to the user, Output translations, What has changed between ASDF 1 and ASDF 2?
@subsection Source Registry Configuration
Configuring ASDF used to require special magic
to be applied just at the right moment,
between the moment ASDF is loaded and the moment it is used,
in a way that is specific to the user,
the implementation he is using and the application he is building.
This made for awkward configuration files and startup scripts
that could not be shared between users, managed by administrators
or packaged by distributions.
ASDF 2 provides a well-documented way to configure ASDF,
with sensible defaults, adequate configuration languages,
and a coherent set of configuration files and hooks.
We believe it's a vast improvement because it decouples
application distribution from library distribution.
The application writer can avoid thinking where the libraries are,
and the library distributor (dpkg, clbuild, advanced user, etc.)
can configure them once and for every application.
Yet settings can be easily overridden where needed,
so whoever needs control has exactly as much as required.
At the same time, ASDF 2 remains compatible
with the old magic you may have in your build scripts
(using @code{*central-registry*} and
@code{*system-definition-search-functions*})
to tailor the ASDF configuration to your build automation needs,
and also allows for new magic, simpler and more powerful magic.
@xref{Controlling where ASDF searches for systems}.
@node Usual operations are made easier to the user, Many bugs have been fixed, Source Registry Configuration, What has changed between ASDF 1 and ASDF 2?
@subsection Usual operations are made easier to the user
In ASDF 1, you had to use the awkward syntax
@code{(asdf:oos 'asdf:load-op :foo)}
to load a system,
and similarly for @code{compile-op}, @code{test-op}.
In ASDF 2, you can use shortcuts for the usual operations:
@code{(asdf:load-system :foo)}, and
similarly for @code{compile-system}, @code{test-system}.
@node Many bugs have been fixed, ASDF itself is versioned, Usual operations are made easier to the user, What has changed between ASDF 1 and ASDF 2?
@subsection Many bugs have been fixed
The following issues and many others have been fixed:
The infamous TRAVERSE function has been revamped completely
between ASDF 1 and ASDF 2, with many bugs squashed.
In particular, dependencies were not correctly propagated
across modules but now are.
It has been completely rewritten many times over
Francois-Rene Rideau
committed
between ASDF 2.000 and ASDF 3,
with fundamental issues in the original model being fixed.
Timestamps were not propagated at all, and now are.
The internal model of how actions depend on each other
is now both consistent and complete.
The @code{:version} and
the @code{:force (system1 .. systemN)} feature have been fixed.
Francois-Rene Rideau
committed
Performance has been notably improved for large systems
(say with thousands of components) by using
hash-tables instead of linear search,
and linear-time list accumulation
instead of quadratic-time recursive appends.
@item
Many features used to not be portable,
especially where pathnames were involved.
Windows support was notably quirky because of such non-portability.
@item
The internal test suite used to massively fail on many implementations.
While still incomplete, it now fully passes
on all implementations supported by the test suite,
except for GCL (due to GCL bugs).
@item
Support was lacking for some implementations.
ABCL and GCL were notably wholly broken.
ECL extensions were not integrated with ASDF release.
The documentation was grossly out of date.
@node ASDF itself is versioned, ASDF can be upgraded, Many bugs have been fixed, What has changed between ASDF 1 and ASDF 2?
@subsection ASDF itself is versioned
Between new features, old bugs fixed, and new bugs introduced,
there were various releases of ASDF in the wild,
and no simple way to check which release had which feature set.
People using or writing systems had to either make worst-case assumptions
as to what features were available and worked,
or take great pains to have the correct version of ASDF installed.
With ASDF 2, we provide a new stable set of working features
that everyone can rely on from now on.
Use @code{#+asdf2} to detect presence of ASDF 2,
@code{(asdf:version-satisfies (asdf:asdf-version) "2.345.67")}
to check the availability of a version no earlier than required.
@node ASDF can be upgraded, Decoupled release cycle, ASDF itself is versioned, What has changed between ASDF 1 and ASDF 2?
@subsection ASDF can be upgraded
When an old version of ASDF was loaded,
it was very hard to upgrade ASDF in your current image
without breaking everything.
Instead you had to exit the Lisp process and
somehow arrange to start a new one from a simpler image.
Something that can't be done from within Lisp,
making automation of it difficult,
which compounded with difficulty in configuration,
made the task quite hard.
Yet as we saw before, the task would have been required
to not have to live with the worst case or non-portable
subset of ASDF features.
With ASDF 2, it is easy to upgrade
from ASDF 2 to later versions from within Lisp,
and not too hard to upgrade from ASDF 1 to ASDF 2 from within Lisp.
We support hot upgrade of ASDF and any breakage is a bug
that we will do our best to fix.
There are still limitations on upgrade, though,
most notably the fact that after you upgrade ASDF,
you must also reload or upgrade all ASDF extensions.
@node Decoupled release cycle, Pitfalls of the transition to ASDF 2, ASDF can be upgraded, What has changed between ASDF 1 and ASDF 2?
@subsection Decoupled release cycle
When vendors were releasing their Lisp implementations with ASDF,
they had to basically never change version
because neither upgrade nor downgrade was possible
without breaking something for someone,
and no obvious upgrade path was visible and recommendable.
With ASDF 2, upgrade is possible, easy and can be recommended.
This means that vendors can safely ship a recent version of ASDF,
confident that if a user isn't fully satisfied,
he can easily upgrade ASDF and deal
with a supported recent version of it.
This means that release cycles will be causally decoupled,
the practical consequence of which will mean faster convergence
towards the latest version for everyone.
@node Pitfalls of the transition to ASDF 2, , Decoupled release cycle, What has changed between ASDF 1 and ASDF 2?
Francois-Rene Rideau
committed
@subsection Pitfalls of the transition to ASDF 2
The main pitfalls in upgrading to ASDF 2 seem to be related
to the output translation mechanism.
@itemize
@item
Output translations is enabled by default. This may surprise some users,
most of them in pleasant way (we hope), a few of them in an unpleasant way.
It is trivial to disable output translations.
@xref{FAQ,,``How can I wholly disable the compiler output cache?''}.
@item
Some systems in the large have been known
not to play well with output translations.
They were relatively easy to fix.
Once again, it is also easy to disable output translations,
or to override its configuration.
@item
The new ASDF output translations are incompatible with ASDF-Binary-Locations.
They replace A-B-L, and there is compatibility mode to emulate
your previous A-B-L configuration.
Francois-Rene Rideau
committed
See @code{enable-asdf-binary-locations-compatibility} in
@pxref{Controlling where ASDF saves compiled files,,Backward Compatibility}.
But thou shalt not load ABL on top of ASDF 2.
Francois-Rene Rideau
committed
@end itemize
Other issues include the following:
@itemize
Francois-Rene Rideau
committed
@item
ASDF pathname designators are now specified
in places where they were unspecified,
Francois-Rene Rideau
committed
and a few small adjustments have to be made to some non-portable defsystems.
Notably, in the @code{:pathname} argument
to a @code{defsystem} and its components,
Francois-Rene Rideau
committed
a logical pathname (or implementation-dependent hierarchical pathname)
must now be specified with @code{#p} syntax
where the namestring might have previously sufficed;
moreover when evaluation is desired @code{#.} must be used,
where it wasn't necessary in the toplevel @code{:pathname} argument
(but necessary in other @code{:pathname} arguments).
Francois-Rene Rideau
committed
@item
There is a slight performance bug, notably on SBCL,
when initially searching for @file{asd} files,
the implicit @code{(directory "/configured/path/**/*.asd")}
for every configured path @code{(:tree "/configured/path/")}
in your @code{source-registry} configuration can cause a slight pause.
Try to @code{(time (asdf:initialize-source-registry))}
to see how bad it is or isn't on your system.
If you insist on not having this pause,
you can avoid the pause by overriding the default source-registry configuration
and not use any deep @code{:tree} entry but only @code{:directory} entries
or shallow @code{:tree} entries.
Or you can fix your implementation to not be quite that slow
when recursing through directories.
@emph{Update}: This performance bug fixed the hard way in 2.010.
@item
On Windows, only LispWorks supports proper default configuration pathnames
based on the Windows registry.
Other implementations make do with environment variables,
that you may have to define yourself
if you're using an older version of Windows.
Francois-Rene Rideau
committed
Windows support is somewhat less tested than Unix support.
Please help report and fix bugs.
@emph{Update}: As of ASDF 2.21, all implementations
should now use the same proper default configuration pathnames
and they should actually work, though they haven't all been tested.
Francois-Rene Rideau
committed
@item
The mechanism by which one customizes a system so that Lisp files
may use a different extension from the default @file{.lisp} has changed.
Previously, the pathname for a component
was lazily computed when operating on a system,
Francois-Rene Rideau
committed
and you would
@code{(defmethod source-file-type ((component cl-source-file) (system (eql (find-system 'foo))))
(declare (ignorable component system)) "lis")}.
Francois-Rene Rideau
committed
Now, the pathname for a component is eagerly computed when defining the system,
Francois-Rene Rideau
committed
and instead you will @code{(defclass cl-source-file.lis (cl-source-file) ((type :initform "lis")))}
and use @code{:default-component-class cl-source-file.lis}
as argument to @code{defsystem},
Francois-Rene Rideau
committed
as detailed in a @pxref{FAQ,How do I create a system definition where all the source files have a .cl extension?} below.
@findex source-file-type
@node Issues with installing the proper version of ASDF, Issues with configuring ASDF, What has changed between ASDF 1 and ASDF 2?, FAQ
@section Issues with installing the proper version of ASDF
@menu
* My Common Lisp implementation comes with an outdated version of ASDF. What to do?::
* I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?::
@end menu
@node My Common Lisp implementation comes with an outdated version of ASDF. What to do?, I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?, Issues with installing the proper version of ASDF, Issues with installing the proper version of ASDF
@subsection ``My Common Lisp implementation comes with an outdated version of ASDF. What to do?''
We recommend you upgrade ASDF.
@xref{Loading ASDF,,Upgrading ASDF}.
If this does not work, it is a bug, and you should report it.
@xref{FAQ, report-bugs, Where do I report a bug}.
In the meantime, you can load @file{asdf.lisp} directly.
@xref{Loading ASDF,Loading an otherwise installed ASDF}.
@node I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?, , My Common Lisp implementation comes with an outdated version of ASDF. What to do?, Issues with installing the proper version of ASDF
@subsection ``I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?''
Francois-Rene Rideau
committed
Since ASDF 2,
it should always be a good time to upgrade to a recent version of ASDF.
You may consult with the maintainer for which specific version they recommend,
but the latest @code{release} should be correct.
Though we do try to test ASDF releases against all implementations that we can,
we may not be testing against all variants of your implementation,
Francois-Rene Rideau
committed
and we may not be running enough tests;
we trust you to thoroughly test it with your own implementation
before you release it.
If there are any issues with the current release,
it's a bug that you should report upstream and that we will fix ASAP.
Francois-Rene Rideau
committed
As to how to include ASDF, we recommend the following:
Francois-Rene Rideau
committed
@itemize
@item
If ASDF isn't loaded yet, then @code{(require "asdf")}
Francois-Rene Rideau
committed
should load the version of ASDF that is bundled with your system.
If possible so should @code{(require "ASDF")}.
Francois-Rene Rideau
committed
You may have it load some other version configured by the user,
if you allow such configuration.
@item
If your system provides a mechanism to hook into @code{CL:REQUIRE},
then it would be nice to add ASDF to this hook the same way that
ABCL, CCL, CLISP, CMUCL, ECL, SBCL and SCL do it.
Please send us appropriate code to this end.
Francois-Rene Rideau
committed
@item
You may, like SBCL, have ASDF be implicitly used to require systems
that are bundled with your Lisp distribution.
If you do have a few magic systems that come with your implementation
in a precompiled way such that one should only use the binary version
that goes with your distribution, like SBCL does,
then you should add them in the beginning of @code{wrapping-source-registry}.
@item
If you have magic systems as above, like SBCL does,
then we explicitly ask you to @emph{NOT} distribute
@file{asdf.asd} as part of those magic systems.
You should still include the file @file{asdf.lisp} in your source distribution
and precompile it in your binary distribution,
but @file{asdf.asd} if included at all,
should be secluded from the magic systems,
in a separate file hierarchy.
Alternatively, you may provide the system
after renaming it and its @file{.asd} file to e.g.
Francois-Rene Rideau
committed
@code{asdf-ecl} and @file{asdf-ecl.asd}, or
@code{sb-asdf} and @file{sb-asdf.asd}.
Indeed, if you made @file{asdf.asd} a magic system,
then users would no longer be able to upgrade ASDF using ASDF itself
to some version of their preference that
they maintain independently from your Lisp distribution.
@item
If you do not have any such magic systems, or have other non-magic systems
that you want to bundle with your implementation,
then you may add them to the @code{wrapping-source-registry},
and you are welcome to include @file{asdf.asd} amongst them.
Non-magic systems should be at the back of the @code{wrapping-source-registry}
while magic systems are at the front.
@item
Since ASDF 3, the library UIOP comes transcluded in ASDF.
Francois-Rene Rideau
committed
But if you want to be nice to users who care for UIOP but not for ASDF,
you may package UIOP separately,
so that one may @code{(require "uiop")} and not load ASDF,
Francois-Rene Rideau
committed
which would implicitly require and load the former.
Francois-Rene Rideau
committed
@item
Please send us upstream any patches you make to ASDF itself,
so we can merge them back in for the benefit of your users
when they upgrade to the upstream version.
Francois-Rene Rideau
committed
@end itemize
@node Issues with configuring ASDF, Issues with using and extending ASDF to define systems, Issues with installing the proper version of ASDF, FAQ
@section Issues with configuring ASDF
@menu
* How can I customize where fasl files are stored?::
* How can I wholly disable the compiler output cache?::
@end menu
@node How can I customize where fasl files are stored?, How can I wholly disable the compiler output cache?, Issues with configuring ASDF, Issues with configuring ASDF
@subsection ``How can I customize where fasl files are stored?''
@xref{Controlling where ASDF saves compiled files}.
Note that in the past there was an add-on to ASDF called
@code{ASDF-binary-locations}, developed by Gary King.
That add-on has been merged into ASDF proper,
Francois-Rene Rideau
committed
then superseded by the @code{asdf-output-translations} facility.
Note that use of @code{asdf-output-translations}
can interfere with one aspect of your systems
--- if your system uses @code{*load-truename*} to find files
(e.g., if you have some data files stored with your program),
then the relocation that this ASDF customization performs
is likely to interfere.
Use @code{asdf:system-relative-pathname} to locate a file
in the source directory of some system, and
use @code{asdf:apply-output-translations} to locate a file
whose pathname has been translated by the facility.
@node How can I wholly disable the compiler output cache?, , How can I customize where fasl files are stored?, Issues with configuring ASDF
@subsection ``How can I wholly disable the compiler output cache?''
To permanently disable the compiler output cache
for all future runs of ASDF, you can:
@example
mkdir -p ~/.config/common-lisp/asdf-output-translations.conf.d/
echo ':disable-cache' > ~/.config/common-lisp/asdf-output-translations.conf.d/99-disable-cache.conf
@end example
This assumes that you didn't otherwise configure the ASDF files
(if you did, edit them again),
and don't somehow override the configuration at runtime
with a shell variable (see below) or some other runtime command
(e.g. some call to @code{asdf:initialize-output-translations}).
To disable the compiler output cache in Lisp processes
Francois-Rene Rideau
committed
run by your current shell, try (assuming @code{bash} or @code{zsh})
(on Unix and cygwin only):