...
 
Commits (74)
......@@ -49,6 +49,7 @@ test/try-reloading-dependency.asd
test/fileMissing.lisp
test/hello-world-example
test/test-include.asd
test/test-multiple-too.asd
build-stamp
......@@ -67,5 +68,7 @@ debian/files
/.DS_Store
core
# Hack: ignore the GNUmakefile here, so you can ln -s Makefile-lisp-scripting GNUmakefile
GNUmakefile
......@@ -4,10 +4,13 @@ ASDF: Another System Definition Facility
For general information about ASDF, consult the web page:
<https://common-lisp.net/project/asdf/>
For instructions on how to use it, read the manual:
For some reference documentation, read the manual:
<https://common-lisp.net/project/asdf/asdf.html>
Below is a guide for ASDF developers; it is not meant for ASDF users.
For a guide on how to use it, read our "best practices" document:
<https://github.com/fare/asdf/blob/master/doc/best_practices.md>
Below is a guide for ASDF developers. It is not meant for ASDF users.
[TOC]
......@@ -198,7 +201,8 @@ How do I navigate this source tree?
* [uiop/](uiop/)
* Utilities of Implementation- and OS- Portability,
the portability layer of ASDF. It has its own [README](uiop/README.md),
and functions all have docstrings.
and exported functions should all have docstrings and other ones comment,
or once again it's a bug.
* [Makefile](Makefile)
* The classical `Makefile` used for development purposes.
......@@ -209,9 +213,9 @@ How do I navigate this source tree?
* [bin/](bin/)
* [bump-version](bin/bump-version) --
a script to bump the version of ASDF, used by the classic `Makefile`.
Use it with e.g. `./bin/bump-version 3.3.0`
Use it with e.g. `./bin/bump-version 3.4.5`
to test with the next version number before you release.
NB: ASDF's version number notably affect the behavior of ASDF
NB: ASDF's version number notably affects the behavior of ASDF
with respect to deprecated functions.
* [tools/](tools/)
......@@ -224,6 +228,7 @@ How do I navigate this source tree?
system definition for asdf-tools
* `*.lisp` -- the source code for the `asdf-tools` system,
except for the few files below.
Check the `.asd` file for the order in which to read them.
* Also a couple scripts to help ASDF users:
* [load-asdf.lisp](tools/load-asdf.lisp) --
a working example script to load, configure and use ASDF
......@@ -284,7 +289,7 @@ How do I navigate this source tree?
* External dependencies, that can be populated with `make ext`
or equivalently with `git submodule update --init`.
Depopulate it with `make noext`
or equivalently with: `submodule deinit .`
or equivalently with: `submodule deinit .`
* [README.md](README.md)
* This file.
......
......@@ -10,13 +10,15 @@
** Maybe remove old versions from upgrade tests; or maybe not:
if we support old versions in any way,
it should be for upgrade only, and "punt" is acceptable.
** Maybe remove upgrade from antique ASDF altogether.
Clear all the code in the header, and/or
move a few elements to asdf/upgrade.
** Upgrade tests might be split in two test sets: short and long.
** Make tests stateless and/or make any modifications private,
so that they can be run in parallel.
* Have a better story for linking C static or dynamic libraries.
** Some groundwork has been done in cffi-toolchain, but an interface must be designed.
** Then you'll have to migrate all the code in Quicklisp to use your new interface.
** See commonqt's qt.asd and qtools's qt-libs.asd for instances of ugliness.
* Testing support
** Replace test-op with run-test-op and test-report-op ?
......@@ -122,6 +124,7 @@
rather than `#(1 2 3 2 3 2 3 2 3)` or even better `#(1 2 3 3 3)`.
*** It purports to support Unicode, but it has only 16-bit characters.
Note that (subtypep 'character 'base-char) says T -- everything is a base char(!)
*** In general, running upgrade tests triggers massive CLOS breakage on CMUCL.
** SCL has bugs:
*** it doesn't like run-program from a modified directory?
......@@ -222,12 +225,6 @@
* Learn to use cl-grid-test
** So we can easily make sure ASDF changes don't break stuff,
and that breakage gets fixed quickly.
** So we see if ASDF-USER can :use most (all?) of UIOP.
SB-GROVEL used (until 25c1769 from 2013-02-23) to :use both ASDF and
SB-EXT, which prevents ASDF from exporting RUN-PROGRAM, but
should not be a problem with ASDF-USER :use'ing UIOP.
A run of quicklisp is necessary, with an eye also for warnings
from .asd's that might be overriding definitions in UIOP(!)
* Diagnostics
** have a mode to explain WHY a component needs to be recompiled.
......@@ -244,39 +241,18 @@
* have a single test .asd that tests as many features as possible, use it for upgrade test.
* Bug found by fe[nl]ix: infinite loop if the definitions in an asd file
are not in strict defsystem-depends-on dependency order.
** At least document the issue.
** Real solution: defer parsing and evaluation of defsystem forms?
Or simply detect circularity and issue an appropriate error?
* Include some ABL test for stassats's (now obsolete?) thing:
(asdf:enable-asdf-binary-locations-compatibility
:centralize-lisp-binaries t :default-toplevel-directory *fasl-dir*)
** Or deprecate and remove ABL compatibility altogether? Move it to contrib?
No one has been using ASDF1 for years.
There is no ABL to be compatible with —
is anyone still using the compatibility mode?
Maybe issue a warning, then a cerror, before to remove the code?
* Make load-op a generic operation that selects the proper strategy?
for each system, module or file, according to component properties and user-configuration:
compile the lisp file then load the fasl (load-fasl-op on a cl-source-file),
load the lisp file without compiling (load-source-op),
compile all files for the system, link them and load the result
(load-fasl-op on the system),
compile all files for the system *and* its dependencies, link and load (monolithic-load-fasl-op),
concatenate all files for the system, then load the result (load-concatenated-source-op),
concatenate all files for the system, then link them and load the result (load-compiled-concatenated-op),
concatenate all files for the system *and* its dependencies, link and load (monolithic-load-compiled-concatenated-op).
* operation cleanup?
** Kill backward-compat functions after all clients have moved on.
** Kill original-initargs -- BEWARE, it currently has clients!
*** current ECL users used to depend on it for bundles, and may or may
not be satisfied with how it's not propagated anymore.
*** SLIME uses it (!)
*** weblocks uses it (!)
** Refactor operation classes in action.lisp to not use slots for metadata
*** See commit message for 8e68ac229c19b7b611caf942a7f0998c88047a79
*** The problem: slots will NOT get properly upgraded with new initform.
*** Bad solution: A U-I-F-R-C method might do but would be much more complex to maintain,
not just for ASDF but potentially for anyone who defines extensions to ASDF.
*** The solution: use defmethod self-operation, not a self-operation slot.
This solution is trivially backward-compatible with older variants of ASDF.
*** Migration: find everyone on Quicklisp that defines an operation, and have them use defmethod.
Document all that in asdf.texinfo. Yet, unless noone is affected in Quicklisp,
don't modify ASDF classes themselves until a year or two afterwards, for compatibility purposes;
and/or when ASDF classes themselves need to be modified for the sake of some fix.
** To allow semantically distinct operations of the same class:
You'd need to have a protocol to canonicalize them
in the *OPERATIONS* memoization table, not by class name,
......@@ -461,8 +437,15 @@ a reified context for featurep checks, etc.
by having all functions and macros under the old package name be
proxies to those in the new package name?
(Exception: special variables can't be proxies, have to be imported.)
** Eradicate Quicklisp systems that depend on asdf-driver or
asdf-package-system, make them depend on uiop and asdf3.1 instead.
** Delete asdf-driver.asd, nobody has used it in years
** Eradicate Quicklisp systems that depend on asdf-package-system, make them depend on asdf3.1 instead.
** Deprecate and remove asdf-binary-locations (ABL) compatibility altogether.
No one has been using ASDF1 for years.
There is no ABL to be compatible with —
is anyone still using the compatibility mode?
Maybe issue a warning, then a cerror, before to remove the code?
* Fix plan
** Goal: Implement proper incrementality for defsystem-depends-on.
......
......@@ -84,7 +84,7 @@
:licence "MIT"
:description "Another System Definition Facility"
:long-description "ASDF builds Common Lisp software organized into defined systems."
:version "3.2.1" ;; to be automatically updated by make bump-version
:version "3.3.2.2" ;; to be automatically updated by make bump-version
:depends-on ()
#+asdf3 :encoding #+asdf3 :utf-8
:class #+asdf3.1 package-inferred-system #-asdf3.1 system
......
......@@ -22,12 +22,9 @@
(in-package :asdf/bundle)
(with-upgradability ()
(defclass bundle-op (operation)
;; NB: use of instance-allocated slots for operations is DEPRECATED
;; and only supported in a temporary fashion for backward compatibility.
;; Supported replacement: Define slots on program-system instead.
((bundle-type :initform :no-output-file :reader bundle-type :allocation :class))
(defclass bundle-op (operation) ()
(:documentation "base class for operations that bundle outputs from multiple components"))
(defgeneric bundle-type (bundle-op))
(defclass monolithic-op (operation) ()
(:documentation "A MONOLITHIC operation operates on a system *and all of its
......@@ -68,10 +65,11 @@ itself."))
(defclass link-op (bundle-op) ()
(:documentation "Abstract operation for linking files together"))
(defclass gather-operation (bundle-op)
((gather-operation :initform nil :allocation :class :reader gather-operation)
(gather-type :initform :no-output-file :allocation :class :reader gather-type))
(defclass gather-operation (bundle-op) ()
(:documentation "Abstract operation for gathering many input files from a system"))
(defgeneric gather-operation (gather-operation))
(defmethod gather-operation ((o gather-operation)) nil)
(defgeneric gather-type (gather-operation))
(defun operation-monolithic-p (op)
(typep op 'monolithic-op))
......@@ -108,11 +106,12 @@ itself."))
`((,go ,@deps) ,@(call-next-method))))
;; Create a single fasl for the entire library
(defclass basic-compile-bundle-op (bundle-op basic-compile-op)
((gather-type :initform #-(or clasp ecl mkcl) :fasl #+(or clasp ecl mkcl) :object
:allocation :class)
(bundle-type :initform :fasb :allocation :class))
(defclass basic-compile-bundle-op (bundle-op basic-compile-op) ()
(:documentation "Base class for compiling into a bundle"))
(defmethod bundle-type ((o basic-compile-bundle-op)) :fasb)
(defmethod gather-type ((o basic-compile-bundle-op))
#-(or clasp ecl mkcl) :fasl
#+(or clasp ecl mkcl) :object)
;; Analog to prepare-op, for load-bundle-op and compile-bundle-op
(defclass prepare-bundle-op (sideway-operation)
......@@ -121,9 +120,7 @@ itself."))
:allocation :class))
(:documentation "Operation class for loading the bundles of a system's dependencies"))
(defclass lib-op (link-op gather-operation non-propagating-operation)
((gather-type :initform :object :allocation :class)
(bundle-type :initform :lib :allocation :class))
(defclass lib-op (link-op gather-operation non-propagating-operation) ()
(:documentation "Compile the system and produce a linkable static library (.a/.lib)
for all the linkable object files associated with the system. Compare with DLL-OP.
......@@ -132,6 +129,8 @@ written in C or another language with a compiler producing linkable object files
On CLASP, ECL, MKCL, these object files _also_ include the contents of Lisp files
themselves. In any case, this operation will produce what you need to further build
a static runtime for your system, or a dynamic library to load in an existing runtime."))
(defmethod bundle-type ((o lib-op)) :lib)
(defmethod gather-type ((o lib-op)) :object)
;; What works: on ECL, CLASP(?), MKCL, we link the many .o files from the system into the .so;
;; on other implementations, we combine (usually concatenate) the .fasl files into one.
......@@ -155,11 +154,11 @@ faster and more resource efficient."))
;; we'd have to have the monolithic-op not inherit from the main op,
;; but instead inherit from a basic-FOO-op as with basic-compile-bundle-op above.
(defclass dll-op (link-op gather-operation non-propagating-operation)
((gather-type :initform :object :allocation :class)
(bundle-type :initform :dll :allocation :class))
(defclass dll-op (link-op gather-operation non-propagating-operation) ()
(:documentation "Compile the system and produce a dynamic loadable library (.so/.dll)
for all the linkable object files associated with the system. Compare with LIB-OP."))
(defmethod bundle-type ((o dll-op)) :dll)
(defmethod gather-type ((o dll-op)) :object)
(defclass deliver-asd-op (basic-compile-op selfward-operation)
((selfward-operation
......@@ -188,27 +187,25 @@ for all the linkable object files associated with the system. Compare with LIB-O
((selfward-operation :initform 'monolithic-compile-bundle-op :allocation :class))
(:documentation "Load a single fasl for the system and its dependencies."))
(defclass monolithic-lib-op (lib-op monolithic-bundle-op non-propagating-operation)
((gather-type :initform :object :allocation :class))
(defclass monolithic-lib-op (lib-op monolithic-bundle-op non-propagating-operation) ()
(:documentation "Compile the system and produce a linkable static library (.a/.lib)
for all the linkable object files associated with the system or its dependencies. See LIB-OP."))
(defclass monolithic-dll-op (dll-op monolithic-bundle-op non-propagating-operation)
((gather-type :initform :object :allocation :class))
(defclass monolithic-dll-op (dll-op monolithic-bundle-op non-propagating-operation) ()
(:documentation "Compile the system and produce a dynamic loadable library (.so/.dll)
for all the linkable object files associated with the system or its dependencies. See LIB-OP"))
(defclass image-op (monolithic-bundle-op selfward-operation
#+(or clasp ecl mkcl) link-op #+(or clasp ecl mkcl) gather-operation)
((bundle-type :initform :image :allocation :class)
(gather-operation :initform 'lib-op :allocation :class)
#+(or clasp ecl mkcl) (gather-type :initform :static-library :allocation :class)
(selfward-operation :initform '(#-(or clasp ecl mkcl) load-op) :allocation :class))
((selfward-operation :initform '(#-(or clasp ecl mkcl) load-op) :allocation :class))
(:documentation "create an image file from the system and its dependencies"))
(defmethod bundle-type ((o image-op)) :image)
#+(or clasp ecl mkcl) (defmethod gather-operation ((o image-op)) 'lib-op)
#+(or clasp ecl mkcl) (defmethod gather-type ((o image-op)) :static-library)
(defclass program-op (image-op)
((bundle-type :initform :program :allocation :class))
(defclass program-op (image-op) ()
(:documentation "create an executable file from the system and its dependencies"))
(defmethod bundle-type ((o program-op)) :program)
;; From the ASDF-internal bundle-type identifier, get a filesystem-usable pathname type.
(defun bundle-pathname-type (bundle-type)
......
......@@ -23,8 +23,8 @@
;;;
(with-upgradability ()
;; Base classes for both regular and monolithic concatenate-source operations
(defclass basic-concatenate-source-op (bundle-op)
((bundle-type :initform "lisp" :allocation :class)))
(defclass basic-concatenate-source-op (bundle-op) ())
(defmethod bundle-type ((o basic-concatenate-source-op)) "lisp")
(defclass basic-load-concatenated-source-op (basic-load-op selfward-operation) ())
(defclass basic-compile-concatenated-source-op (basic-compile-op selfward-operation) ())
(defclass basic-load-compiled-concatenated-source-op (basic-load-op selfward-operation) ())
......
......@@ -61,7 +61,7 @@
(in-files (input-files o c))
(in-stamps (mapcar #'get-file-stamp in-files))
(missing-in (loop :for f :in in-files :for s :in in-stamps :unless s :collect f))
(latest-in (stamps-latest (cons dep-stamp in-stamps))))
(latest-in (timestamps-latest (cons dep-stamp in-stamps))))
(when (and missing-in (not just-done))
(DBG "compute-action-stamp: missing inputs" (cons o c) missing-in)
(return (values nil nil))))
......@@ -69,18 +69,18 @@
(out-files (remove-if 'null (output-files o c)))
(out-stamps (mapcar (if just-done 'register-file-stamp 'get-file-stamp) out-files))
(missing-out (loop :for f :in out-files :for s :in out-stamps :unless s :collect f))
(earliest-out (stamps-earliest out-stamps)))
(earliest-out (timestamps-earliest out-stamps)))
(when (and missing-out (not just-done))
(DBG "compute-action-stamp: missing outputs" (cons o c) missing-out)
(return (values nil nil))))
(let (;; Time stamps from the files at hand, and whether any is missing
(all-present (not (or missing-in missing-out)))
;; Has any input changed since we last generated the files?
;; Note that we use stamp<= instead of stamp< to play nice with generated files.
;; Note that we use timestamp<= instead of timestamp< to play nice with generated files.
;; Any race condition is intrinsic to the limited timestamp resolution.
(up-to-date-p (stamp<= latest-in earliest-out))
(up-to-date-p (timestamp<= latest-in earliest-out))
;; If everything is up to date, the latest of inputs and outputs is our stamp
(done-stamp (stamps-latest (cons latest-in out-stamps))))
(done-stamp (timestamps-latest (cons latest-in out-stamps))))
;; Warn if some files are missing:
;; either our model is wrong or some other process is messing with our files.
(when (and just-done (not all-present))
......
(in-package :asdf)
(defclass fiveam-tester-system (system)
((test-names
:initarg :test-names
:reader test-names
:documentation "A list whose elments are either
cons cells of symbol and package designators or
simply a symbol designator.
In the latter case, the symbols will be interned
in the package designated by the TEST-PACKAGE slot,
which must be bound.")
(test-package
:initarg :default-test-package
:initarg :test-package
:documentation "If all the tests are in one package, you can just
have a list of test names in test-names, and get the package name from
here.")
(num-checks
:initarg :num-checks
:reader num-checks
:type (or null (integer 0))
:initform nil
:documentation "How many tests do you expect to run when
you invoke the test-op on this system. For backwards compatibility,
this slot is ignored if it's NIL. If bound, then when running the
test-op, we will fail if the expected number of checks are not run.")
))
(define-condition fiveam-asdf-failure (error)
((failed-asdf-component
:initarg :failed-asdf-component
:reader failed-asdf-component
))
(:documentation "Superclass of error conditions that indicate that
an ASDF test-op has failed."))
(define-condition fiveam-test-fail (fiveam-asdf-failure)
((failed
:initarg :failed
:reader failed
))
(:report (lambda (x s)
(format s "Tests on system ~a failed: ~{~t~a~%~}"
(component-name (failed-asdf-component x))
(failed x)))))
(define-condition fiveam-wrong-number-of-checks (fiveam-asdf-failure)
((expected-num-checks
:initarg :expected
:initarg :expected-num-checks
:reader expected-num-checks
)
(actual-num-checks
:initarg :actual-num-checks
:initarg :actual
:reader actual-num-checks
))
(:report (lambda (x s)
(format s "Unexpected number of tests on system ~a: Expected ~d got ~d."
(component-name (failed-asdf-component x))
(expected-num-checks x)
(actual-num-checks x)))))
(defgeneric test-package (x)
(:method ((x fiveam-tester-system))
(if (slot-boundp x 'test-package)
(slot-value x 'test-package)
(error "If package is not specified with each test-name, system's TEST-PACKAGE slot must be set."))))
(defmethod perform ((op test-op) (sys fiveam-tester-system))
(let* ((test-syms
(loop for x in (test-names sys)
with test-name and package-name and test-sym and package
if (symbolp x)
do (setf test-name x
package-name (test-package sys))
else
do (assert (and (consp x)
(or (symbolp (car x)) (stringp (car x)))
(or (symbolp (cdr x)) (stringp (cdr x)))))
(setf test-name (car x) package-name (cdr x))
do (setf package (or (find-package package-name)
(error "Unable to find package ~a" package-name)))
(setf test-sym
(intern
(etypecase test-name
(string test-name)
(symbol (symbol-name test-name)))
package))
collect test-sym))
(runner (intern (symbol-name '#:run) :fiveam))
(tester (intern (symbol-name '#:results-status) :fiveam))
(explainer (intern (symbol-name '#:explain!) :fiveam))
(results (loop for test in test-syms
appending (funcall runner test))))
(funcall explainer results)
;; if there's an expected number of checks, verify that we have run
;; exactly that number.
(when (num-checks sys)
(let ((actual-num-checks (length results)))
(unless (= actual-num-checks (num-checks sys))
(error 'fiveam-wrong-number-of-checks
:failed-asdf-component sys
:actual actual-num-checks
:expected (num-checks sys)))))
(multiple-value-bind (success failures)
(funcall tester results)
(unless success
(error 'fiveam-test-fail :failed-asdf-component sys :failed failures)))))
(defmethod component-depends-on ((op load-op) (sys fiveam-tester-system))
(cons '(load-op "fiveam") (call-next-method)))
(defmethod component-depends-on ((op compile-op) (sys fiveam-tester-system))
(cons '(load-op "fiveam") (call-next-method)))
(eval-when (:compile-toplevel :load-toplevel :execute)
(export 'fiveam-tester-system :asdf))
;;;---------------------------------------------------------------------------
;;; Copyright (c) 2012-2018 Smart Information Flow Technologies, d/b/a SIFT, LLC and
;;; Robert P. Goldman
;;; All rights reserved.
;;;
;;; The developers make this software available according to the terms
;;; of the Lisp Lesser GNU Public License (LLGPL):
;;; http://opensource.franz.com/preamble.html.
;;; See the accompanying file, license.txt
;;;
;;;---------------------------------------------------------------------------
;;;
;;; This system provides functionality to support integrating FiveAM tests with
;;; an ASDF system definition in such a way that invocation of the
;;; ASDF:TEST-SYSTEM function will cause the FiveAM tests to run. An error will
;;; be raised if there are test failures.
;;;
;;;---------------------------------------------------------------------------
(defpackage :fiveam-asdf-asd
(:use :common-lisp :asdf)
)
(in-package :fiveam-asdf-asd)
(defsystem fiveam-asdf
:long-description
"System that defines a new system class FIVEAM-TESTER
that provides functionality for running tests using
FIVEAM and raising an error if the tests fail
\(useful for incorporation in a Jenkins or Hudson
build\)."
:depends-on (:asdf)
:components ((:file "code"))
:author "Robert P. Goldman"
:version "2.0"
:license "Lisp LGPL"
)
FIVEAM-ASDF is a library enabling suites of FiveAM tests to be
associated with ASDF systems in such a way that ASDF:TEST-SYSTEM will
cause the tests to be run and will raise an exception if something
unexpected happens.
FIVEAM-ASDF is licensed under the terms of the Lisp Lesser GNU
Public License (http://opensource.franz.com/preamble.html), known as
the LLGPL. The LLGPL consists of a preamble (see above URL) and the
LGPL. Where these conflict, the preamble takes precedence.
FIVEAM-ASDF is referenced in the preamble as the "LIBRARY."
cl-asdf (2:3.3.2-1) unstable; urgency=low
Bug fix release:
* Dependency tracking: fix bugs in handling misnamed secondary systems.
We now distinguish between "syntactic" and "semantic" primariness,
properly warn in case a secondary system is misnamed, yet properly
recompile or avoid spurious recompilation depending on whether
there or not is a conflict or redefinition.
Similarly handle package-inferred-system (thanks to Eric Timmons).
Better warning in a fishy case of compute-action-stamp.
* UIOP: Add package location information to define-package on SBCL.
* Upgrade: fix issue with asdf/bundle.
* Portability: add support for Mezzano (thanks to E. Housel, H. Harrington)
* Documentation: add discussion of best practices.
* Package-inferred system: Improve source file handling.
* Test and documentation improvements.
-- Francois-Rene Rideau <fare@tunes.org> Thu, 03 May 2018 22:46:19 -0400
cl-asdf (2:3.3.1-1) unstable; urgency=low
New milestone:
* UIOP compatibility fix: Introduced new replacement "timestamp"
comparison functions, because of inadvertent change in the
API. Functions that are compatible with the old semantics are retained
as "stamp" comparison functions, but will eventually be deprecated.
* Upgrade fix: Upgrade from 3.2.1 needed repair.
* Syntax manipulation: Fix for bugs that could be introduced when the
default readtable was manipulated during the loading of a
defsystem-depends-on system.
* Tests: tests for new capabilities and bugs
* Documentation: a number of improvements and clarifications.
-- Robert P. Goldman <rpgoldman@sift.net> Sun, 12 Nov 2017 17:39:04 -0600
cl-asdf (2:3.3.0-1) unstable; urgency=low
New milestone:
......@@ -9,14 +44,15 @@ cl-asdf (2:3.3.0-1) unstable; urgency=low
* Internals: to support the above, many ASDF internals have changed.
ASDF now has the notion of multiple build phases to a common build session
(which generalizes the previous build cache). ASDF considers loading a .asd
file as an operation DEFINE-OP, and tracks as dependencies files mentionned
during in :load-file-form statements and such. Some code has moved to new
files or among old files, and accordingly as for packages. Actions are now
uniformly represented as a cons of an operation and a component, where
sometimes just the class-of the operation was preserved. Forcing must be
uniform across all phases of a same build session. Fix the protocol for
resetting systems being (re)defined, allowing for subclasses to define
default slot values. Remove *load-system-operation*, as the current
file as an operation DEFINE-OP, and tracks as dependencies files mentioned
during in :LOAD-FILE-FORM statements, etc. Some code has moved to new
files or among old files, and between packages. Actions are now
uniformly represented as a CONS of an OPERATION and a COMPONENT, where
in some cases previously only the class of the operation was
preserved. Forcing is constrained to be uniform across all phases of a
top level ASDF operation invocation. Fixed the protocol for
resetting systems being (re)defined, allowing subclasses to define
default slot values. Remove *LOAD-SYSTEM-OPERATION*, as the current
maintainer of ECL, for which it was originally designed, decided
that it could never be made to work properly, after all.
* ASDF&UIOP: Tweak dependencies between ASDF and UIOP. To avoid DEFINE-OP
......@@ -28,9 +64,11 @@ cl-asdf (2:3.3.0-1) unstable; urgency=low
* Feature: a new feature :asdf3.3
* ECL: restored the deprecated function MAKE-BUILD, removed in 3.2.0,
in a way that works on top of supported APIs (we still recommend you migrate
to said support APIs). Also stop using the deprecated compute-init-name.
to these supported APIs). Also stop using the deprecated COMPUTE-INIT-NAME.
* Deprecation: starting to emit STYLE-WARNINGs for deprecated
functions. Will gradually escalate to true WARNINGs and then ERRORs.
-- Robert P. Goldman <rpgoldman@sift.net> Sun, 20 August 2017 21:00:00 -0500
-- Robert P. Goldman <rpgoldman@sift.net> Sun, 20 August 2017 21:00:00 -0500
cl-asdf (2:3.2.1-1) unstable; urgency=low
......
......@@ -65,7 +65,7 @@ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@titlepage
@title ASDF: Another System Definition Facility
@subtitle Manual for Version 3.2.1
@subtitle Manual for Version 3.3.2.2
@c The following two commands start the copyright page.
@page
@vskip 0pt plus 1filll
......@@ -82,7 +82,7 @@ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@node Top, Introduction, (dir), (dir)
@top ASDF: Another System Definition Facility
@ifnottex
Manual for Version 3.2.1
Manual for Version 3.3.2.2
@end ifnottex
......@@ -5818,7 +5818,7 @@ then ASDF will reconfigure itself.
Otherwise, you will have to configure ASDF 2 (or older) to find ASDF 3,
then configure ASDF 3.
Notably, @var{*central-registry*} is not maintained across upgrades from ASDF 2.
@xref{reinitializeASDFAfterUpgrade,note about ASDF reconfiguration after upgrade}
@xref{reinitializeASDFAfterUpgrade,note about ASDF reconfiguration after upgrade}.
Problems like this may be experienced if one loads Quicklisp
(which as of this writing bundles an obsolete ASDF version 2.26),
......@@ -6347,9 +6347,10 @@ Also, bugs are currently tracked on launchpad:
Available in updated-for-CL form on the web at
@url{http://nhplace.com/kent/Papers/Large-Systems.html}
@item Dan Weinreb and David Moon:
``Lisp Machine Manual'', MIT, 1981.
``Lisp Machine Manual'', 3rd Edition MIT, March 1981.
The famous CHINE NUAL describes one of the earliest variants of DEFSYSTEM.
@url{https://bitsavers.trailing-edge.com/pdf/mit/cadr/chinual_4thEd_Jul81.pdf}
(NB: Not present in the second preliminary version of January 1979)
@url{http://bitsavers.org/pdf/mit/cadr/chinual_3rdEd_Mar81.pdf}
@end itemize
......
# ASDF Best Practices
ASDF Best Practices
===================
This document presents the current best practices and conventions
for using ASDF 3, as of 2017.
......@@ -9,16 +10,65 @@ that he already knows what seems to be common knowledge among such users,
and tries to complete this knowledge with less obvious points
that are often wrong in systems seen in the wild.
[TOC]
## Trivial Examples
<a name="toc"></a>Table of Contents
-----------------------------------
- [Trivial Examples](#trivial_examples)
+ [Trivial Uses of ASDF](#trivial_asdf)
* [Loading a System](#loading_system)
* [Testing a System](#testing_system)
* [Designating a System](#designating_system)
+ [Trivial System Definition](#trivial_system)
* [Using the system you defined](#using_system)
+ [Trivial Testing Definition](#trivial_testing)
* [Notes on ASDF 2 compatibility](#notes_asdf2)
+ [Trivial Packaging](#trivial_packaging)
* [Digression about symbols and packages](#digression)
- [Simple Examples](#simple_examples)
+ [Simple Uses of a System](#simple_uses)
* [Building a System](#building_system)
* [Inspecting a System](#inspecting_system)
* [Other Operations](#other_operations)
+ [System Naming](#system_naming)
* [Primary Systems](#primary_systems)
* [Secondary Systems](#secondary_systems)
+ [Simple System Definition](#simple_system)
+ [Simple Packaging](#simple_packaging)
* [Initial Package for a Lisp File](#initial_package)
* [Using Symbols from ASDF and UIOP](#using_symbols)
+ [Simple Testing](#simple_testing)
+ [Other Secondary Systems](#other_secondary)
* [Delivering an Executable](#delivering_executable)
* [System Connections](#system_connections)
- [More Elaborate Examples](#moreelaborate_examples)
+ [More Elaborate Uses of ASDF](#moreelaborate_asdf)
* [force](#force)
* [force-not](#force_not)
* [Require](#require)
+ [More Elaborate System Definitions](#elaborate_definitions)
* [package-inferred-system](#package_inferred)
* [Using ASDF Extensions](#using_extensions)
* [Code in .asd files](#code_asd)
* [Conditional Code](#conditional_code)
+ [More Elaborate Testing](#moreelaborate_testing)
+ [Defining ASDF Extensions](#defining_extensions)
* [Source File Types](#source_types)
* [Conditional Outputs and Conditional Perform](#conditional_outputs)
- [Other](#other)
+ [Syntax Control](#syntax_control)
+ [Using ASDF Internals](#using_asdf_internals)
<a name="trivial_examples"></a>Trivial Examples
------------------------------------------------
Let's start with some trivial examples.
We'll see below how these examples evolve as systems grow more complex.
### Trivial Uses of ASDF
### <a name="trivial_asdf"></a>Trivial Uses of ASDF
#### Loading a System
#### <a name="loading_system"></a>Loading a System
The most trivial use of ASDF is by calling `(asdf:load-system :foobar)`
to load your library.
......@@ -29,7 +79,7 @@ then you will be able to call it with `(foobar:some-fun ...)` or with:
(in-package :foobar)
(some-fun ...)
#### Testing a System
#### <a name="testing_system"></a>Testing a System
To run the tests for a system, you may use:
......@@ -37,7 +87,7 @@ To run the tests for a system, you may use:
The convention is that an error SHOULD be signalled if tests are unsuccessful.
#### Designating a System
#### <a name="designating_system"></a>Designating a System
Using keywords to name systems is good and well at the REPL.
However, when writing a program, a bootstrap script, or a system definition,
......@@ -64,7 +114,7 @@ But that is not where we are.
In any case, system names designate `.asd` files, not Lisp bindings,
and this determines their syntax.)
### Trivial System Definition
### <a name="trivial_system"></a>Trivial System Definition
A trivial system would have a single Lisp file called `foobar.lisp`.
That file would depend on some existing libraries,
......@@ -96,7 +146,7 @@ As for contents of that file, they would look like this:
...)
...
#### Using the system you defined
#### <a name="using_system"></a>Using the system you defined
Assuming your system is installed under the `~/common-lisp/` hierarchy
or some other filesystem hierarchy already configured for ASDF,
......@@ -107,7 +157,7 @@ If your Lisp was already started when you created that file,
you may have to `(asdf:clear-configuration)` to re-process the configuration.
### Trivial Testing Definition
### <a name="trivial_testing"></a>Trivial Testing Definition
Even the most trivial of systems needs some tests,
if only because it will have to be modified eventually,
......@@ -137,7 +187,7 @@ and the content of the `perform` method is how to invoke this library
to run the test suite `:foobar`.
Obvious YMMV if you use a different library.
#### Note on ASDF 2 compatibility
#### <a name="notes_asdf2"></a>Note on ASDF 2 compatibility
The `:in-order-to ((test-op (test-op ...)))` idiom will not work with ASDF 2:
attempts to `test-system` will result in failure due to circular dependencies;
......@@ -177,14 +227,14 @@ Two years seems to be the time it takes for a release of ASDF
to become ubiquitously available by default on all implementations;
and, remember, if a user must use an older implementation,
he can always trivially install a newer ASDF on top of it.
Thus, by 2019, people should not be shy about
dropping support for ASDF versions older than 3.2 or 3.3.
Thus, by October 2019, people should not be shy about
dropping support for ASDF versions older than 3.3.
And even before then, if you need a recent ASDF, just document it,
and tell your users to upgrade their implementation and/or
install a recent ASDF on top of their implementation's.
### Trivial Packaging
### <a name="trivial_packaging"></a>Trivial Packaging
In the previous testing code, `symbol-call` is a function defined in package `uiop`.
It helps deal with the fact that the package `:fiveam` isn't defined yet
......@@ -226,7 +276,7 @@ You SHOULD be using the `slime-asdf` extension to SLIME
if you are going to edit `.asd` file and then load them from SLIME,
as it will automatically use `load-asd` to load the file contents.
#### Digression about symbols and packages
#### <a name="digression"></a>Digression about symbols and packages
A crucial notion in Common Lisp is that of *symbols*, to which are associated
functions, variables, macros, properties, and other meanings.
......@@ -247,14 +297,16 @@ where each interned symbol is also a constant bound to its own name as a value.
Using a keyword as designator rather than a string ensures that we can maintain
the same convention of using lower-case in source code while the runtime will use uppercase.
## Simple Examples
<a name="simple_examples"></a>Simple Examples
----------------------------------------------
As systems grow, the above pattern quickly becomes insufficient,
but systems can still remain simple.
### Simple Uses of a System
### <a name="simple_uses"></a>Simple Uses of a System
#### Building a System
#### <a name="building_system"></a>Building a System
Some systems offer operations
that are neither loading in the current image, nor testing.
......@@ -269,7 +321,7 @@ the above will be equivalent to `(asdf:load-system :foobar)`,
but for other Lisp systems, e.g. one that creates a shell command-line executable,
`(asdf:make ...)` will do the Right Thing™, whatever that Right Thing™ is.
#### Inspecting a System
#### <a name="inspecting_system"></a>Inspecting a System
To look at what ASDF knows of your system, try:
......@@ -280,7 +332,7 @@ For the directory in which the `.asd` file resides, try `(asdf:system-source-dir
For a specific file under that directory try
`(asdf:system-relative-pathname "foobar" "path/to/the/file.extension")`.
#### Other Operations
#### <a name="other_operations"></a>Other Operations
ASDF has the concept of an *operation* that can act upon a system (or a smaller component thereof).
Typical operations that matter to end-users include:
......@@ -318,9 +370,9 @@ The operation is typically specified as a symbol that names the operation class.
Since ASDF 3, you can also use a keyword to specify an action in the ASDF package.
Thus, `(asdf:oos :load-op :foobar)` is equivalent to `(asdf:load-system :foobar)`.
### System Naming
### <a name="system_naming"></a>System Naming
#### Primary Systems
#### <a name="primary_systems"></a>Primary Systems
ASDF has a notion of *primary system*,
that it can find in configured parts of the filesystem (the *source-registry*),
......@@ -329,7 +381,7 @@ Thus, primary system `foobar` is defined in a file `foobar.asd`.
While arbitrary strings are accepted in system names, it is strongly discouraged
to use anything but lower-case ASCII letters and digits for primary system names,
plus the separators `-` (dash) and `.` (dot).
plus the separators `-` (hyphen) and `.` (dot).
The `.` itself is only recommended in primary system names that are part of an informal hierarchy;
for instance the popular library `iolib` contains many related systems:
`iolib.asdf`, `iolib.base`, `iolib.common-lisp`, `iolib.conf`, `iolib.examples`,
......@@ -338,13 +390,13 @@ The main one is ostensibly `iolib`, but it contains many systems,
and for some reasons (notably proper phase separation)
they cannot all be secondary systems in the same file.
#### Secondary Systems
#### <a name="secondary_systems"></a>Secondary Systems
A *secondary system* is a system defined in the same file as a primary system.
By convention, its name starts the same as the file's primary system,
followed by a slash `/` and by a *suffix* made of some arbitrary characters,
preferrably ASCII letters and digits
plus the separators `-` (dash), `.` (dot) and `/` (slash).
preferably ASCII letters and digits
plus the separators `-` (hyphen), `.` (dot) and `/` (slash).
We already saw an example of it with system `foobar/tests`
in the trivial testing definition above.
......@@ -357,7 +409,7 @@ and in practice many people used `foo-test` or such for secondary system names i
however, ASDF 1 (and 2) couldn't find those systems by name, and horrific bugs could happen
if a system was simultaneously defined in multiple files.)
### Simple System Definition
### <a name="simple_system"></a>Simple System Definition
A simple system may be made of many files.
......@@ -413,9 +465,9 @@ only in ASDF 3.3 or later, due to a bug in earlier versions of ASDF.
But that precise order shouldn't matter, or it should be reflected
in the `:depends-on` declarations (or in a `:serial t` declaration).
### Simple Packaging
### <a name="simple_packaging"></a>Simple Packaging
#### Initial Package for a Lisp File
#### <a name="initial_package"></a>Initial Package for a Lisp File
You MAY assume that the current package uses `CL` at the beginning of a file,
but you MUST NOT assume that it is any particular package at this point:
......@@ -432,7 +484,7 @@ at which point you may as well `(cl:defpackage ...)` and `(cl:in-package ...)`).
If it's a regular `cl-source-file`, it can assume the language is CL indeed,
and that the readtable something reasonable, etc.
#### Using symbols from ASDF and UIOP
#### <a name="using_symbols"></a>Using Symbols from ASDF and UIOP
You MAY use any of the symbols documented and exported by ASDF or UIOP.
Actually, it is warmly recommended to use them everywhere that it matters,
......@@ -441,11 +493,12 @@ instead of less documented or less portable alternatives.
You MUST NOT use `asdf:run-shell-command`, `asdf:system-definition-pathname`,
or other deprecated functions that were once recommended in the time of ASDF 1.
They will be removed in the near future (one to two year horizon).
ASDF 3.2 will issue a `style-warning` when you do, and ASDF 3.3 will issue a full `warning`,
which will break the SBCL build.
ASDF 3.2 or 3.3 will issue a `style-warning` when you do, and
some future version of ASDF will issue a full `warning`,
which will then break the SBCL build.
See `backward-interface.lisp` for a list of deprecated function — or just heed the damn warnings.
### Simple Testing
### <a name="simple_testing"></a>Simple Testing
Test systems can also be divided in multiple files.
If possible (which is not always the case), the file names for test files
......@@ -472,14 +525,14 @@ As the system and its test system both grow, the test system may be moved to its
(defsystem "foobar-tests" ...)
### Other secondary systems
### <a name="other_secondary"></a>Other secondary systems
Other secondary systems may be created beyond test systems:
for instance systems that provide independent aspects of the system,
or optional add-ons to it.
One case is a command that makes the Lisp functionality accessible from a Unix shell.
#### Delivering an Executable
#### <a name="delivering_executable"></a>Delivering an Executable
To build an executable, define a system as follows
(in this case, it's a secondary system, but it could also be a primary system).
......@@ -537,7 +590,7 @@ Both `cl-launch` and `buildapp` have similar functionality
to handle multicall binaries à la [Busybox](http://busybox.net/),
with the same incompatibility as above.
#### System Connections
#### <a name="system_connections"></a>System Connections
Sometimes, a system can provide an extension to another system.
For instance, if you use both `metacopy` and `contextl`,
......@@ -552,11 +605,13 @@ I recommend against using it, because it introduces side-effects within the buil
Instead I recommend explicitly loading the system connections
as part of the larger system that will use them.
## More Elaborate Examples
### More Elaborate Uses of ASDF
<a name="moreelaborate_examples"></a>More Elaborate Examples
--------------------------------------------------------
### <a name="moreelaborate_asdf"></a>More Elaborate Uses of ASDF
#### force
#### <a name="force"></a>force
Sometimes, you want to force ASDF to re-build some system.
At those times, you can pass the `:force` argument to `asdf:operate`
......@@ -564,7 +619,7 @@ At those times, you can pass the `:force` argument to `asdf:operate`
Passing an argument `t` will force rebuild of just the system, and none of its dependencies
(and that also means none of the dependencies that happen to be secondary systems
with the same primary system name).
Passing a list of system designators (preferrably lower-case strings)
Passing a list of system designators (preferably lower-case strings)
will force the specific systems to be rebuilt (if they appear in the build plan at all, that is).
Finally, passing `:all` as argument will force a rebuild of everything, including all dependencies.
......@@ -581,7 +636,7 @@ In the case of `prove`, we will have to work with its author so that the correct
to use it doesn't violate ASDF invariants, but instead properly declare that
ASDF should not consider tests already run.
#### force-not
#### <a name="force_not"></a>force-not
The converse of `:force` is `:force-not`, and you can specify a list of systems to not rebuild.
In this context `t` means "everything but this system" rather than " this system".
......@@ -594,7 +649,7 @@ You MUST NOT call `asdf:operate` with `:force` or `:force-not` from within a bui
Actually, you should probably not explicitly use `asdf:operate` at all,
except maybe inside a `.asd` file in cases where `defsystem-depends-on` isn't sufficient.
#### Require
#### <a name="require"></a>Require
You SHOULD NOT use `cl:require` as a substitute for `asdf:load-system`.
You SHOULD NOT use `asdf:require-system` except at the toplevel.
......@@ -616,7 +671,7 @@ As ASDF maintainer, I now consider this in bad taste:
* Third, it interferes with ASDF's capacity to detect legitimate vs illegitimate
recursive uses of `operate` at places that defeat tracking of dependencies.
* Fourth, it adds a lot of complexity for dubious gain: at a time you had to type
`(asdf:operate 'load-op :foo)`, `(require :foo)` may have been a nice short-hand,
`(asdf:operate 'asdf:load-op :foo)`, `(require :foo)` may have been a nice short-hand,
but it isn't such a great gain over `(asdf:make :foo)`.
* Fifth, SBCL now uses ASDF 3's `compile-bundle-op` to create a fasl
during the build of SBCL itself, and that fasl can latter be loaded at runtime without ASDF.
......@@ -625,7 +680,7 @@ As ASDF maintainer, I now consider this in bad taste:
Similarly, in ASDF 2.21 I added a function `require-system`
that used to called `load-system` with `:force-not (already-loaded-systems)`,
which was a nice hack at the time, that I latter used as part of the `cl:require` hook.
That was all a big mistake, as `:force-not` interfere with the ability to keep a coherent plan
That was all a big mistake, as `:force-not` interferes with the ability to keep a coherent plan
across recursive uses of `asdf:operate` as required by builds that involve `:defsystem-depends-on`
and other ASDF extensions.
These days, this function only checks whether the requested component is already loaded,
......@@ -633,9 +688,9 @@ and if not calls `asdf:load-system` on it.
This function MUST only be used at the toplevel, never in a script or build.
It may be deprecated in a future version of ASDF.
### More Elaborate System Definitions
### <a name="elaborate_definitions"></a>More Elaborate System Definitions
#### package-inferred-system
#### <a name="package_inferred"></a>package-inferred-system
When you start writing large enough systems,
putting everything in one big package leads to a big mess:
......@@ -681,7 +736,7 @@ rather than the entire humongous library.
This also helps you enforce a discipline wherein it is always clear in which file
each symbol is defined, which files have symbols used by any other file, etc.
#### Using ASDF Extensions
#### <a name="using_extensions"></a>Using ASDF Extensions
If you need an ASDF extension, the recommended way is to use `:defsystem-depends-on`.
The extension will define new classes of operations or components, new functions, etc.
......@@ -705,7 +760,7 @@ in your own extension.
(In a prototype OO system, you could just mix and match extensions without defining a new class,
but CLOS is not a prototype OO system.)
#### Code in .asd files
#### <a name="code_asd"></a>Code in .asd files
ASDF currently allows arbitrary Lisp code in a `.asd` file.
I would like to deprecate that in the future to make ASDF more declarative.
......@@ -726,7 +781,7 @@ It is poor taste to define functions, variables or macros in package `asdf-user`
unless strictly necessary or widely useful, and even then with a long name
that distinctively includes the name of your system.
However it is perfectly acceptable to define methods on existing functions,
preferrably using the `:perform` syntax used above, as part of a `defsystem` form.
preferably using the `:perform` syntax used above, as part of a `defsystem` form.
If you must define new functions, variables or macros,
you MUST define a new package in which to define them.
......@@ -749,7 +804,7 @@ For versions, consider having ASDF extract the version from a file, as in
`:version (:read-file-form "variables.lisp" :at (2 2 2)`.
See the ASDF manual for details, and `asdf.asd` itself for an example.
#### Conditional Code
#### <a name="conditional_code"></a>Conditional Code
The recommended way to conditionally use code is to rely on the CL features;
yet, you SHOULD NOT use read-time conditionals if you can avoid it,
......@@ -779,7 +834,7 @@ in particular, the mere fact of have loaded your system does not warrant a new f
A feature is only warranted if your system is some deep infrastructure
with mostly compatible rivals that it needs to be distinguished from.
### More Elaborate Testing
### <a name="moreelaborate_testing"></a>More Elaborate Testing
You MUST NOT call `asdf:operate` or any of its derivatives,
such as `asdf:load-system` or `asdf:test-system` from within a `perform` method.
......@@ -790,14 +845,15 @@ You MUST NOT call `asdf:clear-system` or in any way interfere with the build
while in a `perform` method.
You SHOULD NOT define methods on `asdf:operate` --- most of the time it's totally the wrong thing
because users would be "operating" on your system, but on their system that depends on it.
because users would *not* be "operating" on *your* system, but on *their* system that depends on it.
Instead you SHOULD define methods on `asdf:perform`, `asdf:component-depends-on`, etc.
### Defining ASDF Extensions
### <a name="defining_extensions"></a>Defining ASDF Extensions
#### Source File Types
#### <a name="source_types"></a>Source File Types
You MUST NOT define methods on `source-file-type`. This bad ASDF 1 interface must die.
You MUST NOT define methods on `source-file-type`.
This bad ASDF 1 interface must die and will be removed in a future version of ASDF.
Instead, override the slot `type` for your file class and provide a proper `:initform`, as in:
(defclass cl-source-file.l (cl-source-file) ((type :initform "l")))
......@@ -805,7 +861,7 @@ Instead, override the slot `type` for your file class and provide a proper `:ini
You can then provide the name of that class as class name for individual components,
or as `:default-component-class` to relevant systems or modules.
#### Conditional Outputs and Conditional Perform
#### <a name="conditional_outputs"></a>Conditional Outputs and Conditional Perform
An operation MUST actually create all the outputs declared by the `output-files` method.
If some of these outputs are conditional, the `output-files` method MUST check the condition
......@@ -823,7 +879,11 @@ operations on a given component, you MAY want to override or wrap around
the `component-if-feature` method for that component instead of defining
all those `perform`, `output-files` and `input-files` methods.
## Other
<a name="other"></a>Other
--------------------------
### <a name="syntax_control"></a>Syntax Control
Do not side-effect the current `*readtable*`,
which is never guaranteed to be writable, and
......@@ -834,3 +894,28 @@ To bind the readtable, use `named-readtables:in-readtable` or `cl-syntax:in-synt
in each file that needs a non-standard readtable, and/or use an `:around-compile` hook
to automatically bind it around every file in a module or system.
See the syntax-control document (in the syntax-control branch if it wasn't merged yet).
### <a name="using_asdf_internals"></a>Using ASDF Internals
At times, you will have to use some ASDF internals to get your software to work.
When this happens, follow these rules to minimize the breakage associated to bitrot:
* You MUST use the `asdf::` package prefix rather than `asdf/foo:`
to name the symbols internal to ASDF.
Indeed, from one version of ASDF to the next,
which internal package a symbol will reside in may change;
for instance `asdf/find-system:primary-system-name` recently became
`asdf/system:primary-system-name`
(it is still present, not exported, in the former system);
but it is always present (though not currently exported) in package `asdf`.
* When you use internals, you MUST notify the ASDF maintainers, and request
that they either export the symbol and support the API, or
offer an alternative API more to their liking for the same functionality.
Otherwise, they will be justified in modifying internals
that no one is supposed to use, and they will blame you when
they later change or remove these internals and your software breaks.
Note that the latter point is true of all software, not just ASDF.
......@@ -43,8 +43,9 @@
and the ability to deliver standalone executables.
Its notable versions include pre-release 2.27 on February 1st 2013,
first stable release 3.0.1 on May 16th 2013,
major releases 3.1.2 on May 6th 2014 and 3.2.0 on January 10th 2017,
and latest release 3.2.1 on April 3rd 2017.
major releases 3.1.2 on May 6th 2014,
3.2.0 on January 10th 2017 and 3.3.0 on October 6th 2017.
The latest release is 3.3.1, published on November 4th 2017.
</p>
<a id="what_it_is"></a>
<h3>What it is</h3>
......@@ -321,6 +322,7 @@
<a href="http://clisp.org/">CLISP</a>,
<a href="http://cmucl.org/">CMUCL</a>,
<a href="https://common-lisp.net/project/ecl/" title="Embeddable Common Lisp">ECL</a>,
<a href="https://github.com/froggey/Mezzano/" title="Mezzano">Mezzano</a>,
<a href="https://common-lisp.net/project/mkcl/" title="ManKai Common Lisp">MKCL</a>,
<a href="http://www.sbcl.org/" title="Steel Bank Common Lisp">SBCL</a>
</td>
......@@ -329,14 +331,17 @@
<a title="GNU Common Lisp" href="https://www.gnu.org/software/gcl/">GCL</a>,
<a title="Macintosh Common Lisp" href="https://en.wikipedia.org/wiki/Macintosh_Common_Lisp">MCL</a>
</td>
<td align="left">XCL</td></tr>
<td align="left"><a href="https://github.com/gnooth/xcl">XCL</a></td></tr>
<tr><th align="left">Proprietary</th>
<td align="left">
<a href="http://franz.com/products/allegro-common-lisp/">Allegro</a>,
<a href="http://www.lispworks.com/">LispWorks</a>
</td>
<td align="left"></td>
<td align="left">Genera, mocl, SCL</td></tr>
<td align="left">
<a href="https://en.wikipedia.org/wiki/Genera_(operating_system)">Genera</a>,
<a href="https://wukix.com/mocl">mocl</a>,
<a title="Scieneer Common Lisp" href="http://www.scieneer.com/scl/">SCL</a></td></tr>
</table>
<p>
To deal with an implementation that does not yet provide ASDF 3.1 or later,
......@@ -412,7 +417,7 @@ Corman Common Lisp <admin@cormanlisp.com>
About the latest developments in ASDF 3.2 and ASDF 3.3, see our demo
<cite><a href="http://fare.tunes.org/files/asdf2017/asdf2017.html"
>Delivering Common Lisp Applications with ASDF 3.3</a></cite>
(<a href="http://fare.tunes.org/files/asdf2017/asdf2017.pdf">PDF</a>,
(2-page <a href="http://fare.tunes.org/files/asdf2017/asdf2017.pdf">PDF</a>,
<a href="http://fare.tunes.org/files/asdf2017/els2017-slides.pdf">slides</a>,
<a href="https://github.com/fare/asdf2017">git</a>).
Regarding the internal design of ASDF in general,
......@@ -638,6 +643,14 @@ Corman Common Lisp <admin@cormanlisp.com>
<p>For a detailed description of changes see our <a href="Changelog">Changelog</a>
</p>
<dl>
<dt>November 2017</dt>
<dd>Release of 3.3.1: bugfix release for 3.3.0 addressing
a backwards-incompatibility issue with timestamp computations,
and other minor breakage.</dd>
<dt>October 2017</dt>
<dd>Release of 3.3.0, with substantial revision to the build plan
to correct handle multiple phases of loading, due to e.g.
<code>DEFSYSTEM-DEPENDS-ON</code> dependencies.</dd>
<dt>April 2017</dt>
<dd>Release of 3.2.1, with many small bugfixes and cleanups.
Presentation at ELS 2017 about ASDF 3.3.
......@@ -751,9 +764,9 @@ Corman Common Lisp <admin@cormanlisp.com>
<a class="nav" href="http://www.catb.org/hacker-emblem/" title="hacker"> <img src="http://common-lisp.net/project/cl-containers/shared/buttons/hacker.png" width="80" height="15" title="hacker emblem" alt="hacker button" /></a>
<a class="nav" href="http://www.lisp.org/" title="Association of Lisp Users"> <img src="http://common-lisp.net/project/cl-containers/shared/buttons/lambda-lisp.png" width="80" height="15" title="ALU emblem" alt="ALU button" /></a>
<a class="nav" href="http://common-lisp.net/" title="Common-Lisp.net"> <img src="http://common-lisp.net/project/cl-containers/shared/buttons/lisp-lizard.png" width="80" height="15" title="Common-Lisp.net" alt="Common-Lisp.net button" /></a>
<p><span class="copyright"Copyright &copy; 2001-2016 Daniel Barlow and contributors</span></p>
<p><span class="copyright"Copyright &copy; 2001-2017 Daniel Barlow and contributors</span></p>
<p>ASDF has an <a href="http://www.opensource.org/licenses/mit-license.php">MIT style</a> license</p>
<div id="timestamp">Last updated 2017-04-30</div>
<div id="timestamp">Last updated 2017-12-01</div>
</div>
</body>
</html>
Today we release ASDF 3.3. The most substantial change is to the build planning process, which has been reformed by François-René Rideau so that DEFSYSTEM-DEPENDS-ON now behaves correctly. There are a number of other improvements, and we are indebted to Daniel Kochmanski, Rommel Martinez, and Chun Tian for contributions. Thanks again to Anton Vodonosov and Dave Cooper for support in testing, and new thanks to Martti Halminen, Jens K. Loewe, and Jason Miller.
I would like to dedicate this ASDF release to the late Elias Pipping. On a technical level, Elias made great improvements in some of the most challenging ASDF internals, notably the RUN-PROGRAM component of UIOP, a place where implementations are not constrained by the language specification, diverge radically from each other, and are poorly documented. On a personal level, Elias was very supportive to me as ASDF maintainer, and while I was going through an illness. He did all these things despite being gravely ill with the cancer that eventually killed him. He will be greatly missed, Z'chrono L'vracha.
We urge implementations that are currently bundling previous versions of ASDF to adopt 3.3.0 at their earliest convenience. The improvements offered in this release will substantially improved the reliability and predictability of ASDF use. Details on bugfixes can be found at https://launchpad.net/asdf and in the discussions in merged branches of https://gitlab.common-lisp.net/asdf/asdf
Here is the changelog entry for 3.3:
cl-asdf (2:3.3.0-1) unstable; urgency=low
New milestone:
* Build-plan: Extensively revised the build plan process so that
:DEFSYSTEM-DEPENDS-ON would work correctly, even when depended on systems
change (which didn't work before). See our ELS demonstration about it:
"Delivering Common Lisp Applications with ASDF 3.3"
< https://github.com/fare/asdf2017 >
* Internals: to support the above, many ASDF internals have changed.
ASDF now has the notion of multiple build phases to a common build session
(which generalizes the previous build cache). ASDF considers loading a .asd
file as an operation DEFINE-OP, and tracks as dependencies files mentioned
during in :LOAD-FILE-FORM statements, etc. Some code has moved to new
files or among old files, and between packages. Actions are now
uniformly represented as a CONS of an OPERATION and a COMPONENT, where
in some cases previously only the class of the operation was
preserved. Forcing is constrained to be uniform across all phases of a
top level ASDF operation invocation. Fixed the protocol for
resetting systems being (re)defined, allowing subclasses to define
default slot values. Remove *LOAD-SYSTEM-OPERATION*, as the current
maintainer of ECL, for which it was originally designed, decided
that it could never be made to work properly, after all.
* ASDF&UIOP: Tweak dependencies between ASDF and UIOP. To avoid DEFINE-OP
circularity, asdf.asd with no longer causes uiop.asd to be loaded.
A standalone UIOP won't be loaded at all unless it's strictly more recent
than ASDF.
* Tests: tests for new capabilities and bugs. Test backtraces can be disabled.
* Documentation: a number of improvements and clarifications.
* Feature: a new feature :asdf3.3
* ECL: restored the deprecated function MAKE-BUILD, removed in 3.2.0,
in a way that works on top of supported APIs (we still recommend you migrate
to these supported APIs). Also stop using the deprecated COMPUTE-INIT-NAME.
* Deprecation: starting to emit STYLE-WARNINGs for deprecated
functions. Will gradually escalate to true WARNINGs and then ERRORs.
-- Robert P. Goldman <rpgoldman@sift.net> Sun, 20 August 2017 21:00:00 -0500
Today we release ASDF 3.3.1, the first bugfix release for the 3.3.0 release series. Of primary interest in this release are
1. a fix for an incompatibility in the UIOP stamp-comparison functions. The handling of infinity as a bound in these functions changed between 3.2 and 3.3, potentially causing problems in backward compatibility. In order to maintain compatibility, the old behavior was restored to the stamp-comparison functions, and new functions -- TIMEstamp comparison functions -- were added to carry the new behaviors.
2. a fix for upgrade from version 3.2.1
3. fixes for bugs that can occur when the default readtable is manipulated inside a dependency loaded by DEFSYSTEM-DEPENDS-ON.
We urge implementations that are currently bundling previous versions of ASDF -- and especially those bundling 3.3.0 -- to upgrade to 3.3.1 at their earliest convenience.
Here is the changelog entry for 3.3.1:
cl-asdf (2:3.3.167-1) unstable; urgency=low
New milestone:
* UIOP compatibility fix: Introduced new replacement "timestamp"
comparison functions, because of inadvertent change in the
API. Functions that are compatible with the old semantics are retained
as "stamp" comparison functions, but will eventually be deprecated.
* Upgrade fix: Upgrade from 3.2.1 needed repair.
* Syntax manipulation: Fix for bugs that could be introduced when the
default readtable was manipulated during the loading of a
defsystem-depends-on system.
* Tests: tests for new capabilities and bugs
* Documentation: a number of improvements and clarifications.
Today we release ASDF 3.3.2, the second bugfix release for the 3.3 release series. Of primary interest in this release are
1. Multiple fixes in handling misnamed (or "legacy-named," if you prefer) secondary systems ("foo-test" instead of "foo/test"). Thanks to Eric Timmons, there is also a similar fix for package-inferred-systems.
2. Fix an annoying warning message for compute-action-stamp.
3. Add support for the Mezzano implementation (thanks to E. Housel, H. Harrington).
4. Miscellaneous minor fixes and documentation tweaks.
We urge implementations that are currently bundling previous versions of ASDF -- and especially those bundling 3.3.0 or 3.3.1 -- to upgrade to 3.3.2 at their earliest convenience.
Immense thanks, as always, to Fare Rideau. Also thanks to the many bug reporters and patch submitters (see the Changelog), and to Anton Vodonosov and Dave Cooper for help with testing.
Here is the changelog entry for 3.3.2:
cl-asdf (2:3.3.2-1) unstable; urgency=low
Bug fix release:
* Dependency tracking: fix bugs in handling misnamed secondary systems.
We now distinguish between "syntactic" and "semantic" primariness,
properly warn in case a secondary system is misnamed, yet properly
recompile or avoid spurious recompilation depending on whether
there or not is a conflict or redefinition.
Similarly handle package-inferred-system (thanks to Eric Timmons).
Better warning in a fishy case of compute-action-stamp.
* UIOP: Add package location information to define-package on SBCL.
* Upgrade: fix issue with asdf/bundle.
* Portability: add support for Mezzano (thanks to E. Housel, H. Harrington)
* Documentation: add discussion of best practices.
* Package-inferred system: Improve source file handling.
* Test and documentation improvements.
-- Francois-Rene Rideau <fare@tunes.org> Thu, 03 May 2018 22:46:19 -0400
......@@ -82,21 +82,21 @@
;; TODO: could this file be refactored so that locate-system is merely
;; the cache-priming call to input-files here?
(defmethod input-files ((o define-op) (s system))
(assert (equal (coerce-name s) (primary-system-name s)))
(if-let ((asd (system-source-file s))) (list asd)))
(defmethod perform ((o define-op) (s system))
(assert (equal (coerce-name s) (primary-system-name s)))
(nest
(if-let ((pathname (first (input-files o s)))))
(let ((readtable *readtable*) ;; save outer syntax tables. TODO: proper syntax-control
(print-pprint-dispatch *print-pprint-dispatch*)))
(with-standard-io-syntax)
(let ((*print-readably* nil)
;; Note that our backward-compatible *readtable* is
;; a global readtable that gets globally side-effected. Ouch.
;; Same for the *print-pprint-dispatch* table.
;; We should do something about that for ASDF3 if possible, or else ASDF4.
(*readtable* *readtable*)
(*print-pprint-dispatch* *print-pprint-dispatch*)
(*readtable* readtable) ;; restore inside syntax table
(*print-pprint-dispatch* print-pprint-dispatch)
(*package* (find-package :asdf-user))
(*default-pathname-defaults*
;; resolve logical-pathnames so they won't wreak havoc in parsing namestrings.
......@@ -193,21 +193,25 @@ Do NOT try to load a .asd file directly with CL:LOAD. Always use ASDF:LOAD-ASD."
(defun locate-system (name)
"Given a system NAME designator, try to locate where to load the system from.
Returns five values: FOUNDP FOUND-SYSTEM PATHNAME PREVIOUS PREVIOUS-TIME
Returns six values: FOUNDP FOUND-SYSTEM PATHNAME PREVIOUS PREVIOUS-TIME PREVIOUS-PRIMARY
FOUNDP is true when a system was found,
either a new unregistered one or a previously registered one.
FOUND-SYSTEM when not null is a SYSTEM object that may be REGISTER-SYSTEM'ed.
PATHNAME when not null is a path from which to load the system,
either associated with FOUND-SYSTEM, or with the PREVIOUS system.
PREVIOUS when not null is a previously loaded SYSTEM object of same name.
PREVIOUS-TIME when not null is the time at which the PREVIOUS system was loaded."
PREVIOUS-TIME when not null is the time at which the PREVIOUS system was loaded.
PREVIOUS-PRIMARY when not null is the primary system for the PREVIOUS system."
(with-asdf-session () ;; NB: We don't cache the results. We once used to, but it wasn't useful,
;; and keeping a negative cache was a bug (see lp#1335323), which required
;; explicit invalidation in clear-system and find-system (when unsucccessful).
(let* ((name (coerce-name name))
(previous (registered-system name)) ; load from disk if absent or newer on disk
(primary (registered-system (primary-system-name name)))
(