Commit c6b0d0ef authored by Francois-Rene Rideau's avatar Francois-Rene Rideau
Browse files

Small tweaks.

parent db030da0
......@@ -264,7 +264,7 @@ force the (re)compilation of \texttt{a},
but having to (re)compile \texttt{b} will
force the (re)compilation of \texttt{a}.
Therefore, {\traverse} first issues \lisp{in-order-to} dependencies,
and \lisp{operation-done-p} status, and
and {\opdonep} status, and
if the operation needs to be done on the component,
issues \lisp{do-first} dependencies;
finally it issues the current operation and component pair
......@@ -274,7 +274,7 @@ While building a plan, {\traverse} avoids issuing unnecessary steps
and forcing steps that have \lisp{in-order-to} dependencies on them.
To determine whether a step not otherwise forced by a dependency
has already been done,
it calls a generic function \lisp{operation-done-p}.
it calls a generic function {\opdonep}.
For a compilation step, it compares the timestamps of
the \lisp{input-files} and \lisp{output-files}
of the {\operation} and {\component}.
......
......@@ -54,20 +54,30 @@ portably define systems had to rely on \lisp{LOAD} scripts and/or
\lisp{REQUIRE}.
The {\ASDF} manual~\cite{ASDF-Manual}, discussing {\mkdefsys} as an inspiration,
explains that it was intended to better use {\CL} features (notably CLOS) for
extensibility. {\mkdefsys} is written in pre-CLOS {\CL}. However, we argue
explains that it was intended to better use modern {\CL} capabilities.
Notably, {\mkdefsys} is written in pre-{\CLOS} {\CL}, whereas
{\ASDF} boasts use of {\CLOS} for features for extensibility,
in a way partly inspired by a design by Kent Pitman~\cite{Pitman-Large-Systems}.
However, we argue
that a primary reason for {\ASDF}'s success was not its CLOS architecture, but
its elegant use of \lisp{*load-truename*} to solve the social problem of
installing and referencing Lisp libraries. The problem of \emph{installing}
lisp libraries was not helped by {\mkdefsys}, but was substantially eased by {\ASDF}.
See
Section~\ref{sec:input-locations} for more discussion of this issue.
BUILD~\cite{AITR-874} was a substantially earlier Lisp build system (antedating
{\CL}), and an inspiration behind {\mkdefsys}.
BUILD attempted to be more declarative than {\make} and other predecessors. It
\emph{may} have introduced the notion of creating a complete operation plan
before executing any operations; we are not certain.
installing and referencing installed Lisp libraries.
The problem of \emph{installing} Lisp libraries was not helped by {\mkdefsys},
but was substantially eased by {\ASDF}.
See Section~\ref{sec:input-locations} for more discussion of this issue.
BUILD~\cite{AITR-874} was an earlier Lisp build system (antedating {\CL})
meant to replace the Symbolics {\defsys}.
It is cited as an influence to {\mkdefsys}.
BUILD attempted to be more declarative than {\make} and other predecessors.
It notably introduced the notion of automatically deducing dependencies
from a graph of inter-file references,
rather than having to manually declare rules that transitively enforce
recompilation upon file modification.
%% It \emph{may} have introduced the notion of creating a complete operation plan
%% before executing any operations; we are not certain.
% \draft{Too-specific notes on {\mkdefsys} that need to be winnowed for the
% paper....
......
......@@ -21,9 +21,10 @@
% in case we later want to typeset these differently...
\newcommand{\class}[1]{\texttt{#1}}
\newcommand{\file}[1]{\texttt{#1}}
\newcommand{\make}{\texttt{make(1)}}
\newcommand{\make}{\texttt{make}} % repeating the (1) was a bit unnerving
\newcommand{\ant}{\texttt{ant}}
\newcommand{\uifrc}{\lisp{update-instance-for-redefined-class}}
\newcommand{\opdonep}{\lisp{operation-done-p}}
\newcommand{\defclass}{\lisp{defclass}}
\newcommand{\defmethod}{\lisp{defmethod}}
\newcommand{\fmakunbound}{\lisp{fmakunbound}}
......@@ -45,11 +46,13 @@
\newcommand{\longCL}{Common Lisp}
\newcommand{\CL}{CL}
\newcommand{\ASDF}{ASDF}
\newcommand{\smallspace}{\hspace{.15em}}
\newcommand{\ASDFiii}{ASDF{\smallspace}3}
\newcommand{\smallspace}{\hspace{.3ex}}
\newcommand{\ASDFi}{ASDF{\smallspace}1}
\newcommand{\ASDFii}{ASDF{\smallspace}2}
\newcommand{\ASDFiii}{ASDF{\smallspace}3}
\newcommand{\POIU}{POIU}
\newcommand{\XCVB}{XCVB}
\newcommand{\ytools}{ytools}
\newcommand{\longABL}{\texttt{ASDF-Binary-Locations}}
\newcommand{\ABL}{\texttt{A-B-L}}
\newcommand{\CLC}{\texttt{C-L-C}}
......@@ -489,7 +492,8 @@ and since {\ABL} was not one of the core parts of {\ASDF}
for which we did guarantee backwards compatibility,
we designed a new mechanism, {\longAOT}.
{\longAOT} ({\AOT}) is based on a few Domain-Specific Languages (DSLs)
{\longAOT} ({\AOT}) is based on a few
Do\-main-Spe\-ci\-fic Languages (DSLs)
that allow to specify pathname patterns and translations in a simple way.
We gave it a sensible default configuration
that redirected the output of compiled files to
......@@ -671,10 +675,6 @@ without being foiled by translated output locations.
\section{Fixing {\Large \traverse}}
% \rtof{Moved some material from the backward compatiblity subsection here,
% because they are really about bug fixing, and not about backward compatibility
% (although they were \emph{limited} by backward compatibility concerns).}
% \ftor{I still think we should only talk about bug fixes from the point of view of
% a general lesson to learned. Otherwise, the reader won't care.
% I say we move it any of it back into the section ``Best Practices'',
......@@ -686,11 +686,6 @@ without being foiled by translated output locations.
% user extensibility, but also keeping things portable, or keeping interfaces
% simple.}
% \rtof{I don't have a strong opinion about this. I am going to write up the
% traverse bug fix, with some of its lessons learned, because it touches on a
% bunch of issues. Then we can move it when we know what we think is
% interesting about it.}
% There were also some implicit dependencies that seemed to be missing.
% For example (we fixed this one in {\ASDFii}),
% there was no implicit dependency indicating that
......@@ -720,13 +715,14 @@ without being foiled by translated output locations.
\label{fig:moduleBugExample}
\end{figure}
One of the most important bugs we fixed in {\ASDFii},
One of the most annoying bugs we fixed for {\ASDFii}
was a problem with dependencies
involving composite components (modules and systems).
%% What are we compatible with and why?
%% See lp#479522
one that illustrates the limits imposed by by backward compatibility,
It illustrates both design issues in {\ASDF}
and limits imposed on us by backward compatibility in solving those issues.
and one indicative of future issues in {\ASDF},
was a problem with dependencies
involving composite components (modules and systems).
% Why introduce name ``composite'' instead of just telling that systems are modules?
In {\ASDFi} there was a notorious bug that meant that
dependencies upstream of a composite component
......@@ -757,7 +753,7 @@ In general, it conflates together
and ``the action of operating on a composite'' in ways that are unfortunate.
% From there to end of section, I'm lost.
More specifically, because these synergistic actions are typically to be done
whenever a system is loaded, the \lisp{OPERATION-DONE-P} method for arbitrary
whenever a system is loaded, the {\opdonep} method for arbitrary
operations, would not work properly to indicate whether the operation had been
done for that composite, but only to indicate whether the
\emph{synergistic actions} had been performed.
......@@ -783,7 +779,7 @@ Trying to fix composite dependencies in {\traverse}, therefore,
opened a big can of worms.
{\ASDFi} originally contained special-purpose, \textit{ad hoc}, logic
for modules to decide when their components needed to be operated on
(since \lisp{operation-done-p} could not be used), and this
(since {\opdonep} could not be used), and this
special-purpose logic had some bugs.
Instead of radically re-engineering the logic, we simply repaired it in place
(and later {\fare} refactored it), fixing the bug illustrated in
......@@ -967,38 +963,19 @@ in a way that leaves as little space for undefined behavior as possible.
with respect to semantic discrepancies between underlying implementations,
we must abstract those discrepancies away.}
% \rtof{I didn't want to have this much repetition, so I condensed the following
% two paragraphs into one. I \emph{think} that's enough. If you agree, please
% cut the following two, and this note.}
% Most importantly, and as mentioned in section \ref{sec:pathnames} above,
% we had to implement our own well-defined syntax and semantics
% for specifying and using relative pathnames whose type component
% may have been independently specified.
% This was necessary because Lisp source code is distributed as
% filesystem subtrees where you need to name files that way,
% but {\CL} does not offer a direct way to specify and use such names.
% We based our syntax on the now semi-standard Unix pathname syntax.
% In addition, we had to deal with the symbol and package wrangling discussed
% in the above section on hot code upgrade. In order to eliminate
% all warnings on all implementations, we also had to be careful
% to include \lisp{ignorable} declarations in \lisp{defmethod}s, and
% to avoid forward references to yet undefined and undeclared functions.
% \rtof{End of the original material. Following is the proposed replacement.}
Most importantly, and as mentioned in Section \ref{sec:pathnames} above,
we had to implement our own well-defined syntax and semantics
for specifying and using relative pathnames whose type component
may have been independently specified.
% \rtof{Can we cut the following sentence or revise it to explain why this is a
% portability issue?}
% In addition, we had to deal with the symbol and package wrangling discussed
% in the section on hot code upgrade (Section \ref{sec:upgradeability}).
In order to eliminate
all warnings on all implementations, we also had to be careful
to include \lisp{ignorable} declarations in \lisp{defmethod}s, and
We also had to either rebind or shadow symbols for redefined functions
depending on implementation as discussed in Section \ref{sec:upgradeability}.
Additionally, in order to eliminate all warnings on all implementations,
we also had to be careful to
include \lisp{ignorable} declarations in \lisp{defmethod}s, and
avoid forward references to yet undefined and undeclared functions.
We also have several portable wrappers over implementation-specific functions
to access the environment, such as \lisp{getenv}, and
tens of small implementation-specific adaptations.
There is, however, one place where we took pains to be transparent
to cross-platform semantic discrepancies:
......@@ -1034,7 +1011,7 @@ The idea is that
\moneyquote{anything that can be provided as an extension
should be provided as an extension and left out of the core}.
{\ASDF} has been successfully extended to support such things as
FFI generation, syntax extension, etc.
FFI generation, syntax extensions, etc.
Although we tried to maintain simplicity,
we found that configuration had to go into the core of {\ASDF}.
You can't bootstrap the configuration of inputs and outputs as an extension,
......@@ -1092,7 +1069,61 @@ Documentation is a pain, and is still a weakness of {\ASDFii}.
\input{history}
\subsection{Other build tools}
\subsection{Competing {\CL} build tools}
% \ftor{I would put them in separate paragraphs.}
In the modern Lisp world, two alternatives to {\ASDF}
provide an interesting contrast.
{\XCVB}~\cite{XCVB} abandons {\ASDF}'s single-image model
to bring the pure functional way of building software in separate processes.
On the other hand, McDermott's {\ytools} system focuses on image maintenance,
allowing the programmer to specify dependencies for units smaller than files,
up to and including individual data structures.
\paragraph{\XCVB}
\label{sec:XCVB}
% squashed the "presented at ILC 2009" bit, because we don't give provenance of
% other papers/presentations.
{\XCVB}~\cite{XCVB} is a proposed replacement for {\ASDF}.
Where {\ASDF} builds software into the current ``One True'' Lisp world
in a context-dependent way,
{\XCVB} deterministically builds software
into multiple virtual Lisp worlds
(as many Unix processes).
Image maintenance is achieved through a small system \texttt{xcvb-master}
(one tenth the size of {\ASDFii}),
that spawns {\XCVB} as an external process to build software,
that it subsequently loads in the current image,
eliminating interferences between the current image and the build process.
{\XCVB} is currently working and
has many features that {\ASDF} doesn't have and could not be evolved to have,
including a deterministic build model, cross-compilation,
and enforcement of declared dependencies.
On the other hand, XCVB requires {\CL} software to be adapted
to its different build specification format and
its slightly stricter build constraints,
and it doesn't currently support as many target platforms as does {\ASDF}.
Development of XCVB is ongoing, but from the simple one-paragraph idea
to a satisfying product, an incredible --- and originally unforeseen ---
amount of work is needed.
XCVB has to handle most of
the issues that we faced in {\ASDFii},
and some others as well,
from the fine semantics of UNIX signals
to distinctions between host and target system
that appear in cross-compilation.
On the other hand, XCVB has the advantages of clean slate redesign on its side.
\paragraph{Yale tools}
\label{sec:ytools}
\fixme{ytools material here.}
\subsection{Build tools beyond {\CL}}
\label{sec:related-works}
% \draft{
......@@ -1143,6 +1174,7 @@ in Section~\ref{sec:analogy-with-make}.
Omake~\cite{OMake} is a notable modern take on the {\make} concept:
of interest to readers of this article are features
such as automated dependency analysis, sub-project management,
use of cryptographic checksums instead of timestamps,
and extension using a DSL (instead of relying on shell commands).
Possibly closer to {\ASDF} in flavor are integrated build systems such as
CONS~\cite{Sidebotham98} and SCONS~\cite{scons},
......@@ -1155,59 +1187,6 @@ to provide more declarative system specifications.
\fixme{Add citation for \texttt{ant}; a cite to the Sun Java web site would be
fine if we can't find a better. I couldn't find a nice short article.}
\subsection{XCVB and ytools}
% \ftor{I would put them in separate paragraphs.}
In the modern Lisp world, two {\ASDF} alternatives
provide an interesting contrast.
XCVB~\cite{XCVB} abandons {\ASDF}'s single-image model
to bring the pure functional way of building software in separate processes.
On the other hand, McDermott's ytools system focuses on image maintenance,
allowing the programmer to specify dependencies for units smaller than files,
up to and including individual data structures.
\paragraph{XCVB}
\label{sec:XCVB}
% squashed the "presented at ILC 2009" bit, because we don't give provenance of
% other papers/presentations.
XCVB~\cite{XCVB} is a proposed replacement for {\ASDF}.
Where {\ASDF} builds software into the current ``One True'' Lisp world
in a context-dependent way,
XCVB deterministically builds software
into multiple virtual Lisp worlds
(as many Unix processes).
Image maintenance is achieved through a tiny system \texttt{xcvb-master}
that spawns XCVB as an external process to build software
that it subsequently loads in the current image.
XCVB is currently working and
has many features that {\ASDF} doesn't have and could not be evolved to have,
including a deterministic build model, cross-compilation,
and enforcement of declared dependencies.
On the other hand, XCVB requires {\CL} software to be adapted
to its different build specification format and
its slightly stricter build constraints,
and it doesn't currently support as many target platforms as does {\ASDF}.
Development of XCVB is ongoing, but from the simple one-paragraph idea
to a satisfying product, an incredible --- and originally unforeseen ---
amount of work is needed.
XCVB has to handle most of
the issues that we faced in {\ASDFii},
and some others as well,
from the fine semantics of UNIX signals
to distinctions between host and target system
that appear in cross-compilation.
On the other hand, XCVB has the advantages of clean slate redesign on its side.
\paragraph{Yale tools}
\label{sec:ytools}
\fixme{ytools material here.}
\section{Future Directions}
......@@ -1274,7 +1253,7 @@ An alternative, possibly for {\ASDFiii} would be
to simply abandon backwards compatibility in this regard
(we do not know how many systems rely on it), and
redefining the semantics of operations on composites to have the more intuitive meaning.
This would allow us to give \lisp{operation-done-p},
This would allow us to give {\opdonep},
\lisp{input-files} and \lisp{output-files} clearer semantics.
One could, at the same time, introduce plan nodes that would permit nesting,
so that the operations performed on a composite's components would be scoped
......@@ -1283,7 +1262,7 @@ so that the operations performed on a composite's components would be scoped
\lisp{around} methods on \lisp{perform} as applied to composite components.
% At the same time, it would be appropriate to improve introspection
%by making {\traverse} part of the {\ASDF} API.
% by making {\traverse} part of the {\ASDF} API.
There have been calls to make {\traverse} part of the {\ASDF} API
in order to enable more introspection.
This would be especially useful for authors of
......@@ -1305,10 +1284,11 @@ value.\footnote{While this might seem an obvious addition, accumulating such a
error, but one would not want a test operation to raise an error and abort on
the first failure; typically one wants a report of all passing and failing
tests.}
One component type that has been repeatedly added by system definers (and often
in a buggy way) is that of
a ``load-only'' {\CL} source file. This is a file that is to be directly loaded
and \emph{never} compiled. Previous system defining facilities, including
One component type that has been repeatedly added by system definers
(and often in a buggy way) is that of
a ``load-only'' {\CL} source file.
This is a file that is to be directly loaded and \emph{never} compiled.
Previous system defining facilities, including
{\mkdefsys}~\cite{kantrowitz:91}, have included such a component type.
%Finally, and perhaps simultaneously most important and least appealing is
......@@ -1368,7 +1348,7 @@ does not work for others using their new classes.
% ancillary {\ASDF} systems (e.g., \lisp{FOO-TEST} as a complement to
% \lisp{FOO}).
% \item
% \lisp{operation-done-p} has odd semantics.
% {\opdonep} has odd semantics.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment