Commit 69bbe281 authored by Robert P. Goldman's avatar Robert P. Goldman
Browse files
parents dc5cee32 9414c493
...@@ -48,13 +48,6 @@ ...@@ -48,13 +48,6 @@
Publisher = {Springer} Publisher = {Springer}
} }
Article{XCVB,
Author = {Fran\c{c}ois-Ren\'{e} Rideau and Spencer Brody},
Title = {{XCVB}: Improving Modularity for {C}ommon {L}isp},
Booktitle={International Lisp Conference},
URL="http://common-lisp.net/projects/xcvb/",
Year = {2009}
}
@InProceedings{XCVB, @InProceedings{XCVB,
author = {Fran\c{c}ois-Ren\'{e} Rideau and Spencer Brody}, author = {Fran\c{c}ois-Ren\'{e} Rideau and Spencer Brody},
title = {{XCVB}: an {eXtensible} {Component} {Verifier} and {Builder} title = {{XCVB}: an {eXtensible} {Component} {Verifier} and {Builder}
...@@ -172,7 +165,7 @@ Article{XCVB, ...@@ -172,7 +165,7 @@ Article{XCVB,
number = {4}, number = {4},
year = {1979}, year = {1979},
pages = {255-65}, pages = {255-65},
bibsource = {DBLP, http://dblp.uni-trier.de} bibsource = {DBLP, http://dblp.uni-trier.de/}
} }
@Article{Sidebotham98, @Article{Sidebotham98,
...@@ -186,6 +179,5 @@ Article{XCVB, ...@@ -186,6 +179,5 @@ Article{XCVB,
@Misc{scons, @Misc{scons,
key = {SCons}, key = {SCons},
title = {SCons website}, title = {SCons website},
url = {www.scons.org}} url = {http://www.scons.org/}
}
...@@ -64,7 +64,7 @@ such as distributing software, managing bugs, connecting people, ...@@ -64,7 +64,7 @@ such as distributing software, managing bugs, connecting people,
solving dependency versioning issues (aka ``DLL hell''), etc. solving dependency versioning issues (aka ``DLL hell''), etc.
\subsection{Analogy with \texttt{make}} % {\make} looks weird here \subsection{Analogy with {\large \make}}
\label{sec:analogy-with-make} \label{sec:analogy-with-make}
It is conventional to explain {\ASDF} as the {\CL} analog of {\make}~\cite{Feldman79}: It is conventional to explain {\ASDF} as the {\CL} analog of {\make}~\cite{Feldman79}:
...@@ -107,12 +107,12 @@ simple pattern matching rules used by an inference engine ...@@ -107,12 +107,12 @@ simple pattern matching rules used by an inference engine
that will run simply parameterized shell scripts. that will run simply parameterized shell scripts.
{\make} is a powerful tool that can express arbitrary programs, {\make} is a powerful tool that can express arbitrary programs,
but the result is a mesh of code that defies any simple analysis. but the result is a mesh of code that defies any simple analysis.
{\ASDF} is a small \CL{} program, {\ASDF} is a small {\CL} program,
and its system definitions are data rather than programs. and its system definitions are data rather than programs.
Any non-trivial code required by an \ASDF{} system definition Any non-trivial code required by an {\ASDF} system definition
has to be expressed either as an extension to {\ASDF} has to be expressed as an extension to {\ASDF},
or as ancillary method definitions for one of the generic functions typically as method definitions for one of the generic functions
that defines the \ASDF{} protocol. that defines the {\ASDF} protocol.
Executing code in the components being built is natural with {\ASDF} Executing code in the components being built is natural with {\ASDF}
and not with {\make} because, and not with {\make} because,
...@@ -121,26 +121,31 @@ the {\CL} compiler and loader are imperative languages. ...@@ -121,26 +121,31 @@ the {\CL} compiler and loader are imperative languages.
When building C programs, side-effects therefore have to be specified When building C programs, side-effects therefore have to be specified
in a different language, typically the shell. in a different language, typically the shell.
When building {\CL} programs, When building {\CL} programs,
side-effects can be written in Lisp itself. side-effects are written in Lisp itself.
Being able to do everything in Lisp without cross-language barriers Being able to do everything in Lisp without cross-language barriers
is a conceptual simplification; is a conceptual simplification;
however, you pay a price in side-effects. however, you pay a price for using side-effects.
% I don't believe the following helps the reader a lot, and our purpose is only \hide{
% to explain make to the extent that it helps understand ASDF... [2010/08/23:rpg] \rtof{I don't believe the following helps the reader a lot, and our purpose is only
% On the one hand, this means that {\ASDF} itself to explain make to the extent that it helps understand ASDF... [2010/08/23:rpg]}
% is much less powerful than {\make}, and On the one hand, this means that {\ASDF} itself
% offers very restricted system definition language. is much less powerful than {\make}, and
% On the other hand, {\ASDF} is part of a whole offers very restricted system definition language.
% that is just as powerful as {\make}, On the other hand, {\ASDF} is part of a whole
% and the strictures of system definitions allow for that is just as powerful as {\make},
% semantically richer analysis and manipulation of such, and the strictures of system definitions allow for
% with tools such as automatic packagers. semantically richer analysis and manipulation of such,
with tools such as automatic packagers.
}
A final difference between \ASDF{} and \make{}, which we will discuss more in A final difference between {\ASDF} and {\make}
Section~\ref{sec:traverse-bug-fix}, is that \ASDF{} builds a full plan for its is that {\ASDF} generates a full plan of the actions required
operations before executing them. By contrast, \make{} performs actions eagerly to fulfill all the dependencies of its goal
during a traversal of its dependency tree~\cite{AITR-874}. before performing the planned actions.
By contrast, {\make} performs actions
as it traverses its dependency tree~\cite{AITR-874}.
We will discuss this further in Section~\ref{sec:plan-generation},
\subsection{Basic {\ASDF} object model} \subsection{Basic {\ASDF} object model}
...@@ -170,7 +175,7 @@ during a traversal of its dependency tree~\cite{AITR-874}. ...@@ -170,7 +175,7 @@ during a traversal of its dependency tree~\cite{AITR-874}.
(:file "goodbye"))))) (:file "goodbye")))))
\end{verbatim} \end{verbatim}
\end{minipage} \end{minipage}
\caption{Sample {\ASDF} system definition, adapted from ASDF manual.} \caption{Sample {\ASDF} system definition.}
\label{fig:sampleASD} \label{fig:sampleASD}
\end{figure} \end{figure}
...@@ -203,7 +208,7 @@ This is done by the generic function \lisp{find-system}, which searches ...@@ -203,7 +208,7 @@ This is done by the generic function \lisp{find-system}, which searches
for the system definition in a configurable system registry for the system definition in a configurable system registry
(and its in-memory cache). (and its in-memory cache).
The system definition search is one of the aspects of {\ASDFi} that we reformed; The system definition search is one of the aspects of {\ASDFi} that we reformed;
the previous protocol had to be initialized using arbitrary lisp code, the previous protocol had to be initialized using arbitrary Lisp code,
and had a number of undesirable features. and had a number of undesirable features.
See Section \ref{sec:input-locations}. See Section \ref{sec:input-locations}.
{\operate} then generates a \emph{plan} for completing the {\operation} {\operate} then generates a \emph{plan} for completing the {\operation}
...@@ -220,6 +225,7 @@ making it easy to fix simple mistakes without interrupting the build. ...@@ -220,6 +225,7 @@ making it easy to fix simple mistakes without interrupting the build.
\subsection{Plan generation} \subsection{Plan generation}
\label{sec:plan-generation}
The semantics of an {\ASDF} system definition only specifies The semantics of an {\ASDF} system definition only specifies
ordering dependencies between build steps ordering dependencies between build steps
...@@ -339,12 +345,12 @@ We discuss this further in Section \ref{sec:future-directions}. ...@@ -339,12 +345,12 @@ We discuss this further in Section \ref{sec:future-directions}.
\rtof{ \rtof{
I don't believe that's correct. The rules in the makefile are not the I don't believe that's correct. The rules in the makefile are not the
plan --- they are the pieces out of which a plan is assembled (if \make{} plan --- they are the pieces out of which a plan is assembled (if {\make}
plans; I am not convinced this is the case). The parallel is the system definition in plans; I am not convinced this is the case). The parallel is the system definition in
\ASDF{}. That is not a plan; that is what the plan is assembled out of. For {\ASDF}. That is not a plan; that is what the plan is assembled out of. For
example, if I ask \ASDF{} to do a load-op on a system, it will first build a example, if I ask {\ASDF} to do a load-op on a system, it will first build a
plan, an ordered sequence of operation - component pairs. \emph{Then} it will plan, an ordered sequence of operation - component pairs. \emph{Then} it will
execute that plan. I don't believe that \make{} does anything like this. In execute that plan. I don't believe that {\make} does anything like this. In
fact, I don't understand how make actually does its job. Similarly, I can fact, I don't understand how make actually does its job. Similarly, I can
have a bunch of rules about how to build the pdf of the ASDF doc from texinfo, but those have a bunch of rules about how to build the pdf of the ASDF doc from texinfo, but those
rules will not be part of the plan if I ask make to build the html version of rules will not be part of the plan if I ask make to build the html version of
......
...@@ -20,7 +20,7 @@ ...@@ -20,7 +20,7 @@
% } % }
% \draft{Symbolics \defsys{} here... Kantrowitz claims that it is a % \draft{Symbolics {\defsys} here... Kantrowitz claims that it is a
% \emph{procedural}, rather than a structural system definition tool. Cites the % \emph{procedural}, rather than a structural system definition tool. Cites the
% BUILD system~\cite{AITR-874} as a structural tool.} % BUILD system~\cite{AITR-874} as a structural tool.}
...@@ -30,7 +30,7 @@ ...@@ -30,7 +30,7 @@
% \begin{itemize} % \begin{itemize}
% \item More declarative than other systems; % \item More declarative than other systems;
% \item Declarative specification of transformations between \emph{grains} (not % \item Declarative specification of transformations between \emph{grains} (not
% offered by various flavors of \defsys{}. % offered by various flavors of {\defsys}.
% \item Clear treatment of distinction between macro dependencies (compile-time) % \item Clear treatment of distinction between macro dependencies (compile-time)
% and call dependencies (run-time --- load time in our jargon). % and call dependencies (run-time --- load time in our jargon).
% \item Dependencies are described directly, rather than in terms of system % \item Dependencies are described directly, rather than in terms of system
...@@ -43,41 +43,41 @@ ...@@ -43,41 +43,41 @@
% \end{itemize} % \end{itemize}
% } % }
A key inspiration for ASDF was \mkdefsys{}, Mark Kan\-tro\-witz's portable A key inspiration for ASDF was {\mkdefsys}, Mark Kan\-tro\-witz's portable
\defsys{} facility~\cite{kantrowitz:91}. At the time when \mkdefsys{} {\defsys} facility~\cite{kantrowitz:91}. At the time when {\mkdefsys}
was developed, there was no portable, non-proprietary system definition was developed, there was no portable, non-proprietary system definition
facility for {\CL}, as Lisp moved off special-purpose platforms and onto facility for {\CL}, as Lisp moved off special-purpose platforms and onto
general-purpose hardware. Prior to this (and substantially prior to a true general-purpose hardware. Prior to this (and substantially prior to a true
\emph{Common} Lisp), there were a number of different system-defining \emph{Common} Lisp), there were a number of different system-defining
facilities, notably the Symbolics \defsys{}~\cite{CHINE-NUAL}\footnote{Cited by Robbins~\cite{AITR-874}}, but people wanting to facilities, notably the Symbolics {\defsys}~\cite{CHINE-NUAL}\footnote{Cited by Robbins~\cite{AITR-874}}, but people wanting to
portably define systems had to rely on \lisp{LOAD} scripts and/or portably define systems had to rely on \lisp{LOAD} scripts and/or
\lisp{REQUIRE}. \lisp{REQUIRE}.
The \ASDF{} manual~\cite{ASDF-Manual}, discussing \mkdefsys{} as an inspiration, The {\ASDF} manual~\cite{ASDF-Manual}, discussing {\mkdefsys} as an inspiration,
explains that it was intended to better use \CL{} features (notably CLOS) for explains that it was intended to better use {\CL} features (notably CLOS) for
extensibility. \mkdefsys{} is written in pre-CLOS \CL{}. However, we argue extensibility. {\mkdefsys} is written in pre-CLOS {\CL}. However, we argue
that a primary reason for \ASDF{}'s success was not its CLOS architecture, but that a primary reason for {\ASDF}'s success was not its CLOS architecture, but
its elegant use of \lisp{*load-truename*} to solve the social problem of its elegant use of \lisp{*load-truename*} to solve the social problem of
installing and referencing lisp libraries. The problem of \emph{installing} installing and referencing Lisp libraries. The problem of \emph{installing}
lisp libraries was not helped by \mkdefsys{}, but was substantially eased by \ASDF{}. lisp libraries was not helped by {\mkdefsys}, but was substantially eased by {\ASDF}.
See See
Section~\ref{sec:input-locations} for more discussion of this issue. Section~\ref{sec:input-locations} for more discussion of this issue.
BUILD~\cite{AITR-874} was a substantially earlier Lisp build system (antedating BUILD~\cite{AITR-874} was a substantially earlier Lisp build system (antedating
\CL{}), and an inspiration behind \mkdefsys{}. {\CL}), and an inspiration behind {\mkdefsys}.
BUILD attempted to be more declarative than \make{} and other predecessors. It BUILD attempted to be more declarative than {\make} and other predecessors. It
\emph{may} have introduced the notion of creating a complete operation plan \emph{may} have introduced the notion of creating a complete operation plan
before executing any operations; we are not certain. before executing any operations; we are not certain.
% \draft{Too-specific notes on \mkdefsys{} that need to be winnowed for the % \draft{Too-specific notes on {\mkdefsys} that need to be winnowed for the
% paper.... % paper....
% Some key ideas present in \mkdefsys{}: % Some key ideas present in {\mkdefsys}:
% \begin{itemize} % \begin{itemize}
% \item parametric \texttt{operate-on-system} function. % \item parametric \texttt{operate-on-system} function.
% \item selective recompilation to minimize work % \item selective recompilation to minimize work
% \item declarative system spec --- specify dependencies and components; % \item declarative system spec --- specify dependencies and components;
% \mkdefsys{} infers the procedure. % {\mkdefsys} infers the procedure.
% \item intended to be extensible % \item intended to be extensible
% \item Use of Unix pathnames as de facto standard --- portability across unix % \item Use of Unix pathnames as de facto standard --- portability across unix
% and mac was punted to logical pathnames. % and mac was punted to logical pathnames.
...@@ -87,7 +87,7 @@ before executing any operations; we are not certain. ...@@ -87,7 +87,7 @@ before executing any operations; we are not certain.
% \emph{Not} present: % \emph{Not} present:
% \begin{itemize} % \begin{itemize}
% \item Clever use of \texttt{*load-truename*} to locate source files. The % \item Clever use of \texttt{*load-truename*} to locate source files. The
% problem of \emph{installing} systems was not helped by \mkdefsys{}. At the % problem of \emph{installing} systems was not helped by {\mkdefsys}. At the
% top, \texttt{:SYSTEM}s had to have absolute pathnames. % top, \texttt{:SYSTEM}s had to have absolute pathnames.
% \item Automatic placement of binary files. I believe this had to be handled % \item Automatic placement of binary files. I believe this had to be handled
% through use of conditional compilation (reader macros) combinated with % through use of conditional compilation (reader macros) combinated with
...@@ -103,7 +103,7 @@ before executing any operations; we are not certain. ...@@ -103,7 +103,7 @@ before executing any operations; we are not certain.
% Features not taken over into ASDF: % Features not taken over into ASDF:
% \begin{itemize} % \begin{itemize}
% \item \mkdefsys{} had \texttt{:PRIVATE-FILE} as a component type that % \item {\mkdefsys} had \texttt{:PRIVATE-FILE} as a component type that
% could be used for, e.g., user-specific configuration files. % could be used for, e.g., user-specific configuration files.
% \item You can do package-wrangling in the system definition, instead of coding % \item You can do package-wrangling in the system definition, instead of coding
% package into files. % package into files.
......
This diff is collapsed.
\section{Dynamic Code Upgrade} \section{Dynamic Code Update}
\label{sec:upgradeability} \label{sec:upgradeability}
...@@ -38,8 +38,7 @@ we achieve better {\it a posteriori} coupling of release cycles. ...@@ -38,8 +38,7 @@ we achieve better {\it a posteriori} coupling of release cycles.
\subsection{Technical Challenge} \subsection{Technical Challenge}
Unlike Unlike other build systems, such as {\make},
other build systems, such as {\make},
\moneyquote{{\ASDF} is an ``in-image'' build system \moneyquote{{\ASDF} is an ``in-image'' build system
managing systems that are compiled and loaded in the current {\CL} image}. managing systems that are compiled and loaded in the current {\CL} image}.
...@@ -61,23 +60,24 @@ We believe that this there are a number of reasons for this design decision: ...@@ -61,23 +60,24 @@ We believe that this there are a number of reasons for this design decision:
Previous build systems were designed around that constraint, Previous build systems were designed around that constraint,
and {\ASDF} followed their design. and {\ASDF} followed their design.
\item Even on modern operating systems that allow this virtualization, \item Even on modern operating systems that allow this virtualization,
starting a {\CL} process is often a relatively expensive process (depending on starting a {\CL} process is often a relatively expensive process
the individual \CL{} implementation). (depending on the individual {\CL} implementation).
\item Because of the presence of code to be executed at compile time and the \item Because of the presence of code to be executed at compile time and
dependence on a substantial amount of compile-time state, the dependence on a substantial amount of compile-time state,
it is difficult to decompose the process of building a {\CL} system it is difficult to decompose the process of building a {\CL} system
into independent pieces and parcel them out to different processes. into independent pieces and parcel them out to different processes.
\item {\CL} programmers often built very extensive state \item {\CL} programmers often build very extensive state
in a long-living {\CL} image, and so prefer to keep them alive. \ASDF{} in a long-living {\CL} image, and so prefer to keep them alive.
supports such a use pattern. {\ASDF} supports such a use pattern.
\end{itemize} \end{itemize}
The Ytools ``chunk manager'' (see Section \ref{sec:related-works}) % \ftor{I don't think this is relevant here, only a distraction.}
takes the \ASDF{} approach to an extreme, maintaining %The Ytools ``chunk manager'' (see Section \ref{sec:related-works})
code units smaller than file-size %takes the {\ASDF} approach to an extreme,
to maintain the integrity of long-lived {\CL} images. %maintaining code units smaller than file-size
On the other hand, XCVB (Section \ref{sec:XCVB}) takes a different perspective, %to maintain the integrity of long-lived {\CL} images.
attempting to isolate system maintenance tasks in separate processes. %On the other hand, XCVB (Section \ref{sec:XCVB}) takes a different perspective,
%attempting to isolate system maintenance tasks in separate processes.
Because {\ASDF} performs its build tasks in the user's current Lisp process, Because {\ASDF} performs its build tasks in the user's current Lisp process,
upgrading {\ASDF} entails modifying some existing upgrading {\ASDF} entails modifying some existing
...@@ -132,7 +132,7 @@ as long as it behaves in a semantically equivalent way.\footnote{ ...@@ -132,7 +132,7 @@ as long as it behaves in a semantically equivalent way.\footnote{
For instance, assuming functions are seldom rebound, For instance, assuming functions are seldom rebound,
dynamic calls may be implemented just like static calls, dynamic calls may be implemented just like static calls,
except that the value of the binding is a cache that gets invalidated except that the value of the binding is a cache that gets invalidated
between the time the function is rebound and the cache is used. between the time the function is rebound and the time the cache is used.
As for static calls, they may be implemented As for static calls, they may be implemented
not just by linking a call to the proper code value, not just by linking a call to the proper code value,
...@@ -142,10 +142,10 @@ as long as it behaves in a semantically equivalent way.\footnote{ ...@@ -142,10 +142,10 @@ as long as it behaves in a semantically equivalent way.\footnote{
that will never be rebound. that will never be rebound.
} }
The two difficulties named above are inherent in redefining functions The two above difficulties are inherent in redefining functions
and are not specific to either {\CL} or {\ASDF}. and are not specific to either {\CL} or {\ASDF}.
However, these difficulties are particularly relevant in the case of {\ASDF} However, these difficulties are particularly relevant in the case of {\ASDF},
which drives compilation and loading of Lisp code that drives compilation and loading of Lisp code
possibly including new versions of {\ASDF} itself. possibly including new versions of {\ASDF} itself.
{\ASDF}'s code is therefore likely to be in the continuation {\ASDF}'s code is therefore likely to be in the continuation
of its own function redefinitions, of its own function redefinitions,
...@@ -245,7 +245,7 @@ Another problem is that {\unintern} runs the risk of causing ``collateral damage ...@@ -245,7 +245,7 @@ Another problem is that {\unintern} runs the risk of causing ``collateral damage
When a symbol has several bindings associated to it, When a symbol has several bindings associated to it,
such as a function or macro; variable or constant; type or class or condition; such as a function or macro; variable or constant; type or class or condition;
property, etc. property, etc.
All of these bindings will simultaneously become inaccessible All of these bindings will simultaneously become inaccessible
when the symbol is uninterned. when the symbol is uninterned.
Consider a user developing Consider a user developing
...@@ -258,7 +258,7 @@ in either of the latter systems, ...@@ -258,7 +258,7 @@ in either of the latter systems,
lest the previously loaded system be in an invalid, unusable state. lest the previously loaded system be in an invalid, unusable state.
Code in these systems may be Code in these systems may be
linked to obsolete, now-uninterned symbols from the old {\ASDF}. linked to obsolete, now-uninterned symbols from the old {\ASDF}.
For these client systems For these client systems
to function properly, they must be linked against to function properly, they must be linked against
the symbols from the new {\ASDF}. the symbols from the new {\ASDF}.
...@@ -323,7 +323,7 @@ Classes can be redefined, ...@@ -323,7 +323,7 @@ Classes can be redefined,
slots can be added to them, removed from them, or modified, slots can be added to them, removed from them, or modified,
and all instances will be automatically updated before their next use and all instances will be automatically updated before their next use
to fit the new definition. to fit the new definition.
The {\CL} Object System (CLOS) \cite{bobrow_etal88}) The {\longCLOS} ({\CLOS}) \cite{bobrow_etal88})
allows users to control this instance update programmatically allows users to control this instance update programmatically
by defining methods on {\uifrc}. by defining methods on {\uifrc}.
We rely on this functionality in {\ASDFii} We rely on this functionality in {\ASDFii}
...@@ -393,7 +393,7 @@ and therefore is not compatible with using existing libraries as black boxes. ...@@ -393,7 +393,7 @@ and therefore is not compatible with using existing libraries as black boxes.
Future Lisp standards and specifications could learn from Erlang. Future Lisp standards and specifications could learn from Erlang.
{\CL} users could incorporate Erlang-like semantics {\CL} users could incorporate Erlang-like semantics
in a mostly transparent way in a most\-ly transparent way
by layering a {\CL} implementation on top of {\CL}, by layering a {\CL} implementation on top of {\CL},
shadowing the usual reader and evaluator to replace them with something shadowing the usual reader and evaluator to replace them with something
that provides well-defined semantics for hot upgrade, that provides well-defined semantics for hot upgrade,
...@@ -401,7 +401,7 @@ assuming all code is (re)compiled on top of this implementation ...@@ -401,7 +401,7 @@ assuming all code is (re)compiled on top of this implementation
rather than directly with the underlying implementation. rather than directly with the underlying implementation.
This, however, would be a large challenging task and not obviously worth the cost. This, however, would be a large challenging task and not obviously worth the cost.
Furthermore, Furthermore,
if one was to design and implement if one were to design and implement
what amounts to a new language on top of {\CL}, what amounts to a new language on top of {\CL},
would it and should it be {\CL} all again? would it and should it be {\CL} all again?
Interestingly, in the presence of concurrent threads Interestingly, in the presence of concurrent threads
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment