Commit 4e2c71b2 authored by Daniel Barlow's avatar Daniel Barlow
Browse files

beginnings of a semi-formal defsystem syntax description

parent b02e59bc
asdf: another system definition facility -*- Text -*- asdf: another system definition facility -*- Text -*-
$Id: README,v 1.8 2002/02/11 14:01:49 dan_b Exp $ $Id: README,v 1.9 2002/02/12 12:24:00 dan_b Exp $
This system definition utility talks in terms of 'components' and This system definition utility talks in terms of 'components' and
'operations'. 'operations'.
...@@ -24,13 +24,9 @@ the grunt work. ...@@ -24,13 +24,9 @@ the grunt work.
asdf is extensible to new operations and to new component types. This asdf is extensible to new operations and to new component types. This
allows the addition of behaviours: for example, a new component could allows the addition of behaviours: for example, a new component could
be added for Java JAR archives, and methods specialised on be added for Java JAR archives, and methods specialised on
compile-system added for it that would accomplish the relevant compile-op added for it that would accomplish the relevant
actions. actions.
asdf is not presently extensible to new syntax (for example, to
emulate other defsystem utilities), but that's on the cards for a
future version.
* Inspiration * Inspiration
** mk-defsystem (defsystem-3.x) ** mk-defsystem (defsystem-3.x)
...@@ -79,10 +75,8 @@ Available in updated-for-CL form on the web at ...@@ -79,10 +75,8 @@ Available in updated-for-CL form on the web at
http://world.std.com/~pitman/Papers/Large-Systems.html http://world.std.com/~pitman/Papers/Large-Systems.html
In our implementation we borrow kmp's overall PROCESS-OPTIONS and In our implementation we borrow kmp's overall PROCESS-OPTIONS and
PROCESS-OPTION concepts to deal with creating component trees from concept to deal with creating component trees from defsystem surface
defsystem surface syntax. We haven't yet decided whether or how to syntax.
present this to users (i.e to specify it) but that will probably
happen one way or another.
* component * component
...@@ -105,12 +99,12 @@ and the method that you use to ask about dependencies ...@@ -105,12 +99,12 @@ and the method that you use to ask about dependencies
Dependencies are between (operation component) pairs. In your Dependencies are between (operation component) pairs. In your
initargs, you can say initargs, you can say
:in-order-to ((compile-system (load-system "a" "b") (compile-system "c")) :in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
(load-system (load-system "foo"))) (load-op (load-op "foo")))
- before performing the compile-system operation on this component, we - before performing compile-op on this component, we must perform
must perform the load-system operation on each of "a" and "b", and load-op on each of "a" and "b", and before performing load-op, we have
before performing load-system, we have to load "foo" to load "foo"
[ This is pretty similar to what ACL defsystem does. mk-defsystem is [ This is pretty similar to what ACL defsystem does. mk-defsystem is
less general: it has an implied dependency less general: it has an implied dependency
...@@ -129,7 +123,7 @@ In asdf, the dependency information for a given component and ...@@ -129,7 +123,7 @@ In asdf, the dependency information for a given component and
operation can be queried using (depends-on operation component), which operation can be queried using (depends-on operation component), which
returns a list returns a list
((load-system "a") (load-system "b") (compile-system "c") ...) ((load-op "a") (load-op "b") (compile-op "c") ...)
depends-on can be subclassed for more specific component/operation depends-on can be subclassed for more specific component/operation
types: these need to (call-next-method) and append the answer to their types: these need to (call-next-method) and append the answer to their
...@@ -140,26 +134,35 @@ the default dependencies ...@@ -140,26 +134,35 @@ the default dependencies
do this transparently. But, we need to support CLISP. If you have do this transparently. But, we need to support CLISP. If you have
the time for some CLISP hacking, I'm sure they'd welcome your fixes) the time for some CLISP hacking, I'm sure they'd welcome your fixes)
*** a filesystem location *** a pathname
Optional, may be inferred from name, type (the subclass of Optional, may be inferred from name, type (the subclass of
source-file), and location of parent. source-file), and location of parent.
The rules for this inference are: The rules for this inference are:
(for source-files)
- the host is taken from the parent - the host is taken from the parent
- the pathname type is (rassoc classname *known-extensions* :test 'equal) - pathname type is (rassoc classname *known-extensions* :test 'equal)
- the pathname case option is :local - the pathname case option is :local
- the pathname is merged against the parent - the pathname is merged against the parent
[ Pathname case can be controlled by some per-component option whose (for modules)
name I have not yet decided ] - the host is taken from the parent
- the name and type are NIL
- the directory is (:relative component-name)
- the pathname case option is :local
- the pathname is merged against the parent
Note that a component doesn't actually have any references to its Note that a component doesn't actually have any references to its
parent, so this inference will only be valid in the context of a parent, so this inference will only be valid in the context of a
system operation (a special variable is bound to the location of system operation (a special variable is bound to the location of
the parent while a child is processed) the parent while a child is processed)
Note that the DEFSYSTEM operator (used to create a "top-level" system)
does additional processing to set the filesystem location of the
top component in that system. This is detailed elsewhere
** components are subclassed into ** components are subclassed into
*** 'source-file' *** 'source-file'
...@@ -211,7 +214,7 @@ defined to accept. ...@@ -211,7 +214,7 @@ defined to accept.
** standard operations ** standard operations
*** compile-system &key proclamations *** compile-op &key proclamations
The definition of `compile' is "do everything that we can do to make The definition of `compile' is "do everything that we can do to make
subsequent loads (in a new image that we didn't do the compile in)" subsequent loads (in a new image that we didn't do the compile in)"
...@@ -220,9 +223,9 @@ faster. ...@@ -220,9 +223,9 @@ faster.
If proclamations are supplied, they will be proclaimed. This is a If proclamations are supplied, they will be proclaimed. This is a
good place to specify optimization settings good place to specify optimization settings
*** load-system *** load-op
This is the `do everything' operation. It depends on compile-system. This is the `do everything' operation. It depends on compile-op.
*** test-system-version &key minimum *** test-system-version &key minimum
...@@ -259,55 +262,137 @@ subclass operation, provide methods for source-file for ...@@ -259,55 +262,137 @@ subclass operation, provide methods for source-file for
* System definitions * System definitions
** Finding them ** System designators
Strings and symbols are treated as system designators. They are
transformed into actual system objects using
(find-system name) => a MODULE object (find-system system-designator) => a MODULE object
Systems are named by strings. If a system is not present in memory In the case of a symbol,
when asked for, attempts are made to load a file which is expected to - the symbol name is taken
define it. This file has the name of the system, the type "system", - if the system name is monocase and not in customary case for the
and is looked for in each of the directories given by evaluating filesystem, its case is inverted
members of *central-registry*
A system may also be referred to by symbol. In this case the symbol No case translation is done for systems named by strings
name is taken before a lookup is performed.
This attempts to find the system in a file on the disk. This file has
the name of the system, the type "system", and is looked for in each
of the directories given by evaluating members of *central-registry*.
The first matching file is considered, whether or not it turns out to
actually define the appropriate system
If a suitable file exists, it is loaded if
- there is no system of that name in memory,
- the file's last-modified time exceeds the last-modified time of the
system in memory
When system definitions are loaded from .system files, a new scratch When system definitions are loaded from .system files, a new scratch
package is created for them to load into, so that different systems package is created for them to load into, so that different systems do
do not overwrite each others operations. The user may also wish not overwrite each others operations. The user may also wish to (and
to include defpackage and in-package forms in his system definition is recommended to) include defpackage and in-package forms in his
files, however, so that they can be loaded manually if need be. system definition files, however, so that they can be loaded manually
if need be.
** Syntax ** Syntax
There is a defsystem form which looks basically similar to Systems can always be constructed programmatically by instantiating
mk-defsystem syntax components using make-instance. For most purposes, however, it is
likely that people will want a static defystem form.
asdf has `pluggable syntax', based around the principle that
components should not have to know defsystem syntax. That is, the
initargs that a component accepts are not necessarily related to the
defsystem form which creates it. defsystem forms are parsed by
methods of generic functions which can be specialised for new
syntaxes.
A defsystem parser must implement a `defsystem' macro, which can
be named for compatibility with whatever other system definition
utility is being emualted. It probably would look something like this:
(defmacro mk:defsystem (name &rest options)
(parse-component-form mk-syntax ,name ,options))
Internally the parser implements a method for parse-component-form,
which is called in the following fashion:
(parse-component-form syntax parent '(:module name :op1 val1 :op2 (blah blah)))
It should return a component. It gets the parent component available
to it, and should test whether the component to be created already
exists (as it might in a system redefinition) and update it in place
if so
[ Why are we specifying this? The complete parsing is under the
control of the person who implements a defsystem macro: given that we
don't even do the recursion for him, is there any point in our
specifying a protocol that he should use? Useful stuff we could
write, though, is the interface that the defsystem form thus created
should use to register the new system where find-system can get at it ]
** Native syntax
The native syntax is inspired by mk-defsystem, to the extent that it
should be possible to take most straightforward mk- system definitions
and run them unchanged. For my convenience, this turns out to be
basically the same as the initargs to the various components, with
a few extensions for convenience
system-definition := ( defsystem system-designator {option}* )
option := :components component-list
| :pathname pathname
| :default-component-class
| :perform method-form
| :explain method-form
| :output-files method-form
| :operation-done-p method-form
| :depends-on ( {simple-component-name}* )
| :serialize [ t | nil ]
| :in-order-to
We extend the defsystem syntax to allow for eql-specialised methods on component-list := ( {component-def}* )
components -
component-def := simple-component-name
| ( component-type name {option}* )
:module "foo" :components ("bar" "baz" "quux") component-type := :module | :file | :system | other-component-type
:perform (compile-system :after (op c)
(do-something c))
:explain (compile-system :after (op c)
(explain-something c))
For example
(defsystem "foo"
:components ((:module "foo" :components (:serial "bar" "baz" "quux")
:perform (compile-op :after (op c)
(do-something c))
:explain (compile-op :after (op c)
(explain-something c)))
(:file "blah")))
The method-form tokens need explaining: esentially,
:perform (compile-op :after (op c)
(do-something c))
:explain (compile-op :after (op c)
(explain-something c)))
has the effect of has the effect of
(defmethod perform :after ((op compile-system) (c (eql ...))) (defmethod perform :after ((op compile-op) (c (eql ...)))
(do-something c)) (do-something c))
(defmethod explain :after ((op compile-system) (c (eql ...))) (defmethod explain :after ((op compile-op) (c (eql ...)))
(explain-something c)) (explain-something c))
where ... is the component in question; note that while :before where ... is the component in question; note that while :before
methods are also supported by this, they may not do what you want them methods are also supported by this, they may not do what you want them
do (a :before method on perform ((op compile-system) (c (eql ...))) do (a :before method on perform ((op compile-op) (c (eql ...)))
will run after all the dependencies and sub-components have been will run after all the dependencies and sub-components have been
processed, but before the component in question has been processed, but before the component in question has been
compiled). The mk-alike :initially-do and :finally-do may be what you compiled).
are looking for.
How do we convert the defsystem form into a system?
* Error handling * Error handling
...@@ -319,31 +404,26 @@ Operations may go wrong (for example when source files contain ...@@ -319,31 +404,26 @@ Operations may go wrong (for example when source files contain
errors). These are signalled using generalised instances of errors). These are signalled using generalised instances of
OPERATION-ERROR OPERATION-ERROR
---------------------------------------------------------- ----------------------------------------------------------
TODO List TODO List
---------------------------------------------------------- ----------------------------------------------------------
defsystem is a macro that basically hands off its arguments to
parse-module-definition. parse-system-definition
* Outstanding spec questions, things to add * Outstanding spec questions, things to add
** Finding code: ":source-pathname " is not really good enough unless ** initially-do
LPNs suddenly develop union filesystem semantics
[ We're looking to get the effect of search lists: "try here, here or The mk-alike :initially-do and :finally-do are ugly, but are they
here" ] doing anything that's (a) useful, and (b) not convenient with
:perform :after?
** case-sensitivity wrt filenames ** case-sensitivity wrt filenames
Convention is to refer to systems with symbols, which means they end Presently we do no case munging except for symbols in find-system. To
up being named by uppercase strings. To find a .system file we find a .system file we (make-pathname :name name :type "SYSTEM" :case
(make-pathname :name name :type "SYSTEM" :case :common) ("use the file :common) ("use the file system's customary case") which translates
system's customary case") which translates this uppercase into this uppercase into whatever case is nicest (lowercase for sbcl on
whatever case is nicest (lowercase for sbcl on unix) unix)
We have excitingly chunky rules for merging component names, which We have excitingly chunky rules for merging component names, which
need to be documented. Oh, and implemented. need to be documented. Oh, and implemented.
...@@ -351,45 +431,22 @@ need to be documented. Oh, and implemented. ...@@ -351,45 +431,22 @@ need to be documented. Oh, and implemented.
** symbols vs strings ** symbols vs strings
Warn the user that they should either use keywords or be careful with Warn the user that they should either use keywords or be careful with
the package that they evaluate defsystem forms in the package that they evaluate defsystem forms in. Otherwise
(defsystem partition ...) being read in the cl-user package will
intern a cl-user:partition symbol, which will then collide with the
partition:partition symbol
<Krystof> You don't want to have (defsystem partition ...) being read in the ** error reporting
+cl-user package
<Krystof> Because that will intern a cl-user:partition symbol
<rahul> yeah, accidental interning could be a problem
<Krystof> Which will then collide with the partition:partition symbol
<Krystof> Hence (defsystem :partition ...)
** defsystem syntax
*** export process-option-list?
*** shortcut to setting up dependencies using :serial, :parallel
options.
*** split into native syntax and mixin for mk- syntax extras
** compiler/loader options
verbosity, proclamations, etc. would be nice to be able to override
or supplement this per-component as well
** the :pathname argument is misnamed
As demonstrated by the way we split it into name and type ourselves
instead of using parse-namestring (to avoid the problem that "foo.lisp"
is an absolute directory when treated as an LPN). As it doesn't behave
like a pathname, we shouldn't fool the user into thinking it is one
I think we should lose its funny behaviour and introduce a :type option
Need an explicit error for "dependency missing" and "required version
of dependency missing" so that we can fit this to an automated
dependency downloading thing
* missing bits in implementation * missing bits in implementation
** all of the above ** all of the above
** reuse the same scratch package whenever a system is reloaded from disk ** reuse the same scratch package whenever a system is reloaded from disk
** defsystem syntax for EQL methods ** defsystem syntax for EQL methods
** versions ** versions
** operation instantiation in traverse sucks
we don't transfer arguments from one op to the next in any meaningful
way. if compile calls load calls compile, how do we access the
original proclamations?
** test suite, insofar as it makes sense ** test suite, insofar as it makes sense
** compiler/loader options
verbosity, proclamations, etc: do this with specials
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment