Commit 43e29bc9 authored by Peter Van Eynde's avatar Peter Van Eynde
Browse files

imported new upstream 1.501

parents ea3aa53b 7e34af2c
......@@ -21,3 +21,4 @@ lift-local.config
*.fas
*.lib
*.o
*.*fsl
......@@ -38,7 +38,7 @@ website-copy: FORCE
bin/rsync-cp.sh tmp/asdf.lisp $(webhome_private)
clean_dirs = $(sourceDirectory)
clean_extensions = fasl dfsl cfsl fasl fas lib dx32fsl
clean_extensions = fasl dfsl cfsl fasl fas lib dx32fsl lx64fsl lx32fsl
clean: FORCE
@for dir in $(clean_dirs); do \
......
===========================
Common Lisp Source Registry
===========================
This file specifies how build systems such as ASDF and XCVB
may be configured with respect to filesystem paths
where to search for Common Lisp source code.
Configurations
==============
Configurations specify paths where to find system files.
1- An application may explicitly initialize the source-registry
configuration using the `Configuration API`_ below,
in which case this takes precedence.
It may itself compute this configuration from the command-line,
from a script, from its own configuration file, etc.
2- The source registry will be configured from
the environment variable ``CL_SOURCE_REGISTRY`` if it exists.
3- The source registry will be configured from
user configuration file
``~/.config/common-lisp/source-registry.conf``
if it exists.
4- The source registry will be configured from
user configuration directory
``~/.config/common-lisp/source-registry.conf.d/``
if it exists.
5- The source registry will be configured from
system configuration file
``/etc/common-lisp/source-registry.conf``
if it exists.
6- The source registry will be configured from
system configuration directory
``/etc/common-lisp/source-registry.conf.d/``
if it exists.
7- The source registry will be configured from a default configuration,
which allows for implementation-specific software to be searched.
(See below `Backward Compatibility`_).
Each of these configuration is specified as a SEXP
in a trival domain-specific language (defined below).
Additionally, a more shell-friendly syntax is available
for the environment variable (defined yet below).
Backward Compatibility
======================
For backward compatibility, ASDF will fall back to its old ways
of searching for ``.asd`` files in the directories specified in
``asdf:*central-registry*``
if it fails to find a configuration for the source registry, or
if it fails to find a requested system in the configured source registry.
This new mechanism will therefore not affect you if you don't use it,
but will take precedence over the old mechanism if you do use it.
Moreover, when using SBCL, now as before, ASDF will first look
for a matching system in the implementation-specific ``contrib`` directory.
This allows for some magic implementation-provided systems
to be loaded specially in a version that matches your implementation.
Configuration DSL
=================
Here is the grammar of the SEXP DSL for source-registry configuration:
;; A configuration is single SEXP starting with keyword :source-registry
;; followed by a list of directives.
CONFIGURATION := (:source-registry DIRECTIVE ...)
;; A directive is one of the following:
DIRECTIVE :=
;; add a single directory to be scanned (no recursion)
(:directory DIRECTORY-PATHNAME-DESIGNATOR) |
;; add a directory hierarchy, recursing but excluding specified patterns
(:tree DIRECTORY-PATHNAME-DESIGNATOR) |
;; override the default defaults for exclusion patterns
(:exclude PATTERN ...) |
;; splice the parsed contents of another config file
(:include REGULAR-FILE-PATHNAME-DESIGNATOR) |
;; Your configuration expression MUST contain
;; exactly one of either of these:
(:inherit-configuration) | ; splices contents of inherited configuration
(:ignore-inherited-configuration) ; drop contents of inherited configuration
;; This directive specifies that some default must be spliced.
(:default-registry)
PATTERN := a string without wildcards, that will be matched exactly
against the name of a any subdirectory in the directory component
of a path. e.g. "_darcs" will match #p"/foo/bar/_darcs/src/bar.asd"
Configuration Directories
=========================
Configuration directories consist in files each contains
a list of directives without any enclosing ``(:source-registry ...)`` form.
The files will be sorted by namestring as if by #'string< and
the lists of directives of these files with be concatenated in order.
An implicit ``:inherit-configuration`` will be included
at the end of the list.
This allows for packaging software with has file granularity
(e.g. Debian's ``dpkg`` or some future version of ``clbuild``)
to easily include configuration information about distributed software.
Directories may be included by specifying a directory pathname
or namestring in an ``:include`` directive, e.g.::
(:include "/foo/bar/")
Shell-friendly syntax for configuration
=======================================
When considering environment variable ``CL_SOURCE_REGISTRY``
ASDF will skip to next configuration if it's an empty string.
It will ``READ`` the string as a SEXP in the DSL
if it begins with a paren ``(``
and it will be interpreted much like ``TEXINPUTS``
as ``:`` (colon) separated list of paths, where
* each entry is a directory to add to the search path.
* if the entry ends with a double slash ``//``
then it instead indicates a tree in the subdirectories
of which to recurse.
* if the entry is the empty string (which may only appear once),
then it indicates that the inherited configuration should be
spliced there.
Search Algorithm
================
In case that isn't clear, the semantics of the configuration is that
when searching for a system of a given name,
directives are processed in order.
When looking in a directory, if the system is found, the search succeeds,
otherwise it continues.
When looking in a tree, if one system is found, the search succeeds.
If multiple systems are found, the consequences are unspecified:
the search may succeed with any of the found systems,
or an error may be raised.
ASDF currently returns the first system found,
XCVB currently raised an error.
If none is found, the search continues.
Exclude statements specify patterns of subdirectories the systems of which
to ignore. Typically you don't want to use copies of files kept by such
version control systems as Darcs.
Include statements cause the search to recurse with the path specifications
from the file specified.
An inherit-configuration statement cause the search to recurse with the path
specifications from the next configuration
(see section Configurations_ above).
Caching Results
===============
The implementation is allowed to either eagerly compute the information
from the configurations and file system, or to lazily re-compute it
every time, or to cache any part of it as it goes.
To explicitly flush any information cached by the system, use the API below.
Configuration API
=================
The specified functions are exported from your build system's package.
Thus for ASDF the corresponding functions are in package ASDF,
and for XCVB the corresponding functions are in package XCVB.
(initialize-source-registry)
will read the configuration and initialize all internal variables.
(clear-source-registry)
undoes any source registry configuration
and clears any cache for the search algorithm.
You might want to call that before you
dump an image that would be resumed with a different configuration,
and return an empty configuration.
Note that this does not include clearing information about
systems defined in the current image, only about
where to look for systems not yet defined.
(ensure-source-registry)
checks whether a source registry has been initialized.
If not, initialize it.
This function will be called before any attempt to find a system
in the source registry.
If your application wants to override the provided defaults,
it will have to use the below function process-source-registry.
(process-source-registry X &key inherit collect)
If X is a CONS, parse it as a SEXP in the configuration DSL,
and extend or override inheritted configuration.
If X is a STRING, first parse it into a SEXP
as for the CL_SOURCE_REGISTRY
environment variable (see above) then process it.
If X is a PATHNAME, read the file as a single SEXP and process it.
The inheritted configuration is provided in keyword argument inherit,
itself a list of functions that take inherit
and collect keyword arguments
and defaulting to a list of functions
that implements the default behavior.
Future
======
If this mechanism is successful, in the future, we may declare
``asdf:*central-registry*`` obsolete and eventually remove it.
Any hook into implementation-specific search mechanisms will by then
have been integrated in the ``:default-configuration`` which everyone
should either explicitly use or implicit inherit. Some shell syntax
for it should probably be added somehow.
But we're not there yet. For now, let's see how practical this new
source-registry is.
Rejected ideas
==============
Alternatives I considered and rejected included:
1- Keep asdf:*central-registry* as the master with its current semantics,
and somehow the configuration parser expands the new configuration
language into a expanded series of directories of subdirectories to
lookup, pre-recursing through specified hierarchies. This is kludgy,
and leaves little space of future cleanups and extensions.
2- Keep asdf:*central-registry* remains the master but extend its semantics
in completely new ways, so that new kinds of entries may be implemented
as a recursive search, etc. This seems somewhat backwards.
3- Completely remove asdf:*central-registry*
and break backwards compatibility.
Hopefully this will happen in a few years after everyone migrate to
a better ASDF and/or to XCVB, but it would be very bad to do it now.
4- Replace asdf:*central-registry* by a symbol-macro with appropriate magic
when you dereference it or setf it. Only the new variable with new
semantics is handled by the new search procedure.
I've been suggested the below features, but have rejected them,
for the sake of keeping ASDF no more complex than strictly necessary.
* More syntactic sugar: synonyms for the configuration directives, such as
(:add-directory X) for (:directory X), or (:add-directory-hierarchy X)
or (:add-directory X :recurse t) for (:tree X).
* The possibility to register individual files instead of directories.
* Integrate Xach Beane's tilde expander into the parser,
or something similar that is shell-friendly or shell-compatible.
I'd rather keep ASDF minimal. But maybe this precisely keeps it
minimal by removing the need for evaluated entries that ASDF has?
i.e. uses of USER-HOMEDIR-PATHNAME and $SBCL_HOME
Hopefully, these are already superseded by the :default-registry
* Using the shell-unfriendly syntax /** instead of // to specify recursion
down a filesystem tree in the environment variable.
It isn't that Lisp friendly either.
Credits
=======
Thanks a lot to Stelian Ionescu for the initial idea.
Thanks to Rommel Martinez for the initial implementation attempt.
All bad design ideas and implementation bugs are to mine, not theirs.
But so are good design ideas and elegant implementation tricks.
-- Francois-Rene Rideau <fare@tunes.org>, Sun, 24 Jan 2010 20:54:19 -0500
This diff is collapsed.
......@@ -35,7 +35,7 @@ fi
new_version="$major.$bumped"
cp asdf.lisp asdf.bak
perl -pi -e "s/REVISION:[^\"]*\"/REVISION:$new_version\"/" asdf.lisp
perl -pi -e "s/VERSION:[^\"]*\"/VERSION:$new_version\"/" asdf.lisp
if [ ! "$?" == "0" ]; then
echo "Unable to perl replace version"
exit -3
......
cl-asdf (2:1.375-1~rc1) UNRELEASED; urgency=low
cl-asdf (2:1.501-1~rc1) UNRELEASED; urgency=low
* new upstream, back to version numbers.
-- Peter Van Eynde <pvaneynd@debian.org> Thu, 31 Dec 2009 10:13:47 +0100
-- Peter Van Eynde <pvaneynd@debian.org> Wed, 27 Jan 2010 22:10:13 +0100
cl-asdf (1:20091221-1) unstable; urgency=low
......
......@@ -5,6 +5,7 @@
# - quit with exit status 0 on getting eof
# - quit with exit status >0 if an unhandled error occurs
export CL_SOURCE_REGISTRY=$PWD
if [ x"$1" = "xhelp" ]; then
echo "$0 [lisp invocation] [scripts-regex]"
......
(load "script-support")
(load "../asdf")
;; TODO:
;; - test for conditions
;; - use with-env for CL_SOURCE_REGISTRY
;; - write clean-up procedures, if needed
(eval-when (:execute :compile-toplevel :load-toplevel)
(require :sb-posix))
(defpackage :test-source-registry
(:use #:cl #:asdf))
(in-package :test-source-registry)
(defun getenv (x)
#+sbcl
(sb-posix:getenv x))
(defun setenv (x v)
#+sbcl
(sb-posix:putenv
(concatenate 'string x "=" v))
v)
(defmacro with-gensyms ((&rest names) &body body)
`(let ,(loop for n in names collect `(,n (gensym)))
,@body))
(defmacro with-env ((&rest kv) &body body)
(let ((temps (loop :for i :upto (length kv) :collect (gensym)))
(keys (mapcar #'(lambda (x) (first x)) kv)))
(flet ((gen-getenvs ()
(mapcar #'(lambda (x y)
`(,y (getenv ,x)))
keys temps))
(gen-setenvs ()
(mapcar #'(lambda (x y) `(setenv ,y ,x))
temps keys)))
`(let ,(gen-getenvs)
(unwind-protect
(progn
,@(loop :for (k v) :in kv :collect `(setenv ,k ,v))
,@body)
(progn ,@(gen-setenvs)))))))
(defun make-temporary-directory ()
(let ((str (with-output-to-string (s)
(sb-ext:run-program "mktemp" '("-d" "-u") :search t :output s))))
(concatenate 'string
(string-trim '(#\newline) str) "/")))
(defvar *base-directory* (make-temporary-directory))
(defun home-directory (&optional (base *base-directory*))
(let ((s (namestring (user-homedir-pathname))))
(merge-pathnames (subseq s 1 (length s)) base)))
(defun system-directory (&optional (base *base-directory*))
(merge-pathnames "etc/" base))
(defun tmp-directory (&optional (base *base-directory*))
(merge-pathnames "tmp/" base))
(defun home-source-registry ()
(asdf::source-registry-under (merge-pathnames ".config/" (home-directory))))
(defun system-source-registry ()
(asdf::source-registry-under (system-directory)))
(defun merge-temp-under (path)
(namestring
(merge-pathnames
(let ((s (string-downcase
(prin1-to-string (gentemp)))))
(concatenate 'string s "/"))
path)))
(defvar *asd-temp-path-1*
(merge-temp-under
(merge-temp-under (tmp-directory))))
(defvar *system-source-registry-config*
`(:source-registry
(:inherit-configuration)
(:tree ,*asd-temp-path-1*)))
(defvar *asd-temp-path-2*
(namestring (tmp-directory)))
(defvar *home-source-registry-config*
`(:source-registry
(:inhert-configuration)
(:directory ,(namestring (tmp-directory)))))
(defun write-registry-file (path contents)
(ensure-directories-exist path)
(with-open-file (out path
:direction :output
:if-exists :supersede)
(with-standard-io-syntax
(format out "~S" contents))))
(defun write-system-source-registry-config
(&key (path (system-source-registry)) (contents *system-source-registry-config*))
(write-registry-file path contents))
(defun write-home-source-registry-config
(&key (path (home-source-registry)) (contents *home-source-registry-config*))
(write-registry-file path contents))
;; use *asd-temp-path-X*?
(defun create-asd-file (&optional (directory *default-pathname-defaults*))
(let ((*default-pathname-defaults* directory))
(with-open-file (out
;; test environment variable
(defun test-environment ()
(with-env-var (("CL_SOURCE_REGISTRY"
(format nil "~A:~A" (home-directory) (system-directory))))
...))
;; test files
(defun test-files ()
...)
;; (cl-user::quit-on-error
;; ...)
\ No newline at end of file
{include resources/header.md}
{include "resources/header.md"}
<div class="contents">
### Contributors
* crhodes - Christophe Rhodes <csr21@cantab.net>
* dan_b - Daniel Barlow
* dan_b - Daniel Barlow
* demoss - Nikodemus Siivola <nikodemus@random-state.net>
* gwking - Gary King <gwking@metabang.com>
* kevinrosenberg - Kevin Rosenberg
* kevinrosenberg - Kevin Rosenberg
* nhabedi - Edi Weitz
* pvaneynd - Peter Van Eynde
* rjain - Rahul Jain
......@@ -19,7 +19,7 @@
</div>
</div>
{include resources/footer.md}
{include "resources/footer.md"}
</div>
{include resources/header.md}
{include "resources/header.md"}
<div class="contents">
### Copyright
ASDF Copyright (C) 2001-2009 Daniel Barlow and contributors
ASDF Copyright (C) 2001-2009 Daniel Barlow and contributors
This website Copyright (C) 2001-2009 Daniel Barlow and
contributors
......@@ -34,7 +34,5 @@ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
</div>
</div>
{include resources/footer.md}
{include "resources/footer.md"}
</div>
{include resources/header.md}
{include "resources/header.md"}
## Glossary
{glossary}
[fixture]> glossary The fixture is the environment in which a testcase runs. The
fixture is the code that prepares the environment for the test and that resets
[fixture]> glossary The fixture is the environment in which a testcase runs. The
fixture is the code that prepares the environment for the test and that resets
the environment after the test. Fixtures can be shared by many test-cases.
{include resources/footer.md}
{include "resources/footer.md"}
......@@ -823,8 +823,6 @@ ASDF includes code to control where the binaries files are places. The location
## Special variables
{docs *asdf-revision*}
{docs *central-registry*}
{docs *compile-file-warnings-behaviour*}
......
{include shared-header.md}
{include "shared-header.md"}
<div class="header">
<span class="logo">
......
{include shared-links.md}
{include "shared-links.md"}
{set-property html yes}
{set-property author "the ASDF group"}
......@@ -17,19 +17,19 @@
[tests]: test-results.html
[git-browse]: http://common-lisp.net/gitweb?p=projects/asdf/asdf.git;a=summary
[git-resources]: git.html
[asdf-devel]: http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
[asdf-announce]: http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-announce
[asdf-home]: http://common-lisp.net/project/asdf/
[asdf.lisp]: http://common-lisp.net/project/asdf/asdf.lisp
[tarball]: http://common-lisp.net/project/asdf/asdf.tar.gz
[kmp-large]: http://www.nhplace.com/kent/Papers/Large-Systems.html
[hs-require]: http://www.lispworks.com/documentation/HyperSpec/Body/f_provid.htm#require
[hs-standard-output]: http://www.lispworks.com/documentation/HyperSpec/Body/v_debug_.htm#STstandard-outputST
[tinaa-project]: http://common-lisp.net/project/tinaa/
[albert-project]: http://www.cliki.net/Albert/
\ No newline at end of file
{include shared-header.md}
{include "shared-header.md"}
{set-property style-sheets
{set-property style-sheets
"user-guide.css"
"http://common-lisp.net/project/cl-containers/shared/style-200.css"}
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment