Commit a7d41ca7 authored by Didier Verna's avatar Didier Verna

Initial commit.

parents
Copyright (C) 2018 EPITA Research and Development Laboratory
This file is part of Quickref.
Permission to use, copy, modify, and distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THIS SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
# Quickref
Quickref is a global documentation project for Common Lisp libraries. It
builds a website containing reference manuals for
[Quicklisp](https://www.quicklisp.org) libraries (either those already
installed or the whole thing). The reference manuals are generated by
[Declt](https://github.com/didierverna/declt) in
[Texinfo](https://www.gnu.org/software/texinfo/) format, and then processed by
`makeinfo`.
The official Quickref [website](http://quickref.common-lisp.net) is kept
up-to-date with Quicklisp. This repository contains the whole infrastructure
needed to build your own version of the website.
## Using Quickref
In order to build your local version of the website, you can use Quickref
either through the provided Docker images, or directly from the source.
### From Docker
If you want documentation for the whole Quicklisp world, this is the simplest
way to use Quickref. The Docker image ships with a complete build environment
that aims to reproduce the one used to create the Quicklisp releases. This
means that it contains (almost) all the dependencies needed to build every
library available in Quicklisp.
Type this to run Quickref in Docker:
```
docker run --name quickref quickref/quickref
```
This will automatically pull down the image and run Quickref inside it. If you
have done this before, you may want to make sure you have the latest image by
typing this first:
```
docker pull quickref/quickref
```
Also, note that you need a running Docker daemon for this to work. The Mac app
takes care of this automatically. Otherwise, if the pull or run commands above
fail, you may need to run the deamon yourself, for example with something like
this:
```
sudo dockerd
```
When Quickref is done (this *will* take a while), you can extract the website
from the container by typing this:
```
docker cp quickref:/home/quickref/quickref .
```
The website will be located under `quickref/html/`. You will also see some
other directories containing the Texinfo files, and some logs in case there
were errors during processing (from both Declt and `makeinfo`).
### From the source
Running Quickref from the source makes sense if you only want to generate
documentation for the libraries already installed on your machine. Of course,
you can also build the whole thing, but beware that this will download the
whole Quicklisp world, and that a *lot* of external dependencies will be
required for processing. Look at the file `docker/debian-9-packages.txt` to
get an idea of what's needed.
Assuming you have made Quickref available to ASDF, type this:
```lisp
(ql:quickload :fr.epita.lrde.quickref)
(fr.epita.lrde.quickref:build :libraries :installed)
```
This will generate a website for your current Quicklisp
installation. Selecting `:all` libraries instead of only those already
installed (this is actually the default) will trigger a complete cycle, which
includes downloading and installing the *whole* Quicklisp thing. The generated
files (logs, Texinfo, and HTML) can be found in `~/quickref/`.
#### Customization
In addition to the `:libraries` option, the `BUILD` function accepts a number
of keyword arguments allowing you to tweak the behavior of Quickref. In
particular, you have control on the output destination, error behavior, and a
(still) rudimentary support for parallelism. Please refer to the docstring for
more information.
## Docker images
You can build your own Quickref Docker images by using the provided Docker
image files located in the `docker` directory. The project is split in 2
separate image files:
* `Buildenv` creates a Debian 9 image with all the dependencies needed to
buid the various Quicklisp libraries inside.
* `Quickref` builds on top of the `Buildenv` image to set up a `quickref`
user, pull the latest versions of Quickref and Declt, and finally launch
Quickref.
With this 2-steps process, it is possible to rebuild the final Quickref image
with the latest Quicklisp without rebuilding the whole environment (including
the hundreds of dependencies installed from `apt`).
### Building the images
The two Quickref Docker images are created by typing respectively (and in that
order):
```
docker build -t quickref/buildenv -f Buildenv .
docker build -t quickref/quickref -f Quickref .
```
Then, as before, run it and extract the result like this:
```
docker run --name quickref quickref/quickref
docker cp quickref:/home/quickref/quickref .
```
In order to avoid conflicting with the official images, you may prefer to use
different values for the `-t` (and possibly `--name`) options, but then, be
sure to also change the line saying `FROM quickref/buildenv` in the `Quickref`
Docker image file.
#!/usr/local/bin/sbcl --script
;;; declt --- Declt script for Quickref
;; Copyright (C) 2018 EPITA Research and Development Laboratory
;; Author: Didier Verna
;; Maintainers: Antoine Martin and Didier Verna
;; Contact: Quickref Maintainers <quickref-devel@common-lisp.net>
;; This file is part of Quickref.
;; Permission to use, copy, modify, and distribute this software for any
;; purpose with or without fee is hereby granted, provided that the above
;; copyright notice and this permission notice appear in all copies.
;; THIS SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
;; WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
;; MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
;; ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
;; WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
;; ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
;; OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
;; Comment:
;; Usage: declt SYSTEM-NAME [OPTIONS]
;; Available options:
;; --quicklisp-setup FILENAME (defaults to ~~/quicklisp/setup.lisp)
;; --texinfo-directory DIRECTORY (default to the current directory)
;; --introduction Make room for an introduction (no value)
;; #### FIXME: maybe this script would rather belong to Declt itself?
;;; Code:
(in-package :cl-user)
;; #### WARNING: some libraries use a #.'ed README file in their system
;; #### definition and assume a UTF8 encoding. It's wrong of them to assume
;; #### this is always the case, but in the meantime, we also need to make it
;; #### so, if we want those systems to be loadable.
(setq sb-impl::*default-external-format* :utf-8)
(unless (> (length sb-ext:*posix-argv*) 1)
(princ "Usage: declt SYSTEM-NAME [OPTIONS]
Available options:
--quicklisp-setup FILENAME (defaults to ~~/quicklisp/setup.lisp)
--cache-directory DIRECTORY (default to the normal caching behavior)
--texinfo-directory DIRECTORY (default to the current directory)
--introduction Make room for an introduction (no value)
"
*error-output*)
(sb-ext:exit :code 1))
;; #### NOTE: defining these as global variables is a bit ugly, but it avoids
;; #### the pain of missing load-time packages (hence the need for interning)
;; #### in an otherwise lexical closure.
(defvar *system-name* (nth 1 sb-ext:*posix-argv*))
(defvar *quicklisp-setup*
(merge-pathnames "quicklisp/setup.lisp" (user-homedir-pathname)))
(defvar *cache-directory* nil)
(defvar *texinfo-directory* #p"./")
(defvar *introduction* nil)
(loop
:for option :on (cddr sb-ext:*posix-argv*)
:do (cond ((string= (car option) "--quicklisp-setup")
(setq *quicklisp-setup*
(or (cadr option)
(error "option --quicklisp-setup requires a value.")))
(setq option (cdr option)))
((string= (car option) "--cache-directory")
(setq *cache-directory*
(or (cadr option)
(error "option --cache-directory requires a value.")))
(setq option (cdr option)))
((string= (car option) "--texinfo-directory")
(setq *texinfo-directory*
(or (cadr option)
(error "option --texinfo-directory requires a value.")))
(setq option (cdr option)))
((string= (car option) "--introduction")
(setq *introduction* t))))
(load *quicklisp-setup*)
(when *cache-directory*
(asdf:initialize-output-translations
`(:output-translations
(,(namestring (asdf:system-source-directory :asdf))
,(namestring (merge-pathnames "asdf" *cache-directory*)))
(,(namestring (asdf:system-source-directory :net.didierverna.declt))
,(namestring (merge-pathnames "declt" *cache-directory*)))
(t ,(namestring (merge-pathnames *system-name* *cache-directory*)))
:inherit-configuration)))
(asdf:load-system :net.didierverna.declt)
;; #### NOTE: Quickref doesn't brutally download the whole Quicklisp world
;; #### before starting to build manuals (it only downloads the releases
;; #### tarballs on demand, in order to figure out whther there's a README
;; #### file inside). Consequently, we may end up running Declt on a library
;; #### which is installed but still missing some dependencies. Because of
;; #### that, we need to QL:QUICKLOAD it (as opposed to just
;; #### ASDF:LOAD-SYSTEM'ing it) before running Declt. At some point, maybe
;; #### Declt could provide an option for using Quicklisp directly, instead of
;; #### just calling ASDF:LOAD-SYSTEM.
;; #### NOTE: there is another consequence of this, due to a flaw in
;; Quicklisp. QL:QUICKLOAD doesn't handle DEFSYSTEM-DEPENDS-ON missing
;; dependencies properly. We thus need to protect ourselves, for instance by
;; catching those errors, asking Quicklisp to download the missing dependency
;; and retry.
(handler-case (ql:quickload *system-name*)
(asdf:missing-dependency (c)
(ql-dist:ensure-installed
(ql-dist:release
(ql-dist:find-system-in-dist
(asdf::missing-requires c)
(ql-dist:find-dist "quicklisp"))))
;; #### NOTE: the exact restart depends on the context (ASDF, Swank, who
;; knows), so we just look for one named RETRY.
(invoke-restart (find :retry (compute-restarts c)
:key #'restart-name :test #'string=))))
(net.didierverna.declt:declt *system-name*
:introduction (when *introduction* "")
:texi-directory *texinfo-directory*)
;;; Local Variables:
;;; Mode: common-lisp
;;; End:
;; declt ends here
;;; cmark.lisp --- Entry points
;; Copyright (C) 2018 EPITA Research and Development Laboratory
;; Author: Antoine Martin
;; Maintainers: Antoine Martin and Didier Verna
;; Contact: Quickref Maintainers <quickref-devel@common-lisp.net>
;; This file is part of Quickref.
;; Permission to use, copy, modify, and distribute this software for any
;; purpose with or without fee is hereby granted, provided that the above
;; copyright notice and this permission notice appear in all copies.
;; THIS SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
;; WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
;; MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
;; ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
;; WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
;; ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
;; OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
;;; Code:
(defpackage :fr.epita.lrde.quickref.cmark
(:documentation "The Quickref/CMark package.")
(:use :cl)
(:export :cmark-markdown-to-html))
(in-package :fr.epita.lrde.quickref.cmark)
(cffi:define-foreign-library libcmark
(:darwin "libcmark.dylib")
(:unix "libcmark.so")
(t (:default "libcmark")))
(cffi:use-foreign-library libcmark)
(cffi:defcfun "cmark_markdown_to_html" :string
(md :string)
(size :unsigned-int)
(options :int))
;;; cmark.lisp ends here
;;; fr.epita.lrde.quickref.cmark.asd --- ASDF system definition
;; Copyright (C) 2018 EPITA Research and Development Laboratory
;; Author: Antoine Martin
;; Maintainers: Antoine Martin and Didier Verna
;; Contact: Quickref Maintainers <quickref-devel@common-lisp.net>
;; This file is part of Quickref.
;; Permission to use, copy, modify, and distribute this software for any
;; purpose with or without fee is hereby granted, provided that the above
;; copyright notice and this permission notice appear in all copies.
;; THIS SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
;; WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
;; MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
;; ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
;; WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
;; ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
;; OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
;;; Code:
(asdf:load-system :fr.epita.lrde.quickref.setup)
(asdf:defsystem :fr.epita.lrde.quickref.cmark
:long-name "CommonMark Foreign Function Interface"
:description "Bindings to libcmark"
:long-description "\
CMark provides bindings to libcmark, the C implementation of CommonMark.
CommonMark[1] is itself an attempt at properly specifying the MarkDown[2]
format.
[1] http://commonmark.org/
[2] https://daringfireball.net/projects/markdown/"
:author "Antoine Martin"
:maintainer ("Antoine Martin" "Didier Verna")
:mailto "quickref-devel@common-lisp.net"
:license "BSD"
:version #.(fr.epita.lrde.quickref.setup:version :short)
:depends-on (:fr.epita.lrde.quickref.setup :cffi)
:serial t
:components ((:file "cmark")))
;;; fr.epita.lrde.quickref.cmark.asd ends here
;;; fr.epita.lrde.quickref.core.asd --- ASDF system definition, core library
;; Copyright (C) 2018 EPITA Research and Development Laboratory
;; Author: Antoine Martin
;; Maintainers: Antoine Martin and Didier Verna
;; Contact: Quickref Maintainers <quickref-devel@common-lisp.net>
;; This file is part of Quickref.
;; Permission to use, copy, modify, and distribute this software for any
;; purpose with or without fee is hereby granted, provided that the above
;; copyright notice and this permission notice appear in all copies.
;; THIS SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
;; WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
;; MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
;; ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
;; WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
;; ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
;; OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
;;; Code:
(asdf:load-system :fr.epita.lrde.quickref.setup)
(asdf:defsystem :fr.epita.lrde.quickref.core
:long-name "Quickref Core"
:description "Quickref's core library"
:long-description "\
Quickref's core library provides the core functionality of Quickref. For a
more complete description of Quickref, see the fr.epita.lrde.quickref system."
:author "Antoine Martin"
:maintainer ("Antoine Martin" "Didier Verna")
:mailto "quickref-devel@common-lisp.net"
:license "BSD"
:version #.(fr.epita.lrde.quickref.setup:version :short)
:depends-on (:fr.epita.lrde.quickref.setup
:fr.epita.lrde.quickref.cmark
:quicklisp :cl-ppcre :3bmd :net.didierverna.declt)
:serial t
:components ((:file "meta")
(:module "src"
:components ((:file "util")
(:file "texinfo" :depends-on ("util"))
(:file "html" :depends-on ("util"))
(:file "quickref" :depends-on ("texinfo"
"html"))))))
;;; fr.epita.lrde.quickref.core.asd ends here
;;; meta.lisp --- Meta utilities
;; Copyright (C) 2018 EPITA Research and Development Laboratory
;; Author: Antoine Martin
;; Maintainers: Antoine Martin and Didier Verna
;; Contact: Quickref Maintainers <quickref-devel@common-lisp.net>
;; This file is part of Quickref.
;; Permission to use, copy, modify, and distribute this software for any
;; purpose with or without fee is hereby granted, provided that the above
;; copyright notice and this permission notice appear in all copies.
;; THIS SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
;; WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
;; MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
;; ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
;; WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
;; ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
;; OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
;;; Code:
(in-package :cl-user)
;; -------------------
;; Package definition:
;; -------------------
(defpackage :fr.epita.lrde.quickref
(:documentation "The Quickref package.")
(:use :cl :fr.epita.lrde.quickref.setup)
(:import-from :fr.epita.lrde.quickref.cmark
:cmark-markdown-to-html)
(:export
;; From the :fr.epita.lrde.quickref.setup package:
:*release-major-level*
:*release-minor-level*
:*release-status*
:*release-status-level*
:*release-name*
:version
;; From meta.lisp (this file):
:nickname-package
;; From src/util.lisp:
:*update*
:*libraries*
:*output-directory*
:*makeinfo-path*
:*caching-behavior*
:*error-behavior*
:*parallel*
;; From src/quickref.lisp:
:build))
(in-package :fr.epita.lrde.quickref)
;; ------------------
;; External Utilities
;; ------------------
(defun nickname-package (&optional (nickname :quickref))
"Add NICKNAME (:QUICKREF by default) to the :FR.EPITA.LRDE.QUICKREF package."
(rename-package :fr.epita.lrde.quickref
(package-name :fr.epita.lrde.quickref)
(adjoin nickname (package-nicknames :fr.epita.lrde.quickref)
:test #'string-equal)))
;;; meta.lisp ends here
This diff is collapsed.
;;; quickref.lisp --- Entry points
;; Copyright (C) 2018 EPITA Research and Development Laboratory
;; Author: Antoine Martin
;; Maintainers: Antoine Martin and Didier Verna
;; Contact: Quickref Maintainers <quickref-devel@common-lisp.net>
;; This file is part of Quickref.
;; Permission to use, copy, modify, and distribute this software for any
;; purpose with or without fee is hereby granted, provided that the above
;; copyright notice and this permission notice appear in all copies.
;; THIS SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
;; WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
;; MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
;; ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
;; WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
;; ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
;; OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
;;; Comment:
;; #### NOTE: About parallelism.
;; The current support for parallelism is very basic: a thread waits for the
;; texinfo manuals to be generated sequentially, and then runs Makeinfo on
;; them, also sequentially. There is in fact not much to be gained from that.
;; Here are some ideas.
;; 1. HTML manuals can be built in parallel without any precaution, as there
;; are no dependencies between them. If the Texinfo manuals continue to be
;; built sequentially, it's not obvious if we would gain something (it
;; depends on the respective speed of Declt vs. Makeinfo).
;; 2. For Texinfo manuals, it's much more difficult becasue Declt loads the
;; libraries, which implies compilation. When removing compilation caches
;; between builds, it's impossible to parallelize, unless we use a
;; different cache directory for every build, so we would need to customize
;; ASDF's output translations. If we want to keep compilation caches, we
;; risk compiling the same library several times in parallel as a shared
;; dependency. What we need to do then, is consider the dependency graph,
;; and start building from the leaves, going upward.
;; 3. Finally, if we ever build the Texinfo manuals in parallel, it's not
;; clear whether we want to keep threads waiting for them, or whether we
;; want to simply call Makeinfo right after Declt, within the same
;; thread. In some sense, it's just like the sequential version: currently,
;; we build all Texinfos and then all HTMLs, but we could also
;; alternate. In fact, I think it would be even better to do it like this
;; because of the dependency chain (currently, we scan the Texinfo
;; directory for files and we need to get back to release names from file
;; names etc.).
;;; Code:
(in-package :fr.epita.lrde.quickref)
(defgeneric build-manual (library)
(:documentation "Build LIBRARY's reference manual.
Return :html, :texi or NIL, indicating how far the process went.")
(:method ((system-name string))
"Build ASDF SYSTEM-NAME's reference manual.
Return :html, :texi or NIL, indicating how far the process went."
(format t "Building ~A manual...~%" system-name)
(when (build-texinfo-manual system-name)
(if (build-html-manual system-name) :html :texi)))
(:method ((system ql-dist:system))
"Build Quicklisp SYSTEM's reference manual.
Return :html, :texi or NIL, indicating how far the process went."
(build-manual (ql-dist:name system)))
(:method ((release ql-dist:release))
"Build Quicklisp RELEASE's reference manual.
Return :html, :texi or NIL, indicating how far the process went."
;; #### NOTE: this call is only here to ensure the presence of a potential
;; README file when calling the Declt script. It's needed beforehand to
;; make room in the Texinfo file for insertion as introduction (Declt
;; doesn't do this by itself yet).
(ql-dist:ensure-installed release)
(build-manual (primary-system release))))
(defun build-manuals (libraries)
"Build LIBRARIES reference manuals."
(loop :with total := (length libraries)
:and htmls := 0
:and texis := 0
:for count :from 1
:for library :in libraries
:do (case (progn (format t "(~D/~D) " count total)
(build-manual library))
(:texi (incf texis))
(:html (incf texis) (incf htmls)))
:finally (format t "~
Successfully built ~D/~D manuals (~D Declt errors, ~D Makeinfo errors).~%"
htmls total (- total texis) (- texis htmls)))
(values))
;; #### FIXME: the parallel option depends on SBCL having been compiled with
;; support for multi-threading, so the feature should be in a separate module
;; with conditional inclusion.
(defun build-manuals-// (libraries)
"Build LIBRARIES reference manuals.
This is the parallel version."
(setq *libraries-queue* nil)
(let* ((total (length libraries))
texis
htmls
(mutex (sb-thread:make-mutex))
(new-manual (sb-thread:make-waitqueue))
(thread (sb-thread:make-thread
(lambda () (build-html-manuals-// mutex new-manual)))))
(setq texis (build-texinfo-manuals-// libraries mutex new-manual))
(setq htmls (sb-thread:join-thread thread))
(format t "~
Successfully built ~D/~D manuals (~D Declt errors, ~D Makeinfo errors).~%"
htmls total (- total texis) (- texis htmls)))
(values))
(defvar *other-systems-names* '("asdf" "fr.epita.lrde.quickref")
"Non-Quicklisp systems that need to be documented.")
(defun build (&key (update *update*)
(libraries *libraries*)
(output-directory *output-directory*)
(makeinfo-path *makeinfo-path*)
(cross-reference-validation *cross-reference-validation*)
(error-behavior *error-behavior*)
(caching-behavior *caching-behavior*)
(parallel *parallel*))
"Build the Quickref website.
All keywords to this function are defaulted to their corresponding global
parameter, which see."
(let ((*output-directory* output-directory)
(*makeinfo-path* makeinfo-path)
(*cross-reference-validation* cross-reference-validation)
(*error-behavior* error-behavior)
(*caching-behavior* caching-behavior)
(*parallel* parallel))
(when update
(ql:update-client :prompt nil)
(ql:update-dist (quicklisp-distribution) :prompt nil))
(ensure-quickref-directory :declt t :makeinfo t)
(setq libraries (append
*other-systems-names*
(if (eq libraries :all)
(ql-dist:provided-releases (quicklisp-distribution))
(ql-dist:installed-releases (quicklisp-distribution)))))
(format t "~D manuals to build.~%" (length libraries))
(funcall (if parallel #'build-manuals-// #'build-manuals) libraries)
(build-index-file)
(copy-style-sheets))
(values))
;; quickref.lisp ends here
;;; texinfo.lisp --- Texinfo generation part
;; Copyright (C) 2018 EPITA Research and Development Laboratory
;; Author: Didier Verna
;; Maintainers: Antoine Martin and Didier Verna
;; Contact: Quickref Maintainers <quickref-devel@common-lisp.net>
;; This file is part of Quickref.
;; Permission is hereby granted, free of charge, to any person obtaining a
;; copy of this software and associated documentation files (the "Software"),
;; to deal in the Software without restriction, including without limitation
;; the rights to use, copy, modify, merge, publish, distribute, sublicense,
;; and/or sell copies of the Software, and to permit persons to whom the
;; Software is furnished to do so, subject to the following conditions:
;; The above copyright notice and this permission notice shall be included in
;; all copies or substantial portions of the Software.
;; THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
;; IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
;; FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
;; AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
;; LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
;; FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
;; DEALINGS IN THE SOFTWARE.
;;; Code:
(in-package :fr.epita.lrde.quickref)
(defvar *declt-script* (merge-pathnames #p"declt" *bin-directory*)
"Quickref's Declt script.")
(defun declt (system-name)
"Run Declt on ASDF SYSTEM-NAME. Return T on success."
(let ((log-errors (eq *error-behavior* :log))
(readme-file (readme-file system-name)))
(multiple-value-bind (output error-output status)
(uiop:run-program
`(,sb-ext:*runtime-pathname*
"--script" ,(namestring *declt-script*)
,system-name
"--quicklisp-setup" ,(namestring (ql-setup:qmerge "setup.lisp"))
,@(unless (eq *caching-behavior* :global)
`("--cache-directory" ,(namestring (cache-directory))))
"--texinfo-directory" ,(namestring (texinfo-directory))
,@(when readme-file '("--introduction")))
:output (if log-errors nil *standard-output*)
:error-output (if log-errors :string *error-output*)
:ignore-error-status log-errors)
(declare (ignore output))
(unless (zerop status)
(with-open-file (log (declt-log-file system-name)
:direction :output
:if-exists :supersede
:if-does-not-exist :create)
(format log "~A~%" error-output)))
(zerop status))))
(defgeneric build-texinfo-manual (library &key silent)
(:documentation "Build LIBRARY's Texinfo manual. Return T on success.")
(:method ((system-name string) &key silent)
"Build ASDF SYSTEM-NAME's Texinfo manual. Return T on success."
(let ((log-errors (eq *error-behavior* :log)))
(unless silent (format t " Running Declt~:[~%~;... ~]" log-errors))
(cond ((declt system-name)
(when (and log-errors (not silent)) (format t "ok~%"))
t)
(t
;; #### NOTE: if we reach that point (that is, without an error
;; signaled, it means that we're logging errors.
(unless silent (format t "ko~%"))
nil))))
(:method ((system ql-dist:system) &key silent)
"Build Quicklisp SYSTEM's Texinfo manual. Return T on success."
(build-texinfo-manual (ql-dist:name system) :silent silent))
(:method ((release ql-dist:release) &key silent)
"Build Quicklisp RELEASE's Texinfo manual. Return T on success."
;; #### NOTE: this call is only here to ensure the presence of a potential
;; README file when calling the Declt script. It's needed beforehand to
;; make room in the Texinfo file for insertion as introduction (Declt
;; doesn't do this by itself yet).
(ql-dist:ensure-installed release)
(build-texinfo-manual (primary-system release) :silent silent)))
#+()(defun build-texinfo-manuals (libraries)
"Build LIBRARIES Texinfo manuals."
(format t "~A Texinfo manuals to generate.~%" (length libraries))
(mapc #'build-texinfo-manual libraries)
(values))
(defvar *libraries-queue* nil)
(defun build-texinfo-manuals-// (libraries mutex new-manual)
"Build LIBRARIES Texinfo manuals, parallel version.
Return the number of Texinfo files successfully built."
(loop :with total := (length libraries)
:and texis := 0
:for count :from 1
:for library :in libraries
:do (cond ((build-texinfo-manual library :silent t)
(format-// t "(~D/~D) ~A.texi... ok.~%"