README.md 10.5 KB
Newer Older
1 2
UIOP, the Utilities for Implementation- and OS- Portability
===========================================================
3 4 5 6 7 8 9

UIOP is the portability layer of ASDF.
It provides utilities that abstract over discrepancies between implementations,
between operating systems, and between what the standard provides and
what programmers actually need, to write portable Common Lisp programs.

It is organized by topic in many files, each of which defines its own package
10
according to its topic: e.g [pathname.lisp](pathname.lisp)
11 12
will define package `UIOP/PATHNAME` and contain utilities related to
the handling of pathname objects.
13 14 15
All exported symbols are reexported in a convenience package `UIOP`,
except for those from `UIOP/COMMON-LISP`.
We recommend package `UIOP` be used to access all the symbols.
16

17
The files that constitute UIOP are, in dependency loading order:
18

19 20 21 22 23 24 25 26 27
* [package](package.lisp):
  deals with packages and their symbols, most notably including
  `define-package`, a variant of `defpackage` capable of hot-upgrade,
  or `symbol-call` and `find-symbol*` that are also useful for use in `.asd`
  files before packages have been defined.

* [common-lisp](common-lisp.lisp):
  lets you paper over various sub-standard implementations.
  Big offenders are Corman, GCL, Genera, MCL, none of them regularly maintained.
28
  Supported without serious issues are:
29
  ABCL, Allegro, CCL, CMUCL, CLASP, CLISP, ECL, LispWorks, MKCL, SBCL, SCL, XCL.
30

31 32 33 34
* [utility](utility.lisp):
  provides macros and functions that do not involve I/O;
  it handles control-flow, (p)lists, characters, strings, functions, classes,
  conditions, "stamps" (real number or boolean for +/- infinity), etc.
35
  It also sports `uiop-debug`, a useful tool to help you debug programs.
36

37 38 39 40 41 42 43
* [version](version.lisp):
  manages ASDF-style versioning and a related `with-deprecation` facility
  to gracefully declare that users should stop using some deprecated functions.

* [os](os.lisp):
  extracts information from your environment, including an ABI identifier,
  features that distinguish Unix vs Windows,
44
  `getenv`, `hostname`, `getcwd` and `chdir`, etc.
45

46 47 48 49
* [pathname](pathname.lisp):
  overcomes the gruesome non-portability trap that are CL pathnames
  (and their lovecraftian "logical" variant), offering a vast array of functions
  and a sensible, usable abstraction to specify relative pathnames.
50 51 52 53
  It has a function `merge-pathnames*` to use instead of `merge-pathnames`, or
  even better, `subpathname` and its variant `subpathname*`; it has also plenty
  of functions for dealing with pathnames being directory vs file,
  physical vs logical, absolute vs relative, and more.
54

55 56
* [filesystem](filesystem.lisp):
  provides portable access to the filesystem, inspecting it,
57
  only using truename when desired, using native OS namestrings,
58 59
  atomic file renaming, creating or deleting directories, etc.

60 61 62 63 64 65
* [stream](stream.lisp):
  portably deals with `*stderr*` vs `*error-output*`, character encodings
  (external formats), element types, safe `read`ing and `write`ing,
  opening files, using temporary files, flushing output buffers,
  providing `format`-like designators for streams, consuming or copying streams,
  concatenating streams or files, copying files, etc.
66

67 68
* [image](image.lisp):
  portably deals with images, dumping them, restoring from them,
69
  registering hooks to run at suitable events in the image lifetime,
70 71 72
  printing backtraces, handling fatal conditions, using or avoiding debug modes,
  accessing command line arguments or quitting the process.

73 74
* [lisp-build](lisp-build.lisp):
  portably compiles Common Lisp code, handles compilation results,
75 76 77
  muffles uninteresting conditions, saves and restores deferred warnings,
  runs hooks around compilation (to e.g. control optimizations or syntax),
  identifies the pathname of the current file, combines FASLs, etc.
78

79 80 81 82 83 84 85 86 87 88 89 90
* [launch-program](launch-program.lisp):
  semi-portably launches a program as an asynchronous external subprocess.
  Available functionality may depend on the underlying implementation.

* [run-program](run-program.lisp):
  fully portably runs a program as a synchronous external subprocess,
  feed it input and capture its output.
  Most implementations also allow interactive console subprocesses.

* [configuration](configuration.lisp):
  portably locates and parses configuration files, using best practices to
  define and validate syntax, search standard paths,
91
  let users specify pathnames or pathname patterns, etc.
92

93 94
* [backward-driver](backward-driver.lisp):
  provides backward-compatibility with earlier incarnations of this library
95
  (i.e. ASDF internals that have leaked, ASDF-UTILS, or older versions of UIOP).
96

97 98
* [driver](driver.lisp):
  reexports all the above utilities in a single package `UIOP`.
99 100 101 102 103 104 105 106 107 108


Documentation
-------------

Each file starts with a package definition form that lists the exported symbols.

All the exported functions, macros and variables ought to have proper docstrings.
If not, then it's a legitimate bug that we invite you to report.

109 110 111 112
You can extract a manual from the docstrings
by running `make` in the directory `uiop/doc`.

Other automated tools may hopefully extract all that information and
113
make a webpage from it, at which point it would be nice to insert a link here.
114
But many tools fail to extract useful data.
115

116 117 118 119 120 121 122 123 124 125 126 127
Tools with which you can extract all the documentation include
[Declt](https://www.lrde.epita.fr/~didier/software/lisp/misc.php#declt)
and [HEΛP](http://helambdap.sourceforge.net/).
See the Quickref UIOP reference manual <https://quickref.common-lisp.net/uiop.html>
as extracted by Declt.

There is also a pre-extracted HEΛP documentation page
<http://bimib.disco.unimib.it/people/Marco.Antoniotti/Projects/CL/HELAMBDAP/tests/asdf-uiop/docs/html/dictionary/dictionary.html>.
Note however that the HEΛP interface is not very usable at this time:
it isn't obvious at all that you can indeed use a scrollbar
on the right of the top left side panel to navigate the many packages;
once you click on the package you're interested in, you can see its defined symbols.
128 129

Another automated documentation tool is quickdocs, but unhappily, at the time of this writing,
130 131
it only extracts information from the first package
(see [bug #24](https://github.com/fukamachi/quickdocs/issues/24)):
132
<http://quickdocs.org/uiop/api>
133

134 135 136 137 138

Using UIOP
----------

UIOP is part of ASDF 3, and any modern Common Lisp implementation
139 140
will have all of UIOP available when you `(require "asdf")`.
NB: `(require :asdf)` also works on all implementations but CLISP.
141 142 143
Every implementation has sported ASDF 3 for years, and if yours only provides
ASDF 2, we recommend you install ASDF 3 on top of it,
using the facility in [tools/install-asdf.lisp](../tools/install-asdf.lisp).
144 145 146

If you need some functionality only available in a recent version of UIOP,
but cannot or will not upgrade ASDF, UIOP is also distributed separately;
147 148 149
see e.g. in Quicklisp. You may then have to load it like any other library,
by adding `"uiop"` or some versioned constraint `(:version "uiop" "3.2.0")`
in your system's `:depends-on` declaration, or at the REPL using:
150

151 152
	(asdf:load-system :uiop)

153 154 155 156 157 158
When refering to symbols in UIOP, we recommend you either have your package
`:use` the package `:uiop` or `:import-from` it, or that you shall use `uiop:`
as a prefix to the symbols. Please *DO NOT* refer to specific subpackages such as
`uiop/run-program` from the outside of UIOP, because functions may occasionally
be moved from one internal package to the other, without notification.
They have in the past and will in the future.
159 160


161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201
When to use UIOP
----------------

UIOP is the ideal tool to use when:

*   You need utilities that are always available,
    portably, with no installation needed.
*   You work in a cooperative environment, where the user is a developer
    who understands what he's doing and is trusted not to be malicious.
*   You are writing a build system, build tools, developer-facing tools.
*   You are writing bootstrap scripts, in which you cannot suppose
    that any third-party library has been installed (yet),
    much less a C compiler or any external tool.
*   You are trying to make existing Common Lisp code more robust and portable,
    or replacing developer "scripts"
    (in shell, perl, python, ruby, js, and other blub languages)
    with Common Lisp code, but without concerns about
    either end-user usability or security
    (at the very least, you, not end-users, are fully controlling pathnames,
    and filtering off or portably encoding any unusual character, etc.)

UIOP is the wrong tool when:

*   You need to have total control on syscalls,
    to use special characters in pathnames, to handle symlinks yourself,
    or otherwise to have low-level system access.
*   You work in an adversarial environment, where some users are stupid,
    uneducated or outright malicious, and cannot be trusted not to try and
    abuse the system with pathnames, symlinks, race conditions, etc.
    (or be tricked into it by attackers).
*   You are writing end-user facing tools that pass along user-provided
    pathnames, with bad usability implications if a user tries to use weird
    pathnames, or even security implications if an attackers crafts bad
    pathnames or filesystem setups.

In those latter cases, we recommend you use IOlib, or osicat,
or some similar library that isn't as portable as UIOP,
but provides fine-grained control over low-level system access.
Also, please use extreme caution.


202 203 204
Some history
------------

205 206
UIOP, formerly known as ASDF-DRIVER (the package and system nicknames are
deprecated), evolved from ASDF 2's internal utilities and portability layer.
207
It has since fully superseded functionality from the following libraries:
208
ASDF-UTILS (UIOP carries on the ASDF 2 utilities that this exported),
209
CL-FAD (UIOP completely replaces it with better design and implementation),
210
CL-LAUNCH (UIOP took its image and command-line argument handling),
211 212 213
EXTERNAL-PROGRAM, TRIVIAL-SHELL and XCVB-DRIVER
(UIOP's `run-program` and now `launch-program` evolved from XCVB-DRIVER,
from which UIOP also initially got its condition muffling),
214
SLIME's swank-loader (UIOP has better compilation and ABI identification),
215
TRIVIAL-BACKTRACE (UIOP/IMAGE has all of it and more), etc.
216 217

UIOP also captures a large subset of the functionality from TRIVIAL-FEATURES,
218
and a small subset of the functionality from ALEXANDRIA or FARE-UTILS.
219

220 221
We recommend you use UIOP instead of any of the above, where applicable,
since UIOP is more portable, more robust, more ubiquitous, better designed,
222
better documented, etc. If you see any way in which UIOP isn't superior,
223
please tell us: we're interested in improving it so it become so.