README.md 10.1 KB
Newer Older
1 2
UIOP, the Utilities for Implementation- and OS- Portability
===========================================================
3 4 5 6 7 8 9

UIOP is the portability layer of ASDF.
It provides utilities that abstract over discrepancies between implementations,
between operating systems, and between what the standard provides and
what programmers actually need, to write portable Common Lisp programs.

It is organized by topic in many files, each of which defines its own package
10
according to its topic: e.g [pathname.lisp](pathname.lisp)
11 12
will define package `UIOP/PATHNAME` and contain utilities related to
the handling of pathname objects.
13 14 15
All exported symbols are reexported in a convenience package `UIOP`,
except for those from `UIOP/COMMON-LISP`.
We recommend package `UIOP` be used to access all the symbols.
16

17
The files that constitute UIOP are, in dependency loading order:
18

19 20 21 22 23 24 25 26 27
* [package](package.lisp):
  deals with packages and their symbols, most notably including
  `define-package`, a variant of `defpackage` capable of hot-upgrade,
  or `symbol-call` and `find-symbol*` that are also useful for use in `.asd`
  files before packages have been defined.

* [common-lisp](common-lisp.lisp):
  lets you paper over various sub-standard implementations.
  Big offenders are Corman, GCL, Genera, MCL, none of them regularly maintained.
28
  Supported without serious issues are:
29
  ABCL, Allegro, CCL, CMUCL, CLASP, CLISP, ECL, LispWorks, MKCL, SBCL, SCL, XCL.
30

31 32 33 34
* [utility](utility.lisp):
  provides macros and functions that do not involve I/O;
  it handles control-flow, (p)lists, characters, strings, functions, classes,
  conditions, "stamps" (real number or boolean for +/- infinity), etc.
35
  It also sports `uiop-debug`, a useful tool to help you debug programs.
36

37 38 39 40 41 42 43
* [version](version.lisp):
  manages ASDF-style versioning and a related `with-deprecation` facility
  to gracefully declare that users should stop using some deprecated functions.

* [os](os.lisp):
  extracts information from your environment, including an ABI identifier,
  features that distinguish Unix vs Windows,
44
  `getenv`, `hostname`, `getcwd` and `chdir`, etc.
45

46 47 48 49
* [pathname](pathname.lisp):
  overcomes the gruesome non-portability trap that are CL pathnames
  (and their lovecraftian "logical" variant), offering a vast array of functions
  and a sensible, usable abstraction to specify relative pathnames.
50 51 52 53
  It has a function `merge-pathnames*` to use instead of `merge-pathnames`, or
  even better, `subpathname` and its variant `subpathname*`; it has also plenty
  of functions for dealing with pathnames being directory vs file,
  physical vs logical, absolute vs relative, and more.
54

55 56
* [filesystem](filesystem.lisp):
  provides portable access to the filesystem, inspecting it,
57
  only using truename when desired, using native OS namestrings,
58 59
  atomic file renaming, creating or deleting directories, etc.

60 61 62 63 64 65
* [stream](stream.lisp):
  portably deals with `*stderr*` vs `*error-output*`, character encodings
  (external formats), element types, safe `read`ing and `write`ing,
  opening files, using temporary files, flushing output buffers,
  providing `format`-like designators for streams, consuming or copying streams,
  concatenating streams or files, copying files, etc.
66

67 68
* [image](image.lisp):
  portably deals with images, dumping them, restoring from them,
69
  registering hooks to run at suitable events in the image lifetime,
70 71 72
  printing backtraces, handling fatal conditions, using or avoiding debug modes,
  accessing command line arguments or quitting the process.

73 74
* [lisp-build](lisp-build.lisp):
  portably compiles Common Lisp code, handles compilation results,
75 76 77
  muffles uninteresting conditions, saves and restores deferred warnings,
  runs hooks around compilation (to e.g. control optimizations or syntax),
  identifies the pathname of the current file, combines FASLs, etc.
78

79 80 81 82 83 84 85 86 87 88 89 90
* [launch-program](launch-program.lisp):
  semi-portably launches a program as an asynchronous external subprocess.
  Available functionality may depend on the underlying implementation.

* [run-program](run-program.lisp):
  fully portably runs a program as a synchronous external subprocess,
  feed it input and capture its output.
  Most implementations also allow interactive console subprocesses.

* [configuration](configuration.lisp):
  portably locates and parses configuration files, using best practices to
  define and validate syntax, search standard paths,
91
  let users specify pathnames or pathname patterns, etc.
92

93 94
* [backward-driver](backward-driver.lisp):
  provides backward-compatibility with earlier incarnations of this library
95
  (i.e. ASDF internals that have leaked, ASDF-UTILS, or older versions of UIOP).
96

97 98
* [driver](driver.lisp):
  reexports all the above utilities in a single package `UIOP`.
99 100 101 102 103 104 105 106 107 108 109 110


Documentation
-------------

Each file starts with a package definition form that lists the exported symbols.

All the exported functions, macros and variables ought to have proper docstrings.
If not, then it's a legitimate bug that we invite you to report.

Maybe some automated tool will extract all that information and
make a webpage from it, at which point it would be nice to insert a link here.
111 112

One tool with which you can extract all the documentation is HEΛP.
113 114 115
At this time, the interface is not great: it isn't obvious at all that you can indeed
use a scrollbar on the right of the top left side panel to navigate the many packages;
once you click on the package you're interested in, you can see its defined symbols:
116

117
* <http://bimib.disco.unimib.it/people/Marco.Antoniotti/Projects/CL/HELAMBDAP/tests/asdf-uiop/docs/html/dictionary/dictionary.html>
118 119

Another automated documentation tool is quickdocs, but unhappily, at the time of this writing,
120 121
it only extracts information from the first package
(see [bug #24](https://github.com/fukamachi/quickdocs/issues/24)):
122

123
* <http://quickdocs.org/uiop/api>
124

125

126
Help wanted extracting working documentation from UIOP's docstrings.
127

128 129 130 131 132

Using UIOP
----------

UIOP is part of ASDF 3, and any modern Common Lisp implementation
133 134
will have all of UIOP available when you `(require "asdf")`.
NB: `(require :asdf)` also works on all implementations but CLISP.
135 136 137
Every implementation has sported ASDF 3 for years, and if yours only provides
ASDF 2, we recommend you install ASDF 3 on top of it,
using the facility in [tools/install-asdf.lisp](../tools/install-asdf.lisp).
138 139 140

If you need some functionality only available in a recent version of UIOP,
but cannot or will not upgrade ASDF, UIOP is also distributed separately;
141 142 143
see e.g. in Quicklisp. You may then have to load it like any other library,
by adding `"uiop"` or some versioned constraint `(:version "uiop" "3.2.0")`
in your system's `:depends-on` declaration, or at the REPL using:
144

145 146
	(asdf:load-system :uiop)

147 148 149 150 151 152
When refering to symbols in UIOP, we recommend you either have your package
`:use` the package `:uiop` or `:import-from` it, or that you shall use `uiop:`
as a prefix to the symbols. Please *DO NOT* refer to specific subpackages such as
`uiop/run-program` from the outside of UIOP, because functions may occasionally
be moved from one internal package to the other, without notification.
They have in the past and will in the future.
153 154


155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195
When to use UIOP
----------------

UIOP is the ideal tool to use when:

*   You need utilities that are always available,
    portably, with no installation needed.
*   You work in a cooperative environment, where the user is a developer
    who understands what he's doing and is trusted not to be malicious.
*   You are writing a build system, build tools, developer-facing tools.
*   You are writing bootstrap scripts, in which you cannot suppose
    that any third-party library has been installed (yet),
    much less a C compiler or any external tool.
*   You are trying to make existing Common Lisp code more robust and portable,
    or replacing developer "scripts"
    (in shell, perl, python, ruby, js, and other blub languages)
    with Common Lisp code, but without concerns about
    either end-user usability or security
    (at the very least, you, not end-users, are fully controlling pathnames,
    and filtering off or portably encoding any unusual character, etc.)

UIOP is the wrong tool when:

*   You need to have total control on syscalls,
    to use special characters in pathnames, to handle symlinks yourself,
    or otherwise to have low-level system access.
*   You work in an adversarial environment, where some users are stupid,
    uneducated or outright malicious, and cannot be trusted not to try and
    abuse the system with pathnames, symlinks, race conditions, etc.
    (or be tricked into it by attackers).
*   You are writing end-user facing tools that pass along user-provided
    pathnames, with bad usability implications if a user tries to use weird
    pathnames, or even security implications if an attackers crafts bad
    pathnames or filesystem setups.

In those latter cases, we recommend you use IOlib, or osicat,
or some similar library that isn't as portable as UIOP,
but provides fine-grained control over low-level system access.
Also, please use extreme caution.


196 197 198
Some history
------------

199 200
UIOP, formerly known as ASDF-DRIVER (the package and system nicknames are
deprecated), evolved from ASDF 2's internal utilities and portability layer.
201
It has since fully superseded functionality from the following libraries:
202
ASDF-UTILS (UIOP carries on the ASDF 2 utilities that this exported),
203
CL-FAD (UIOP completely replaces it with better design and implementation),
204
CL-LAUNCH (UIOP took its image and command-line argument handling),
205 206 207
EXTERNAL-PROGRAM, TRIVIAL-SHELL and XCVB-DRIVER
(UIOP's `run-program` and now `launch-program` evolved from XCVB-DRIVER,
from which UIOP also initially got its condition muffling),
208
SLIME's swank-loader (UIOP has better compilation and ABI identification),
209
TRIVIAL-BACKTRACE (UIOP/IMAGE has all of it and more), etc.
210 211

UIOP also captures a large subset of the functionality from TRIVIAL-FEATURES,
212
and a small subset of the functionality from ALEXANDRIA or FARE-UTILS.
213

214 215
We recommend you use UIOP instead of any of the above, where applicable,
since UIOP is more portable, more robust, more ubiquitous, better designed,
216
better documented, etc. If you see any way in which UIOP isn't superior,
217
please tell us: we're interested in improving it so it become so.