README.org 10 KB
Newer Older
1
#+TITLE: Common Lisp Package Manager - CLPM
Eric Timmons's avatar
Eric Timmons committed
2
3
#+AUTHOR: CLPM Developers
#+EMAIL: clpm-devel@common-lisp.net
4
5
#+OPTIONS: email:t toc:2 num:nil

Eric Timmons's avatar
Eric Timmons committed
6
7
**WARNING**: This software is ALPHA quality. I use it as my daily driver, but it
is still a little rough around the edges and it may accidentally eat your files.
8
9
10

* Description

Eric Timmons's avatar
Eric Timmons committed
11
  CLPM is a package manager for Common Lisp. It is focused on managing packages
Eric Timmons's avatar
Eric Timmons committed
12
  for both user contexts and project specific contexts (bundles).
13

Eric Timmons's avatar
Eric Timmons committed
14
15
16
  It consists of two major pieces. First is a standalone program that is
  responsible for all the heavy lifting of fetching and unpacking releases in
  the correct place and managing project specific environments. This piece is
17
18
19
20
21
22
23
24
25
26
  generally referred to as CLPM, the CLPM core, or =clpm= and is distributed as
  a precompiled image using SBCL (but it is possible to compile it from
  source). The second is a small client library written in portable Common Lisp
  that interfaces with ASDF and calls CLPM as necessary to find and install
  missing systems. This piece is generally referred to as the CLPM client or
  =clpm-client= and is meant to be loaded into development images.

  To contact the developers, send email to and/or join
  clpm-devel@common-lisp.net
  [[https://mailman.common-lisp.net/listinfo/clpm-devel]].
Eric Timmons's avatar
Eric Timmons committed
27

Eric Timmons's avatar
Eric Timmons committed
28
* Project Goals
29
30
31

  CLPM is far from the only package manager available for Common Lisp[fn:1], but
  it makes very different assumptions and design choices than the other
32
33
  available solutions. In this section we describe our high level goals and how
  they affect our design decisions.
Eric Timmons's avatar
Eric Timmons committed
34
35
36
37
38
39
40
41

  + Minimize footprint in development Lisp image :: Ultimately, dependency
    management should not be a large part of developing a library or
    application - you should be focused on creating your cool functionality
    instead of downloading and installing dependencies. So why should all the
    (nontrivial amounts of) code necessary for downloading libraries, resolving
    their dependencies, unpacking them, and installing them ever be present in
    your development Lisp image? This drives CLPM's separation into the core and
42
    the client. Additionally, some Lisps (I'm looking at you, SBCL) have a large
Eric Timmons's avatar
Eric Timmons committed
43
44
45
46
47
48
49
    memory footprint and this helps keep that down.
  + Use existing libraries where possible :: Because the CLPM core never needs
    to be loaded into a development image, there are no issues with version
    conflicts or incompatibilities between third party libraries used by CLPM
    and the code being developed. This means that existing libraries can be
    leveraged where it makes sense and we don't need to reinvent the wheel.
  + Support cleanup on image dump :: Many Common Lisp implementations allow you
50
    to deliver programs by dumping an in-memory image to file. For most programs
Eric Timmons's avatar
Eric Timmons committed
51
52
53
    generated there is no need to have a bundled package manager. Therefore,
    CLPM provides an option to remove the client from the image when it is
    dumped (using UIOP provided functions).
Eric Timmons's avatar
Eric Timmons committed
54
  + Support multiple workflows and configurations :: We want it to be equally
Eric Timmons's avatar
Eric Timmons committed
55
56
57
58
    easy to hack on CLPM, to use it for CI, and to use it for development by
    coders uninterested in hacking on CLPM itself. This means we strive to
    support both distribution of a (mostly) statically linked program as well as
    using CLPM directly from source.
59
  + Support installing multiple package versions :: CLPM supports installing
Eric Timmons's avatar
Eric Timmons committed
60
61
62
    multiple versions of the same package simultaneously. This is an enabling
    feature for managing project specific dependencies as well as global
    dependencies.
Eric Timmons's avatar
Eric Timmons committed
63
  + Support multiple package sources :: At first, CLPM will support only
Eric Timmons's avatar
Eric Timmons committed
64
    Quicklisp-style distributions as a package source/repository. But we
65
66
    envision adding support for new sources, preferably including one that
    stores the upstream repos for each project.
Eric Timmons's avatar
Eric Timmons committed
67

Eric Timmons's avatar
Eric Timmons committed
68
* Installing
Eric Timmons's avatar
Eric Timmons committed
69

Eric Timmons's avatar
Eric Timmons committed
70
  CLPM is distributed in both source and binary form. Additionally, the binaries
71
72
  are distributed in both dynamically and statically linked forms (note: the
  binaries are built with SBCL, which does not support statically linking against
Eric Timmons's avatar
Eric Timmons committed
73
74
75
76
77
78
79
80
81
82
83
84
  the C library[fn:3], so you need to get the one that matches your OS's libc
  (most likely the =gnu= variant)).

  For either version, first install the dependencies:

  + tar :: Required[fn:2].
  + git :: If using bundles with git repos specified.
  + sbcl or ccl :: If systems need to be groveled for dependencies (likely
    needed if you use bundles).
  + openssl :: Needed if accessing sources over https *and* compiling from
    source or using the dynamically linked version.

85
86
  After installing CLPM core, it is recommended you also install the CLPM client
  (see [[file:doc/client.org][client.org]]), but it is not needed for the quickstart.
Eric Timmons's avatar
Eric Timmons committed
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104

** Prebuilt binaries
   To install CLPM in binary form, download the appropriate file from
   [[https://common-lisp.net/project/clpm/downloads/][https://common-lisp.net/project/clpm/downloads/]]. It is highly recommended that
   you use the static variants. Each release of CLPM consists of the following
   files:

   + =clpm-$VERSION-x86_64-linux-gnu-dynamic= :: CLPM compiled for 64bit
     Linux. Dynamically links against glibc (GNU's libc) and all supporting
     libraries.
   + =clpm-$VERSION-x86_64-linux-gnu-static= :: CLPM compiled for 64bit
     Linux. Dynamically links against glibc (GNU's libc) and statically linked
     against all supporting libraries.
   + =clpm-$VERSION-x86_64-linux-musl-dynamic= :: CLPM compiled for 64bit
     Linux. Dynamically links against musl libc and all supporting libraries.
   + =clpm-$VERSION-x86_64-linux-musl-static= :: CLPM compiled for 64bit
     Linux. Dynamically links against musl libc and statically linked against
     all supporting libraries.
Eric Timmons's avatar
Eric Timmons committed
105
   + =clpm-$VERSION-x86_64-windows-dynamic.exe= :: CLPM compiled for 64bit
Eric Timmons's avatar
Eric Timmons committed
106
     Windows. Dynamically links against all supporting libraries.
Eric Timmons's avatar
Eric Timmons committed
107
   + =clpm-$VERSION-x86_64-windows-static.exe= :: CLPM compiled for 64bit
Eric Timmons's avatar
Eric Timmons committed
108
     Windows. Statically linked against all supporting libraries.
Eric Timmons's avatar
Eric Timmons committed
109
110
   + =clpm-$VERSION-x86_64-darwin-dynamic= :: CLPM compiled for 64bit
     MacOS. Dynamically links against all supporting libraries.
Eric Timmons's avatar
Eric Timmons committed
111
112
113
114
115
116
   + =clpm-$VERSION.DIGESTS= :: Text file containing the SHA512 sums for every
     previously mentioned file.
   + =clpm-$VERSION.DIGESTS.asc= :: Same as =clpm-$VERSION.DIGESTS=, but signed
     with GPG key =0x10327DE761AB977333B1AD7629932AC49F3044CE=.

   After downloading the binary and validating the SHA512 sum, make sure the
117
118
   file is executable, and save it as =clpm= (or =clpm.exe=) somewhere on your
   PATH.
Eric Timmons's avatar
Eric Timmons committed
119
120
121

** Source install
   The next easiest way to install CLPM is to install SBCL, clone the CLPM
122
   sources (including the git submodules!), and symlink the [[file:scripts/clpm-live][clpm-live]] script as
Eric Timmons's avatar
Eric Timmons committed
123
124
125
126
127
128
129
   =clpm= somewhere on your PATH. Alternatively, you can build a dynamically or
   statically linked executable by running one of the following:

   #+begin_src shell
     sbcl --script scripts/clpm-build-dynamic-sbcl
     sbcl --script scripts/clpm-build-static-sbcl
   #+end_src
Eric Timmons's avatar
Eric Timmons committed
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188

* Quickstart

  Now that you have CLPM installed, you can begin using it! First, you need to
  add a software repository. You likely want to start with the main Quicklisp
  distribution, so create a file called =~/.config/common-lisp/clpm/clpm.conf=
  with the following contents:

  #+begin_src common-lisp
    (clpm-config
     :sources
     ("quicklisp"
      (:type :quicklisp
       :url "https://beta.quicklisp.org/dist/quicklisp.txt"
       :force-https t)))
  #+end_src

  This configures CLPM to use the primary Quicklisp distribution and configures
  it to always fetch files from it over HTTPS [fn:5] [fn:6]. See [[file:doc/config.org][config.org]] for
  more details.

  Then, sync your local copy of the Quicklisp metadata by running:

  #+begin_src shell
    clpm sync
  #+end_src

  Syncing may take a while the first time. When it's finished, you can install a
  project, such as CFFI, by running:

  #+begin_src shell
    clpm install cffi
  #+end_src

  This will install all ASDF systems belonging to the CFFI project (currently
  =cffi=, =cffi-uffi-compat=, =cffi-toolchain=, =cffi-tests=, =cffi-libffi=,
  =cffi-grovel=, and =cffi-examples=) and all their dependencies.

  If you want to install only a single system and its dependencies[fn:4] you can use
  the =-s= (system) option, such as:

  #+begin_src shell
    clpm install -s cffi
  #+end_src

  You can test this was successful by starting your favorite Lisp and running:

  #+begin_src common-lisp
    (require :asdf)
    (asdf:load-system :cffi)
  #+end_src

  Note how you do not need to load any CLPM code inside your Lisp image in order
  to load CFFI after installing it. (Assuming you have ASDF 3+ installed).

* In-depth Documentation

  For more documentation on CLPM, you may find the following files useful:

Eric Timmons's avatar
Eric Timmons committed
189
  + [[file:doc/client.org][client.org]] :: Summary of CLPM's client.
Eric Timmons's avatar
Eric Timmons committed
190
191
192
193
  + [[file:doc/config.org][config.org]] :: Summary of all of CLPM's configuration options.
  + [[file:doc/sources.org][sources.org]] :: Summary of all supported software repositories.
  + [[file:doc/bundle.org][bundle.org]] :: Information on how to use CLPM to manage and repeatably
    install dependencies for a single project.
Eric Timmons's avatar
Eric Timmons committed
194
  + [[file:doc/storage.org][storage.org]] :: Information on where CLPM writes data to your hard drive.
Eric Timmons's avatar
Eric Timmons committed
195

Eric Timmons's avatar
Eric Timmons committed
196
197
* Footnotes

Eric Timmons's avatar
Eric Timmons committed
198
199
200
201
202
203
204
205
206
207
208
209
[fn:6] On Windows, CLPM will fetch over HTTPS, but it will *not* currently
validate the certificates. See [[https://gitlab.common-lisp.net/clpm/clpm/issues/1][issue#1]] for more info.

[fn:5] All files in the primary Quicklisp distribution are served over both
HTTPS and HTTP, even though the Quicklisp client cannot use HTTPS itself.

[fn:4] Really, the entire project the system belongs to will be installed, but
only the dependencies of the specified system will be installed.

[fn:3] If you know a way to statically link SBCL against libc, please let me
know!

Eric Timmons's avatar
Eric Timmons committed
210
[fn:1] See, for example: [[https://www.quicklisp.org/beta/][Quicklisp]], [[https://github.com/fukamachi/qlot/][Qlot]], and [[https://github.com/CodyReichert/qi][Qi]].
Eric Timmons's avatar
Eric Timmons committed
211

212
[fn:2] Using the archive CL library for a full Common Lisp solution is on the
Eric Timmons's avatar
Eric Timmons committed
213
roadmap, but it needs a decent amount of work.