README.org 12.7 KB
Newer Older
1
#+TITLE: Common Lisp Package Manager - CLPM
2
#+AUTHOR: Eric Timmons
Eric Timmons's avatar
Eric Timmons committed
3
#+EMAIL: clpm-devel@common-lisp.net
4
#+OPTIONS: email:t toc:1 num:nil
5

Eric Timmons's avatar
Eric Timmons committed
6
7

*WARNING*: This software is BETA quality. I use it as my daily driver, but it
Eric Timmons's avatar
Eric Timmons committed
8
9
is still a little rough around the edges and it may accidentally eat your
files.
10
11
12

* Description

Eric Timmons's avatar
Eric Timmons committed
13
14
  CLPM is a package manager for Common Lisp. It can manage packages in both
  global and project specific contexts.
15

Eric Timmons's avatar
Eric Timmons committed
16
  It consists of two major pieces. First is a standalone program that is
Eric Timmons's avatar
Eric Timmons committed
17
18
19
20
21
22
23
24
25
26
  responsible for all the heavy lifting of downloading and unpacking releases
  in the correct place, resolving dependencies, and managing project specific
  environments. This piece is generally referred to as CLPM, the CLPM core, or
  =clpm= and is distributed as a precompiled executable using SBCL (but it is
  possible to compile it from source). The second is a small client library
  written in portable Common Lisp that interfaces with ASDF and calls CLPM as
  necessary to find and install missing systems. This piece is generally
  referred to as the CLPM client or =clpm-client= and is meant to be loaded
  when you are interactively developing code.

Eric Timmons's avatar
Eric Timmons committed
27
28
29
30
31
32
33
34
35
36
  CLPM is licensed under the two clause BSD license (BSD-2). So, basically do
  what you want with it (while providing appropriate attribution) and don't
  complain if it breaks things. CLPM is currently beta quality so expect some
  breakage and incompatibilites going forward.

  To receive help or report issues please send email to
  [[mailto:clpm-devel@common-lisp.net][clpm-devel@common-lisp.net]]. Additionally, feel free to join the email list at
  [[https://mailman.common-lisp.net/listinfo/clpm-devel]]. To join in the
  development of CLPM, you can find the project hosted on the common-lisp.net
  Gitlab server at [[https://gitlab.common-lisp.net/clpm/clpm]].
Eric Timmons's avatar
Eric Timmons committed
37

Eric Timmons's avatar
Eric Timmons committed
38
* Installing
Eric Timmons's avatar
Eric Timmons committed
39

Eric Timmons's avatar
Eric Timmons committed
40
41
42
  CLPM is distributed in both source and binary form. For either version, first
  install the dependencies:

43
44
45
46
  * A Lisp implementation :: SBCL is currently required if you are installing
    from source. SBCL or CCL are the most tested implementations for groveling
    dependencies from .asd files (necessary if you are installing a development
    version of a project).
Eric Timmons's avatar
Eric Timmons committed
47
  * git :: If you want to use development versions of your dependencies.
48
  * tar :: Required.
Eric Timmons's avatar
Eric Timmons committed
49
50
51

** Prebuilt binaries
   To install CLPM in binary form, download the appropriate file from
52
53
   [[https://files.clpm.dev/clpm/][https://files.clpm.dev/clpm/]]. The current release files for each platform
   are:
54

Eric Timmons's avatar
Eric Timmons committed
55
56
   + [[https://files.clpm.dev/clpm/clpm-amd64-darwin-v0.3.0-beta.6.tar.gz][clpm-amd64-darwin-v0.3.0-beta.6.tar.gz]] :: CLPM compiled for 64bit Mac OS.
   + [[https://files.clpm.dev/clpm/clpm-amd64-linux-gnu-v0.3.0-beta.6.tar.gz][clpm-amd64-linux-gnu-v0.3.0-beta.6.tar.gz]] :: CLPM compiled for 64bit Linux
Eric Timmons's avatar
Eric Timmons committed
57
     using GNU libc (by far the most common).
Eric Timmons's avatar
Eric Timmons committed
58
   + [[https://files.clpm.dev/clpm/clpm-amd64-linux-musl-v0.3.0-beta.6.tar.gz][clpm-amd64-linux-musl-v0.3.0-beta.6.tar.gz]] :: CLPM compiled for 64bit
Eric Timmons's avatar
Eric Timmons committed
59
     Linux using musl libc (e.g., Alpine Linux).
Eric Timmons's avatar
Eric Timmons committed
60
   + [[https://files.clpm.dev/clpm/clpm-arm64-linux-gnu-v0.3.0-beta.6.tar.gz][clpm-arm64-linux-gnu-v0.3.0-beta.6.tar.gz]] :: CLPM compiled for 64bit Linux
Eric Timmons's avatar
Eric Timmons committed
61
     on ARM processors using GNU libc.
Eric Timmons's avatar
Eric Timmons committed
62
   + [[https://files.clpm.dev/clpm/clpm-arm64-linux-musl-v0.3.0-beta.6.tar.gz][clpm-arm64-linux-musl-v0.3.0-beta.6.tar.gz]] :: CLPM compiled for 64bit
Eric Timmons's avatar
Eric Timmons committed
63
     Linux on ARM processors using musl libc.
Eric Timmons's avatar
Eric Timmons committed
64
   + [[https://files.clpm.dev/clpm/clpm-armv7-linux-gnu-v0.3.0-beta.6.tar.gz][clpm-armv7-linux-gnu-v0.3.0-beta.6.tar.gz]] :: CLPM compiled for 32bit Linux
Eric Timmons's avatar
Eric Timmons committed
65
     on ARM processors using GNU libc.
Eric Timmons's avatar
Eric Timmons committed
66
   + [[https://files.clpm.dev/clpm/clpm-armv7-linux-musl-v0.3.0-beta.6.tar.gz][clpm-armv7-linux-musl-v0.3.0-beta.6.tar.gz]] :: CLPM compiled for 32bit
Eric Timmons's avatar
Eric Timmons committed
67
     Linux on ARM processors using musl libc.
Eric Timmons's avatar
Eric Timmons committed
68
69
   + [[https://files.clpm.dev/clpm/clpm-amd64-windows-v0.3.0-beta.6.msi][clpm-amd64-windows-v0.3.0-beta.6.msi]] :: CLPM installer for 64 bit Windows.
   + [[https://files.clpm.dev/clpm/clpm-v0.3.0-beta.6.DIGESTS][clpm-v0.3.0-beta.6.DIGESTS]] :: Text file containing the SHA512 sums for
Eric Timmons's avatar
Eric Timmons committed
70
     every previously mentioned file.
Eric Timmons's avatar
Eric Timmons committed
71
   + [[https://files.clpm.dev/clpm/clpm-v0.3.0-beta.6.DIGESTS.asc][clpm-v0.3.0-beta.6.DIGESTS.asc]] :: Same as =clpm-v0.3.0-beta.6.DIGESTS=,
Eric Timmons's avatar
Eric Timmons committed
72
     but signed with GPG key =0x10327DE761AB977333B1AD7629932AC49F3044CE=.
Eric Timmons's avatar
Eric Timmons committed
73

74
*** GNU/Linux and Darwin
Eric Timmons's avatar
Eric Timmons committed
75
76
77
78

    After downloading the tarball and validating the SHA512 sum, unpack it and
    run =sh install.sh=. By default, it is installed to =/usr/local/=, but that
    prefix con be changed using the =INSTALL_ROOT= environment
Eric Timmons's avatar
Eric Timmons committed
79
    variable.
Eric Timmons's avatar
Eric Timmons committed
80
81
82
83
84
85
86
87
88

*** Windows

    After downloading the installer and validating the SHA512 sum, simply run
    the installer and follow the directions.

** Source Install

   See [[file:INSTALL.org][INSTALL]] for more details on installing from source.
Eric Timmons's avatar
Eric Timmons committed
89
90
91

* Quickstart

Eric Timmons's avatar
Eric Timmons committed
92
  Now that you have CLPM installed, this section walks you through how to set
93
  it up, using the packages provided by the primary Quicklisp distribution.
Eric Timmons's avatar
Eric Timmons committed
94

95
96
97
98
  First, configure CLPM to use the primary Quicklisp distribution as a source
  for packages. Create a file called =~/.config/clpm/sources.conf= (or
  =C:\Users\$USER\AppData\Local\config\clpm\sources.conf= on Windows) with the
  following contents:
Eric Timmons's avatar
Eric Timmons committed
99
100

  #+begin_src common-lisp
Eric Timmons's avatar
Eric Timmons committed
101
    ("quicklisp"
102
103
     :type :quicklisp
     :url "https://beta.quicklisp.org/dist/quicklisp.txt")
Eric Timmons's avatar
Eric Timmons committed
104
105
  #+end_src

Eric Timmons's avatar
Eric Timmons committed
106
  See [[file:docs/sources.org][sources]] for more details on configuring sources.
Eric Timmons's avatar
Eric Timmons committed
107

108
109
110
111
112
113
114
115
116
117
  Then, configure ASDF to find the CLPM client. Assuming you haven't modified
  your [[https://common-lisp.net/project/asdf/asdf.html#Controlling-where-ASDF-searches-for-systems][ASDF source registry]] too much, place the output of the following command
  at =~/.config/common-lisp/source-registry.conf.d/20-clpm-client.conf=. If
  you've modified your source registry a lot, you probably know what to do with
  the output.

  #+begin_src shell
    clpm client source-registry.d
  #+end_src

Eric Timmons's avatar
Eric Timmons committed
118
119
  Next, configure your favorite Lisp to load the client by placing the output
  of the following command in your Lisp's init file (such as =~/.sbclrc=).
Eric Timmons's avatar
Eric Timmons committed
120
121

  #+begin_src shell
122
    clpm client rc
Eric Timmons's avatar
Eric Timmons committed
123
124
  #+end_src

125
126
  Now you can start your favorite Common Lisp implementation and enter into the
  default CLPM context.
127
128
129
130
131
132
133

  #+begin_src common-lisp
    (clpm-client:activate-context "default" :activate-asdf-integration t)
  #+end_src

  When using Quicklisp metadata directly, you need to sync in order to get all
  the metadata locally (other types of sources are able to lazily sync). This
134
  step will take a while the first time as it downloads and processes every
135
136
137
138
139
140
141
  version of the distribution.

  #+begin_src common-lisp
    (clpm-client:sync :sources "quicklisp")
  #+end_src

  Now you can try loading a system, such as alexandria:
Eric Timmons's avatar
Eric Timmons committed
142

Eric Timmons's avatar
Eric Timmons committed
143
144
  #+begin_src common-lisp
    (asdf:load-system :alexandria)
Eric Timmons's avatar
Eric Timmons committed
145
146
  #+end_src

147
148
  CLPM will see that alexandria is not present locally and ask you if you would
  like to install it automatically.
Eric Timmons's avatar
Eric Timmons committed
149

Eric Timmons's avatar
Eric Timmons committed
150
151
* Project Goals

152
  CLPM is far from the only package manager available for Common Lisp[fn:1], but
Eric Timmons's avatar
Eric Timmons committed
153
154
155
  it makes very different assumptions and design choices than the other
  available solutions. In this section we describe our high level goals and how
  they affect our design decisions.
Eric Timmons's avatar
Eric Timmons committed
156

Eric Timmons's avatar
Eric Timmons committed
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
** Use existing libraries where possible

   If you look at Quicklisp, you'll quickly notice that the client does not
   rely on any third party code and all functionality, from an HTTP client to
   untaring, are implemented completely as part of the Quicklisp client
   project. The upsides of this strategy are that only required functionality
   is loaded into the development image, it prevents version conflicts between
   code the Quicklisp client depends on and the code you're developing depends
   on, and it helps ensure that Quicklisp works on a broad set of Common Lisp
   implementations. However, it has a major downside: the maintenance and
   development costs of the package manager are high, potentially making it
   difficult to implement new features and the package manager does not really
   drive the quality of code in the community at large higher.

   The desire to use existing libraries drives the decision for the CLPM core
172
173
174
175
176
   and client to be separated. The client has no external dependencies outside
   of ASDF/UIOP, which allows it to share all the benefits of Quicklisp's
   model, and the core never needs to be loaded into a development image, so it
   can leverage any dependency that makes sense without impacting the
   development image in the slightest.
Eric Timmons's avatar
Eric Timmons committed
177
178
179
180

** Support HTTPS

   HTTPS is becoming more and more ubiquitous. Some websites (such as Github
Eric Timmons's avatar
Eric Timmons committed
181
   and Gitlab) are only served over HTTPS and some people insist on HTTPS
Eric Timmons's avatar
Eric Timmons committed
182
183
184
   everywhere possible. This trend is not going away, therefore CLPM should
   natively support HTTPS.

185
186
187
188
189
190
   As only LispWorks is the only Common Lisp implementation I am aware of that
   has native support for TLS, this means that CLPM has to use third party
   tools to achieve this support. This further drives the separation of the
   core and client, as CLPM can use foreign libraries to provide TLS support
   and this is not something that should be brought into a development image
   that does not otherwise need it.
Eric Timmons's avatar
Eric Timmons committed
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206

   Additionally, Quicklisp packages are served over HTTPS. While the Quicklisp
   client cannot take advantage of that (without
   [[https://github.com/snmsts/quicklisp-https]]), CLPM can, providing a little
   more of a guarantee that packages have not been tampered with.

** Minimize footprint in development images and deployments

   Many Common Lisp implementations allow you to deliver programs by dumping an
   image to file. For most programs generated this way there is no need to have
   a bundled package manager. Therefore, there should be an easy way to use a
   package manager without leaving traces if it in your deployed system.

   To this end, CLPM can be used without ever loading the client (for example,
   =clpm bundle exec= configures ASDF entirely through environment variables)
   and if you do choose to use the client for better interactive development,
207
   the client is able to remove itself from the image when it is dumped.
Eric Timmons's avatar
Eric Timmons committed
208
209
210
211
212
213
214
215
216
217
218
219
220
221

** Support CI/CD workflows

   CI and CD are nice to have and nicer still when the jobs can run quickly and
   give fast feedback. To that end, CLPM is distributed in both binary and
   source form. Source for hackers or people who want to use a different
   feature set and binary for quick and easy installation in other cases.

** Support installing multiple package versions

   Ideally, updates to packages would never introduce regressions. However, we
   live in reality and this happens frequently (just look at the packages that
   get removed from Quicklisp in any given release for failing to
   build). Additionally, sometimes you just really need to use an old version
Eric Timmons's avatar
Eric Timmons committed
222
   of a dependency for some project while using the latest version of the same
Eric Timmons's avatar
Eric Timmons committed
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
   dependency in another project.

   To this end, CLPM supports installing multiple versions of the same package
   simultaneously. This is additionally an enabling feature for managing
   project specific contexts as well as global contexts.

** Support and encourage explicitly versioned systems

   ASDF allows developers to provide version numbers for their systems and
   associate version requirements with dependencies. IMHO this is a criminally
   underutilized feature of ASDF and it should be *required* that any release
   of a package in any package index should declare a version number. However,
   it is a lot of work to convince others to believe the same way and even if a
   critical mass did buy in, things wouldn't change overnight.

   Therefore, CLPM supports both the status quo (a Quicklisp package index
239
   versioned by date) as well as a [[https://gitlab.common-lisp.net/clpm/clpi][work in progress package index]] (working name
240
   of CLPI). This new Common Lisp Project Index includes both extra metadata
Eric Timmons's avatar
Eric Timmons committed
241
   about projects (such as the upstream repo which can be used to check out
242
243
   development versions) as well as information on the ASDF system version
   numbers.
Eric Timmons's avatar
Eric Timmons committed
244
245
246
247
248

* In-depth Documentation

  For more documentation on CLPM, you may find the following files useful:

Eric Timmons's avatar
Eric Timmons committed
249
250
251
252
  + [[file:docs/client.org][client]] :: Summary of CLPM's client.
  + [[file:docs/config.org][config]] :: Summary of all of CLPM's configuration options.
  + [[file:docs/sources.org][sources]] :: Summary of all supported software repositories.
  + [[file:docs/bundle.org][bundle]] :: Information on how to use CLPM to manage and repeatably
Eric Timmons's avatar
Eric Timmons committed
253
    install dependencies for a single project.
Eric Timmons's avatar
Eric Timmons committed
254
  + [[file:docs/storage.org][storage]] :: Information on where CLPM writes data to your hard drive.
Eric Timmons's avatar
Eric Timmons committed
255

Eric Timmons's avatar
Eric Timmons committed
256
257
* Footnotes

258
[fn:1] See, for example: [[https://www.quicklisp.org/beta/][Quicklisp]], [[https://github.com/fukamachi/qlot/][Qlot]], and [[https://github.com/CodyReichert/qi][Qi]].