README.org 8.51 KB
Newer Older
1
#+TITLE: Common Lisp Package Manager - CLPM
Eric Timmons's avatar
Eric Timmons committed
2
3
#+AUTHOR: CLPM Developers
#+EMAIL: clpm-devel@common-lisp.net
4
5
#+OPTIONS: email:t toc:2 num:nil

Eric Timmons's avatar
Eric Timmons committed
6
7
8
**WARNING**: This software is BETA quality. I use it as my daily driver, but it
is still a little rough around the edges and it may accidentally eat your
files.
9
10
11

* Description

Eric Timmons's avatar
Eric Timmons committed
12
13
  CLPM is a package manager for Common Lisp. It can manage packages in both
  global and project specific contexts.
14

Eric Timmons's avatar
Eric Timmons committed
15
  It consists of two major pieces. First is a standalone program that is
Eric Timmons's avatar
Eric Timmons committed
16
17
18
19
20
21
22
23
24
25
26
  responsible for all the heavy lifting of downloading and unpacking releases
  in the correct place, resolving dependencies, and managing project specific
  environments. This piece is generally referred to as CLPM, the CLPM core, or
  =clpm= and is distributed as a precompiled executable using SBCL (but it is
  possible to compile it from source). The second is a small client library
  written in portable Common Lisp that interfaces with ASDF and calls CLPM as
  necessary to find and install missing systems. This piece is generally
  referred to as the CLPM client or =clpm-client= and is meant to be loaded
  when you are interactively developing code.

  To contact the developers, send email to clpm-devel@common-lisp.net
27
  [[https://mailman.common-lisp.net/listinfo/clpm-devel]].
Eric Timmons's avatar
Eric Timmons committed
28

Eric Timmons's avatar
Eric Timmons committed
29
* Installing
Eric Timmons's avatar
Eric Timmons committed
30

Eric Timmons's avatar
Eric Timmons committed
31
  See [[file:INSTALL.org][INSTALL.org]] for detailed installation instructions.
Eric Timmons's avatar
Eric Timmons committed
32
33
34

* Quickstart

Eric Timmons's avatar
Eric Timmons committed
35
36
37
38
39
40
  Now that you have CLPM installed, this section walks you through how to set
  it up as a Quicklisp alternative.

  First, configure CLPM to use the Quicklisp index as a source for packages and
  to use HTTPS to fetch files from it [fn:1]. Create a file called
  =~/.config/clpm/sources.conf= with the following contents:
Eric Timmons's avatar
Eric Timmons committed
41
42

  #+begin_src common-lisp
Eric Timmons's avatar
Eric Timmons committed
43
44
45
    ("quicklisp"
     :type :quicklisp
     :url "https://beta.quicklisp.org/dist/quicklisp.txt")
Eric Timmons's avatar
Eric Timmons committed
46
47
  #+end_src

Eric Timmons's avatar
Eric Timmons committed
48
  See [[file:doc/sources.org][sources.org]] for more details on configuring sources.
Eric Timmons's avatar
Eric Timmons committed
49

Eric Timmons's avatar
Eric Timmons committed
50
51
  Next, configure your favorite Lisp to load the client by placing the output
  of the following command in your Lisp's init file (such as =~/.sbclrc=).
Eric Timmons's avatar
Eric Timmons committed
52
53

  #+begin_src shell
Eric Timmons's avatar
Eric Timmons committed
54
    clpm client rc --quicklisp-alternative
Eric Timmons's avatar
Eric Timmons committed
55
56
  #+end_src

Eric Timmons's avatar
Eric Timmons committed
57
  Then, sync your local copy of the Quicklisp metadata by running:
Eric Timmons's avatar
Eric Timmons committed
58
59

  #+begin_src shell
Eric Timmons's avatar
Eric Timmons committed
60
    clpm sync
Eric Timmons's avatar
Eric Timmons committed
61
62
  #+end_src

Eric Timmons's avatar
Eric Timmons committed
63
64
  Syncing may take a while the first time as it needs to fetch and process the
  metadata for every Quicklisp release.
Eric Timmons's avatar
Eric Timmons committed
65

Eric Timmons's avatar
Eric Timmons committed
66
67
  Then you can start your favorite lisp and try loading a system, such as
  alexandria:
Eric Timmons's avatar
Eric Timmons committed
68

Eric Timmons's avatar
Eric Timmons committed
69
70
  #+begin_src common-lisp
    (asdf:load-system :alexandria)
Eric Timmons's avatar
Eric Timmons committed
71
72
  #+end_src

Eric Timmons's avatar
Eric Timmons committed
73
74
  CLPM will see that alexandria is not present locally and automatically
  install it before loading it.
Eric Timmons's avatar
Eric Timmons committed
75

Eric Timmons's avatar
Eric Timmons committed
76
77
78
79
80
81
* Project Goals

  CLPM is far from the only package manager available for Common Lisp[fn:2], but
  it makes very different assumptions and design choices than the other
  available solutions. In this section we describe our high level goals and how
  they affect our design decisions.
Eric Timmons's avatar
Eric Timmons committed
82

Eric Timmons's avatar
Eric Timmons committed
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
** Use existing libraries where possible

   If you look at Quicklisp, you'll quickly notice that the client does not
   rely on any third party code and all functionality, from an HTTP client to
   untaring, are implemented completely as part of the Quicklisp client
   project. The upsides of this strategy are that only required functionality
   is loaded into the development image, it prevents version conflicts between
   code the Quicklisp client depends on and the code you're developing depends
   on, and it helps ensure that Quicklisp works on a broad set of Common Lisp
   implementations. However, it has a major downside: the maintenance and
   development costs of the package manager are high, potentially making it
   difficult to implement new features and the package manager does not really
   drive the quality of code in the community at large higher.

   The desire to use existing libraries drives the decision for the CLPM core
   and client to be separated. The client has no external dependencies which
   allows it to share all the benefits of Quicklisp's model, and the core never
   needs to be loaded into a development image, so it can leverage any
   dependency that makes sense without impacting the development image in the
   slightest.

** Support HTTPS

   HTTPS is becoming more and more ubiquitous. Some websites (such as Github
   and Gitlab) are only served over HTTPS and some users insist on HTTPS
   everywhere possible. This trend is not going away, therefore CLPM should
   natively support HTTPS.

   As no Lisp implementation I am aware of has native support for TLS, this
   means that CLPM has to use third party tools to achieve this support. This
   further drives the separation of the core and client, as CLPM can use
   foreign libraries to provide TLS support and this is not something that
   should be brought into a development image that does not need it
   otherwise.

   Additionally, Quicklisp packages are served over HTTPS. While the Quicklisp
   client cannot take advantage of that (without
   [[https://github.com/snmsts/quicklisp-https]]), CLPM can, providing a little
   more of a guarantee that packages have not been tampered with.

** Minimize footprint in development images and deployments

   Many Common Lisp implementations allow you to deliver programs by dumping an
   image to file. For most programs generated this way there is no need to have
   a bundled package manager. Therefore, there should be an easy way to use a
   package manager without leaving traces if it in your deployed system.

   To this end, CLPM can be used without ever loading the client (for example,
   =clpm bundle exec= configures ASDF entirely through environment variables)
   and if you do choose to use the client for better interactive development,
   CLPM is able to remove itself from the image when it is dumped.

** Support CI/CD workflows

   CI and CD are nice to have and nicer still when the jobs can run quickly and
   give fast feedback. To that end, CLPM is distributed in both binary and
   source form. Source for hackers or people who want to use a different
   feature set and binary for quick and easy installation in other cases.

** Support installing multiple package versions

   Ideally, updates to packages would never introduce regressions. However, we
   live in reality and this happens frequently (just look at the packages that
   get removed from Quicklisp in any given release for failing to
   build). Additionally, sometimes you just really need to use an old version
   of a dependcy for some project while using the latest version of the same
   dependency in another project.

   To this end, CLPM supports installing multiple versions of the same package
   simultaneously. This is additionally an enabling feature for managing
   project specific contexts as well as global contexts.

** Support and encourage explicitly versioned systems

   ASDF allows developers to provide version numbers for their systems and
   associate version requirements with dependencies. IMHO this is a criminally
   underutilized feature of ASDF and it should be *required* that any release
   of a package in any package index should declare a version number. However,
   it is a lot of work to convince others to believe the same way and even if a
   critical mass did buy in, things wouldn't change overnight.

   Therefore, CLPM supports both the status quo (a Quicklisp package index
   versioned by date) as well as a work in progress package index (working name
   of CLPI). This new Common Lisp Package Index includes both extra metadata
   about projects (such as the upstream repo which can be used to check out
   development versions) and requires that systems are versioned in order to be
   released.
Eric Timmons's avatar
Eric Timmons committed
170
171
172
173
174

* In-depth Documentation

  For more documentation on CLPM, you may find the following files useful:

Eric Timmons's avatar
Eric Timmons committed
175
  + [[file:doc/client.org][client.org]] :: Summary of CLPM's client.
Eric Timmons's avatar
Eric Timmons committed
176
177
178
179
  + [[file:doc/config.org][config.org]] :: Summary of all of CLPM's configuration options.
  + [[file:doc/sources.org][sources.org]] :: Summary of all supported software repositories.
  + [[file:doc/bundle.org][bundle.org]] :: Information on how to use CLPM to manage and repeatably
    install dependencies for a single project.
Eric Timmons's avatar
Eric Timmons committed
180
  + [[file:doc/storage.org][storage.org]] :: Information on where CLPM writes data to your hard drive.
Eric Timmons's avatar
Eric Timmons committed
181

Eric Timmons's avatar
Eric Timmons committed
182
183
* Footnotes

Eric Timmons's avatar
Eric Timmons committed
184
[fn:1] All files in the primary Quicklisp distribution are served over both
Eric Timmons's avatar
Eric Timmons committed
185
186
HTTPS and HTTP, even though the Quicklisp client cannot use HTTPS itself.

Eric Timmons's avatar
Eric Timmons committed
187
[fn:2] See, for example: [[https://www.quicklisp.org/beta/][Quicklisp]], [[https://github.com/fukamachi/qlot/][Qlot]], and [[https://github.com/CodyReichert/qi][Qi]].