README.org 9.82 KB
Newer Older
1
#+TITLE: Common Lisp Package Manager - CLPM
Eric Timmons's avatar
Eric Timmons committed
2
3
#+AUTHOR: CLPM Developers
#+EMAIL: clpm-devel@common-lisp.net
4
5
#+OPTIONS: email:t toc:2 num:nil

Eric Timmons's avatar
Eric Timmons committed
6
7
**WARNING**: This software is ALPHA quality. I use it as my daily driver, but it
is still a little rough around the edges and it may accidentally eat your files.
8
9
10

* Description

Eric Timmons's avatar
Eric Timmons committed
11
  CLPM is a package manager for Common Lisp. It is focused on managing packages
Eric Timmons's avatar
Eric Timmons committed
12
  for both user contexts and project specific contexts (bundles).
13

Eric Timmons's avatar
Eric Timmons committed
14
15
16
17
18
19
20
  It consists of two major pieces. First is a standalone program that is
  responsible for all the heavy lifting of fetching and unpacking releases in
  the correct place and managing project specific environments. This piece is
  generally referred to as CLPM, the CLPM core, or =clpm=. The second is a small
  client library written in portable Common Lisp that interfaces with ASDF and
  calls CLPM as necessary to find and install missing systems. This piece is
  generally referred to as the CLPM client or =clpm-client=.
Eric Timmons's avatar
Eric Timmons committed
21

Eric Timmons's avatar
Eric Timmons committed
22
* Project Goals
23
24
25

  CLPM is far from the only package manager available for Common Lisp[fn:1], but
  it makes very different assumptions and design choices than the other
Eric Timmons's avatar
Eric Timmons committed
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
  available solutions. We try to describe our high level goals and how they
  affect our design decisions in this section.

  + Minimize footprint in development Lisp image :: Ultimately, dependency
    management should not be a large part of developing a library or
    application - you should be focused on creating your cool functionality
    instead of downloading and installing dependencies. So why should all the
    (nontrivial amounts of) code necessary for downloading libraries, resolving
    their dependencies, unpacking them, and installing them ever be present in
    your development Lisp image? This drives CLPM's separation into the core and
    client. Additionally, some Lisps (I'm looking at you, SBCL) have a large
    memory footprint and this helps keep that down.
  + Use existing libraries where possible :: Because the CLPM core never needs
    to be loaded into a development image, there are no issues with version
    conflicts or incompatibilities between third party libraries used by CLPM
    and the code being developed. This means that existing libraries can be
    leveraged where it makes sense and we don't need to reinvent the wheel.
  + Support cleanup on image dump :: Many Common Lisp implementations allow you
    to deliver programs by dumping an in memory image to file. For most programs
    generated there is no need to have a bundled package manager. Therefore,
    CLPM provides an option to remove the client from the image when it is
    dumped (using UIOP provided functions).
Eric Timmons's avatar
Eric Timmons committed
48
  + Support multiple workflows and configurations :: We want it to be equally
Eric Timmons's avatar
Eric Timmons committed
49
50
51
52
    easy to hack on CLPM, to use it for CI, and to use it for development by
    coders uninterested in hacking on CLPM itself. This means we strive to
    support both distribution of a (mostly) statically linked program as well as
    using CLPM directly from source.
53
  + Support installing multiple package versions :: CLPM supports installing
Eric Timmons's avatar
Eric Timmons committed
54
55
56
    multiple versions of the same package simultaneously. This is an enabling
    feature for managing project specific dependencies as well as global
    dependencies.
Eric Timmons's avatar
Eric Timmons committed
57
  + Support multiple package sources :: At first, CLPM will support only
Eric Timmons's avatar
Eric Timmons committed
58
59
60
    Quicklisp-style distributions as a package source/repository. But we
    envision adding support for new sources, such as a new work in progress
    repository geared toward projects that maintain their own release schedule.
Eric Timmons's avatar
Eric Timmons committed
61

Eric Timmons's avatar
Eric Timmons committed
62
* Installing
Eric Timmons's avatar
Eric Timmons committed
63

Eric Timmons's avatar
Eric Timmons committed
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
  CLPM is distributed in both source and binary form. Additionally, the binaries
  are distributed in both dynamic and statically linked forms (note: the
  binaries are built with SBCL which does not support statically linking against
  the C library[fn:3], so you need to get the one that matches your OS's libc
  (most likely the =gnu= variant)).

  For either version, first install the dependencies:

  + tar :: Required[fn:2].
  + git :: If using bundles with git repos specified.
  + sbcl or ccl :: If systems need to be groveled for dependencies (likely
    needed if you use bundles).
  + openssl :: Needed if accessing sources over https *and* compiling from
    source or using the dynamically linked version.
  + sqlite :: Needed if compiling from source or using the dynamically linked
    version.

Eric Timmons's avatar
Eric Timmons committed
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
  After installing CLPM, it is recommended you also install the CLPM client (see
  [[file:doc/client.org][client.org]]), but it is not needed for the quickstart.

** Prebuilt binaries
   To install CLPM in binary form, download the appropriate file from
   [[https://common-lisp.net/project/clpm/downloads/][https://common-lisp.net/project/clpm/downloads/]]. It is highly recommended that
   you use the static variants. Each release of CLPM consists of the following
   files:

   + =clpm-$VERSION-x86_64-linux-gnu-dynamic= :: CLPM compiled for 64bit
     Linux. Dynamically links against glibc (GNU's libc) and all supporting
     libraries.
   + =clpm-$VERSION-x86_64-linux-gnu-static= :: CLPM compiled for 64bit
     Linux. Dynamically links against glibc (GNU's libc) and statically linked
     against all supporting libraries.
   + =clpm-$VERSION-x86_64-linux-musl-dynamic= :: CLPM compiled for 64bit
     Linux. Dynamically links against musl libc and all supporting libraries.
   + =clpm-$VERSION-x86_64-linux-musl-static= :: CLPM compiled for 64bit
     Linux. Dynamically links against musl libc and statically linked against
     all supporting libraries.
Eric Timmons's avatar
Eric Timmons committed
101
   + =clpm-$VERSION-x86_64-windows-dynamic.exe= :: CLPM compiled for 64bit
Eric Timmons's avatar
Eric Timmons committed
102
     Windows. Dynamically links against all supporting libraries.
Eric Timmons's avatar
Eric Timmons committed
103
   + =clpm-$VERSION-x86_64-windows-static.exe= :: CLPM compiled for 64bit
Eric Timmons's avatar
Eric Timmons committed
104
     Windows. Statically linked against all supporting libraries.
Eric Timmons's avatar
Eric Timmons committed
105
106
   + =clpm-$VERSION-x86_64-darwin-dynamic= :: CLPM compiled for 64bit
     MacOS. Dynamically links against all supporting libraries.
Eric Timmons's avatar
Eric Timmons committed
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
   + =clpm-$VERSION.DIGESTS= :: Text file containing the SHA512 sums for every
     previously mentioned file.
   + =clpm-$VERSION.DIGESTS.asc= :: Same as =clpm-$VERSION.DIGESTS=, but signed
     with GPG key =0x10327DE761AB977333B1AD7629932AC49F3044CE=.

   After downloading the binary and validating the SHA512 sum, make sure the
   file is executable and save it as =clpm= somewhere on your PATH.

** Source install
   The next easiest way to install CLPM is to install SBCL, clone the CLPM
   sources (including the git submodules!), and link the [[file:scripts/clpm-live][clpm-live]] script as
   =clpm= somewhere on your PATH. Alternatively, you can build a dynamically or
   statically linked executable by running one of the following:

   #+begin_src shell
     sbcl --script scripts/clpm-build-dynamic-sbcl
     sbcl --script scripts/clpm-build-static-sbcl
   #+end_src
Eric Timmons's avatar
Eric Timmons committed
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183

* Quickstart

  Now that you have CLPM installed, you can begin using it! First, you need to
  add a software repository. You likely want to start with the main Quicklisp
  distribution, so create a file called =~/.config/common-lisp/clpm/clpm.conf=
  with the following contents:

  #+begin_src common-lisp
    (clpm-config
     :sources
     ("quicklisp"
      (:type :quicklisp
       :url "https://beta.quicklisp.org/dist/quicklisp.txt"
       :force-https t)))
  #+end_src

  This configures CLPM to use the primary Quicklisp distribution and configures
  it to always fetch files from it over HTTPS [fn:5] [fn:6]. See [[file:doc/config.org][config.org]] for
  more details.

  Then, sync your local copy of the Quicklisp metadata by running:

  #+begin_src shell
    clpm sync
  #+end_src

  Syncing may take a while the first time. When it's finished, you can install a
  project, such as CFFI, by running:

  #+begin_src shell
    clpm install cffi
  #+end_src

  This will install all ASDF systems belonging to the CFFI project (currently
  =cffi=, =cffi-uffi-compat=, =cffi-toolchain=, =cffi-tests=, =cffi-libffi=,
  =cffi-grovel=, and =cffi-examples=) and all their dependencies.

  If you want to install only a single system and its dependencies[fn:4] you can use
  the =-s= (system) option, such as:

  #+begin_src shell
    clpm install -s cffi
  #+end_src

  You can test this was successful by starting your favorite Lisp and running:

  #+begin_src common-lisp
    (require :asdf)
    (asdf:load-system :cffi)
  #+end_src

  Note how you do not need to load any CLPM code inside your Lisp image in order
  to load CFFI after installing it. (Assuming you have ASDF 3+ installed).

* In-depth Documentation

  For more documentation on CLPM, you may find the following files useful:

Eric Timmons's avatar
Eric Timmons committed
184
  + [[file:doc/client.org][client.org]] :: Summary of CLPM's client.
Eric Timmons's avatar
Eric Timmons committed
185
186
187
188
  + [[file:doc/config.org][config.org]] :: Summary of all of CLPM's configuration options.
  + [[file:doc/sources.org][sources.org]] :: Summary of all supported software repositories.
  + [[file:doc/bundle.org][bundle.org]] :: Information on how to use CLPM to manage and repeatably
    install dependencies for a single project.
Eric Timmons's avatar
Eric Timmons committed
189
  + [[file:doc/storage.org][storage.org]] :: Information on where CLPM writes data to your hard drive.
Eric Timmons's avatar
Eric Timmons committed
190

Eric Timmons's avatar
Eric Timmons committed
191
192
* Footnotes

Eric Timmons's avatar
Eric Timmons committed
193
194
195
196
197
198
199
200
201
202
203
204
[fn:6] On Windows, CLPM will fetch over HTTPS, but it will *not* currently
validate the certificates. See [[https://gitlab.common-lisp.net/clpm/clpm/issues/1][issue#1]] for more info.

[fn:5] All files in the primary Quicklisp distribution are served over both
HTTPS and HTTP, even though the Quicklisp client cannot use HTTPS itself.

[fn:4] Really, the entire project the system belongs to will be installed, but
only the dependencies of the specified system will be installed.

[fn:3] If you know a way to statically link SBCL against libc, please let me
know!

Eric Timmons's avatar
Eric Timmons committed
205
[fn:1] See, for example: [[https://www.quicklisp.org/beta/][Quicklisp]], [[https://github.com/fukamachi/qlot/][Qlot]], and [[https://github.com/CodyReichert/qi][Qi]].
Eric Timmons's avatar
Eric Timmons committed
206
207
208

[fn:2] Using the archive library for a full Common Lisp solution is on the
roadmap, but it needs a decent amount of work.