Commit ac88e9dd authored by Eric Timmons's avatar Eric Timmons
Browse files

Merge branch 'add-docs-for-novices' into 'main'

Add documentation for package management novices

See merge request !16
parents 2d810852 a7d90f14
Pipeline #5312 failed with stages
in 33 minutes and 13 seconds
......@@ -44,8 +44,10 @@ appropriate tag or see [[http://www.clpm.dev]].
* Tutorial
It is highly recommended that you page through the [[file:tutorial/tutorial.org][tutorial]]. It will walk you
through installing, configuring, and using CLPM. Abbreviated descriptions are
below and the docs folder contains more in-depth technical discussions.
through installing, configuring, and using CLPM. Below, you'll find a high-
level depiction of lisp dependency management and how CLPM aids in it. See
also the introduction to dependency management and the in-depth technical
discussions listed at the bottom of this document.
* Installing
......@@ -247,6 +249,130 @@ appropriate tag or see [[http://www.clpm.dev]].
development versions) as well as information on the ASDF system version
numbers.
* Introduction to Dependency Management for Common Lisp Novices
A project in Common Lisp (CL) is made of systems. The project and its systems
may have external dependencies. Using a dependency can be as easy as loading
another ~.lisp~ file from your filesystem. Or it can be much more complex,
especially when you're facing the realities of distributing useful software with
complex functionality across platforms and architectures. What's needed in that
case is a build tool. For CL, that tool is [[https://common-lisp.net/project/asdf/][ASDF]]. ASDF manages the tasks of
loading and building systems in SBCL. The ASDF documentation sums it up as:
#+begin_quote
...it orchestrates the compilation and dependency management, handles some of
the portability issues, dynamically finds and loads code, and offers some
portable system access library...
#+end_quote
Systems are defined in ~.asd~ files. A single project may have 1+ ~.asd~
file(s) based on how the authors see fit to organize and/or test the components
they've written.
ASDF is also very clear in its documentation that it "will not download missing
software components for you." For that, users must turn to other tools. Manually
installing dependencies at a path ASDF recognizes is always an option.
Otherwise, [[http://quicklisp.org/][Quicklisp]] is a popular system installer with an associated online
registry of metadata it uses to search and retrieve systems. For CL, it's the /de facto/
choice. That being said, it falls short on modern features - no support for
HTTPS, no support for git repos, and no versioning. That's where [[https://clpm.dev][CLPM]] comes in.
CLPM's core functionality is the same as Quicklisp - (1) it tells ASDF where to
find external systems, and (2) it takes care of installing (downloading)
external systems. Let's break down what happens in each phase at a high-level.
1. CLPM registers a source registry with ASDF by calling
~(asdf:initialize-source-registry)~ with a list of paths to all the ~.asd~
files a project requires
2. CLPM downloads all the systems to ~$CLPM_DATA_DIR~, which defaults to
~$XDG_DATA_HOME/clpm~. It may use Quicklisp's existing metadata registry or a
newer standard called [[https://gitlab.common-lisp.net/clpm/clpi][CLPI]] to find systems
Thus, when a user asks ASDF to find an external system, ASDF knows to look in
~$CLPM_DATA_DIR~ to find the ~.asd~ and ~.lisp~ files it needs.
The last piece of core functionality of CLPM worth understanding is its ability
for users to organize dependencies using contexts. When you want to use CLPM,
you must first activate a context. Contexts may be global (usually called the
~"default"~ context) or per-project. Global contexts might be useful for
exploring systems, while per-project contexts provide a technique for locking
dependency versions for reproducible builds. Each project may also declare its
dependencies to CLPM in an aptly named ~clpmfile~.
While the concept of per-project contexts may sound similar to, say, using
~bundler~ with Ruby, ~npm~'s ~node_modules~ directory for JavaScript, or
activating a virtualenv in Python, you should be aware of how the implementation
details of a context and their interaction with CL impacts their behavior. For
starters, all systems, whether global or per-project, are installed to the same
~$CLPM_DATA_DIR~. CLPM will happily keep different versions of the same project
installed for you, though. CLPM is also recursive in that it installs
dependencies of dependencies. However, it achieves recursiveness by looking at
Quicklisp's metadata or by parsing the ~.asd~ files of dependencies, not by
parsing each dependency's ~clpmfile~. Like a ~package-lock.json~ file in a
Node.js project, CLPM writes a ~clpmfile.lock~ automatically with the full tree
of all recursive dependencies. Finally, and most importantly, it is not safe to
switch contexts on the fly. The issue with switching contexts gets to the heart
of how CL is different than many other languages.
Remember that a CL image (the functions and variables loaded in a REPL) is
both longer-living and more dynamic than the runtimes for other languages. You
switch contexts /from within the runtime itself/, not through an external
process (like a shell) that invokes the runtime. Thus, if you load a system with
ASDF and then switch contexts, /that first system is still loaded/. If you try
to then load another version of the same system in your new context, you're
still in the same runtime and you may find that the old system is still there.
In the interaction between CLPM, ASDF, SLIME, and CL, both ASDF and CLPM have
the concept of a version (more on this below). However, ASDF does not have the
concept of installing multiple versions simultaneously and switching between
them given project requirements. To ASDF, asking it to load a different version
of a system is the same as asking it to load the same system twice. The new
version might clean up the old one, but it likely won't. It depends on whether
the author(s) of the system programmed clean up behavior.
So here is what you should be thinking when you fire up SLIME:
1. I need to activate a CLPM context so CLPM knows where I'm working
2. If I haven't installed my dependencies yet, let me tell CLPM to start
installing the external systems I need
3. Now that I have my dependencies on my machine, I can start asking ASDF to
load them so I can use them
4. Time to ~C-c C-c~ on some CL!
*** Dos and Don'ts of Versioning with CLPM and ASDF
As noted, both CLPM and ASDF have the concept of a system version. It's best
practice to *only specify dependency versions in* ~.asd~ *files*.
Think of a ~clpmfile~ as a /constraint/ on the environment moreso than a
/definition/ of the environment. The primary function of a ~clpmfile~ is to tell
CLPM where it can find dependencies, eg. private git repositories, Quicklisp
metadata, and CLPI.
As a rule of thumb, you should avoid including versions in a ~clpmfile~.
However, here are a few reasons why you might want to constrain the versions of
your dependencies with CLPM.
1. DO version with CLPM if you want to use an unreleased version of a system.
Say you want to use the latest commits from another repository that haven't
made it to Quicklisp. CLPM allows you to provide a directive with a git repo
and commit. Note however this should be considered a temporary measure and
reverted to a normal release (ie. pointing to Quicklisp metadata) once the
functionality you need in the system is available normally.
2. DO put bounds on versions with CLPM. If you want to bound the versions of
systems for whatever reason, use CLPM.
3. DO add development-only dependencies with CLPM. This is similar to the
concept of ~devDependencies~ in a JavaScript ~package.json~ file. If there
are systems you need to enable unit testing or build tasks that are only needed
during developing, eg. building documentation sites, CLPM is a great choice.
Here are a few reasons to avoid versioning with CLPM.
1. DO NOT add constraints to a ~clpmfile~ if you have already specified a
version in ASDF and you are happy with the version of the system that CLPM
installs.
2. DO NOT version with CLPM just because you can provide a version. Assume ASDF
is the source of truth for the versions of dependencies.
* In-depth Documentation
For more documentation on CLPM, you may find the following files useful:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment