Musing: Is CLPM the Right Thing for generating Lisp library package descriptions for the Nix package manager?
I think that it is. Nix wants a data structure describing a complete Lisp package ecosystem: the packages, their versions, the exact source locations, the Lisp systems that each package provides, and the dependency graph between the systems. CLPM seems to be capable of producing exactly this, if I understand correctly?
The process of creating or updating a Lisp library ecosystem for Nix would then be:
- Run CLPM to generate a complete snapshot of library metadata in files (e.g. S-expressions.)
- Compile this snapshot into the Nix expression format e.g. using a custom Lisp program.
- Build and install the libraries using standard Nix tooling (without involving CLPM.)
The practical steps to make this happen might be:
- Package CLPM for Nix. Ideally as a Lisp library rather than a command-line program.
- Write Lisp client code to make CLPM produce the metadata snapshot.
- Write Lisp code to translate the database into Nix expressions.
Build the packages using the existing
development/lisp-modulestooling in Nix.
EDIT: Let me add some snippets and links to the concrete implementation in the upstream Nix package repository.
- lisp-packages.nix contains hand-written descriptions of certain Lisp packages e.g. that are not included in Quicklisp.
- quicklisp-to-nix-output/ contains automatically generated package descriptions, each one an individual file specifying the package name, version, source location, and systems provided. These are the "nodes" of the package graph.
- quicklisp-to-nix.nix contains automatically generated dependency information between packages. These are the "edges" of the package graph.
- quicklisp-to-nix-overrides.nix contains manual tweaks to be applied to the automatically generated packages e.g. patches required to make them work in the Nix environment and information about dependencies on external programs and shared libraries (which typically have already been manually packaged for Nix by someone else.) This mechanism is also available to end-users e.g. if their project depends on patches being applied to a dependency.
- quicklisp-to-nix is the Lisp application that grovels Quicklisp to create the auto-generated files mentioned above.
So what's the role for CLPM in all of this? The potential I see is a replacement for
quicklisp-to-nix built on the CLPM API. This could potentially be more flexible (able to address curated CLPM package collections), and more efficient (running the groveller seems to take about one hour for me), and more maintainable (reuse the groveller from CLPM instead of maintaining a separate one.) This seems worth considering to me.
I should also note that there's a similar project called ql2nix that is an out-of-tree solution. I have been using that for the past year or so and it works well too, only takes a couple of minutes to bootstrap from Quicklisp for me. However since people seem to be actively working on the Lisp package collection in upstream nixpkgs that seems worth engaging with now.