tutorial.org 34.6 KB
Newer Older
Eric Timmons's avatar
Eric Timmons committed
1
2
3
4
5
6
7
8
#+TITLE: CLPM Tutorial
#+AUTHOR: Eric Timmons
#+EMAIL: clpm-devel@common-lisp.net
#+REVEAL_HLEVEL: 1
#+REVEAL_THEME: black
#+REVEAL_TRANS: slide
#+REVEAL_EXTRA_CSS: css/custom.css
#+REVEAL_PLUGINS: (highlight notes)
Eric Timmons's avatar
Eric Timmons committed
9
#+REVEAL_ROOT: https://cdnjs.cloudflare.com/ajax/libs/reveal.js/3.9.2/
Eric Timmons's avatar
Eric Timmons committed
10
#+REVEAL_REVEAL_JS_VERSION: 3
Eric Timmons's avatar
Eric Timmons committed
11
12
13
14
15
16
17
#+OPTIONS: reveal_slide_number:c reveal_width:1920 reveal_height:1080
#+OPTIONS: email:t toc:1 num:nil date:nil timestamp:nil

* Intro to CLPM

** What is it?

18
   CLPM stands for Common Lisp Project Manager.
Eric Timmons's avatar
Eric Timmons committed
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

   #+ATTR_REVEAL: :frag t
   CLPM has several primary goals:

   #+ATTR_REVEAL: :frag t
   1. Manage packages in both global and project specific contexts.
   2. Use existing libraries where possible.
   3. Support HTTPS.
   4. Do as little work as possible in the development image.

** Terminology

   Below are some common terms used by CLPM and in this tutorial:

   + Package :: This is typically used in the colloquial sense, meaning some
     unit of software that is installed. There is an unfortunate collision with
     [[http://www.lispworks.com/documentation/HyperSpec/Body/11_a.htm][Common Lisp's packages]], but hopefully what is meant is clear based on
     context.
   + System :: An [[https://common-lisp.net/project/asdf/][ASDF]] defined system.
   + Project :: Primary development unit of code. Typically corresponds to the
     contents of a single VCS repository. May contain multiple systems.

   #+REVEAL: split:t
   + Release :: A versioned snapshot of a project.
   + System release :: A snapshot of a system contained within a project's
     release.
   + Source :: Typically used to refer to a repository of projects, releases,
     and metadata, such as [[https://www.quicklisp.org/beta/][Quicklisp]].
   + Context :: A set of requirements and installed releases that satisfy said
     requirements.

** Architecture

   #+REVEAL_HTML: <div class='leftcol'>
   Two major pieces:

55
   + CLPM :: The core of the project manager, intended to be run in a separate
Eric Timmons's avatar
Eric Timmons committed
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
     process. Additionally, has a command line interface for use outside of
     Lisp.
   + CLPM Client :: A small, portable system meant to be loaded into
     development images. Launches CLPM processes and communicates with them
     using a REPL.

   #+REVEAL_HTML: </div>

   #+REVEAL_HTML: <div class='rightcol'>

   #+begin_src plantuml :file architecture.svg
     Client -> CLPM: Start process
     activate CLPM

         Client -> CLPM: (install :systems '("fiveam") :context "default")
         CLPM -> Client: t
         Client -> CLPM: (source-registry :context "default")
         CLPM -> Client: (:source-registry ...)
         Client -> CLPM: (quit)
     deactivate CLPM
   #+end_src

   #+RESULTS:
   [[file:architecture.svg]]
   #+REVEAL_HTML: </div>

*** Architecture Benefits

    This architecture is different from existing solutions (see: [[https://www.quicklisp.org/beta/][Quicklisp]],
    [[https://github.com/fukamachi/qlot][Qlot]], and [[https://qi-cl.com/][Qi]]). It gives us some very nice benefits, though:

    1. CLPM can load and use foreign libraries you likely do not need in your
       development image (such as OpenSSL).
    2. CLPM can use existing libraries without bloating your image or dumped
       executable footprint.
    3. CLPM can use existing Lisp libraries without introducing version or
       package conflicts in your development image.
    4. CLPM can take advantage of implementation specific features without
       sacrificing portability.

** Contexts

   CLPM supports two types of contexts.

*** Global Contexts

    Global contexts have a name and all information about these contexts is
    stored in the CLPM data directory.

    #+ATTR_REVEAL: :frag t :frag_idx 1
    Similar in spirit to Python virtualenvs (at least those created with
    [[https://pypi.org/project/virtualenvwrapper/][virtualenvwrapper]]).

    #+ATTR_REVEAL: :frag t
    Additionally, can write information to standard ASDF source-registry
    locations (see: ~(:contexts "NAME" :source-registry-files)~ config option),
    so ASDF can find the systems without involving CLPM at all. Great for
    installing libraries for global use or when working on projects by
    yourself.

*** Bundle/Anonymous Contexts

    Identified by a pathname, typically to a file named =clpmfile=.

    #+ATTR_REVEAL: :frag t :frag_idx 1
    Designed to be portable, so it can be shared (e.g., in a git repo):

    #+ATTR_REVEAL: :frag t :frag_idx 1
    + =clpmfile= contains sources and top-level requirements.
    + =clpmfile.lock= contains the same information, but also includes the set
      of releases that satisfy the requirements. =clpmfile.lock= is enough to
      recreate the context on another machine.

    #+ATTR_REVEAL: :frag t :frag_idx 2
    Project local configuration of CLPM (such as overriding git checkouts to
    use a copy you already have cloned) accomplished using a file
    =.clpm/bundle.conf= (relative to =clpmfile=).

    #+ATTR_REVEAL: :frag t :frag_idx 3
    Inspired by Ruby's [[https://bundler.io/][Bundler]] and [[https://github.com/fukamachi/qlot][Qlot]]. Great for working on a project with
    many collaborators and deploying projects using a known, working
    configuration.

** Requirements

   Contexts are specified by the user via /requirements/. CLPM understands
   several types, described below. The included examples show how to specify
   that requirement in a =clpmfile=.

*** System requirement

    A system requirement says that a specified system and all its dependencies
    must be available in the context. Can specify:

    + A version range (refering to the ASDF parsed system version). However,
      this is not currently available for Quicklisp sources as that data is not
      present in the metadata.
    + The source from which to install the system.

    #+begin_src common-lisp
      (:system "fiveam")

      (:system "cffi"
       :version ((>= "0.20.0")))
    #+end_src

*** Project requirement

    A project requirement says that all of a project's systems, and all of the
    dependencies of its systems must be available in the context. Can specify:


    + A version range, refering to the version number of the release, which may
      or may not correspond to the version of any system contained in the
      release (depending on the project developer's practices). For Quicklisp
      sources, the release version is date based.
    + The source from which to install the project.

    #+begin_src common-lisp
      (:project "fiveam")

      (:project "cffi"
       :version ((>= "2018-10-20")))
    #+end_src

*** VCS Project requirement

    Similar to a project requirement, except it states that the project must be
    checked out from version control. Can specify:

    + Branch, tag, or commit.
    + The repository (required if the source metadata does not include VCS
      information).

    Note that any system or project directly included in the context via this
    type of requirement satisfies *any* version constraint specified by other
    explicit or implicit requirements.

    #+begin_src common-lisp
      (:project "cffi"
       :vcs (:branch "bugfix-1"))

      (:github "cffi"
       :branch "bugfix-1")
    #+end_src

*** System File Requirement

    This requirement states that a specified .asd file located on the local
    filesystem, outside of one of CLPM's directories, is included in the
    context, as well as any dependencies of the systems defined in it. Can
    specify:

    + Path to the .asd file.

    Note that with this type of requirement, you may need to occasionally have
    CLPM reresolve the requirements. For instance if you add a new dependency
    since the last time you asked CLPM to install things into the context. How
    to do this is covered in the demos below.

    #+begin_src common-lisp
      (:asd "super-awesome.asd")
    #+end_src

** Sources

   CLPM can understand two types of sources for package metadata

*** Quicklisp

    Quicklisp style sources are those that are also compatible with the
    Quicklisp library manager. These sources version releases of projects by
    date.

    There are several disadvantages to Quicklisp sources:

    + They contain no system version information.
    + They routinely do not contain all the dependencies for ASDF
      [[https://common-lisp.net/project/asdf/asdf.html#The-package_002dinferred_002dsystem-extension][package-inferred-systems]].
    + They do not contain version control information for projects.
    + CLPM cannot (yet?) lazily download metadata, an explicit sync step is
      required that may take some time.

*** CLPI

    CLPI based sources are defined by the [[https://gitlab.common-lisp.net/clpm/clpi][CLPI Project]]. This source
    specification was created to serve CLPM's needs, so it contains the
    information for all of CLPM's features, and is designed to enable lazy
    syncing. However, it is nowhere near as mature or battle tested as
    Quicklisp's metadata format.

    #+REVEAL: split:t
    A mirror of the main Quicklisp distribution is stored in CLPI format at
    [[http://quicklisp.common-lisp-project-index.org/]]. However, it is a straight
    translation of the Quicklisp metadata and is missing the same
    information. The CLPM author plans to eventually extend it to include at
    least better dependencies for package-inferred-systems and system version
    information. If you choose to use this mirror, realize that it is currently
    a best-effort project.

** What files does it touch?

   CLPM expects to read and write files in several locations on your hard
   drive. This section summarizes them.

*** Config

263
264
265
266
    On non-Windows systems, CLPM's configuration is stored in the =clpm/=
    subfolder of your [[https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html][XDG]] config directory. On Windows, it is stored in the
    =config/= subfolder of the clpm directory in your local app data
    folder. Therefore, the typical locations are:
Eric Timmons's avatar
Eric Timmons committed
267

268
269
270
    + Linux :: =/home/$USER/.config/clpm/=
    + MacOS :: =/Users/$USER/.config/clpm/=
    + Windows :: =%LOCALAPPDATA%\clpm\config\=
Eric Timmons's avatar
Eric Timmons committed
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285

    #+ATTR_REVEAL: :frag t
    This can be overriden using the =CLPM_CONFIG_DIRS= environment variable.

    #+ATTR_REVEAL: :frag t
    Last, when using bundles, CLPM will project specific configuration from the
    =.clpm/bundle.conf= file, relative to the =clpmfile=.

    #+ATTR_REVEAL: :frag t
    CLPM does not currently write to config locations, but may in future
    versions.

*** Cache

    CLPM caches many files during its operation, including tarballs and package
286
    metadata.
Eric Timmons's avatar
Eric Timmons committed
287

288
289
290
291
292
293
294
295
    On non-Windows systems, these files are stored in the =clpm/= subfolder of
    your XDG cache directory. On Windows, it is stored in the =cache/=
    subfolder of the clpm directory in your local app data folder. Therefore,
    the typical locations are:

    + Linux :: =/home/$USER/.cache/clpm/=
    + MacOS :: =/Users/$USER/.cache/clpm/=
    + Windows :: =%LOCALAPPDATA%\clpm\cache\=
Eric Timmons's avatar
Eric Timmons committed
296
297
298
299
300
301
302
303
304
305
306
307

    #+ATTR_REVEAL: :frag t
    This location can be overriden using the =CLPM_CACHE_DIR= environment
    variable.

    #+ATTR_REVEAL: :frag t
    Typically these files can be deleted any time CLPM is not running and it
    will recreate them as necessary.

*** Data

    CLPM stores context data, unpacked releases, metadata for installed
308
309
310
311
312
313
    packages, etc. in its data directory.

    On non-Windows systems, these files are stored in the =clpm/= subfolder of
    your XDG data directory. On Windows, it is stored in the =data/= subfolder
    of the clpm directory in your local app data folder. Therefore, the typical
    locations are:
Eric Timmons's avatar
Eric Timmons committed
314

315
316
317
    + Linux :: =/home/$USER/.local/share/clpm/=
    + MacOS :: =/Users/$USER/.local/share/clpm/=
    + Windows :: =%LOCALAPPDATA%\clpm\data\=
Eric Timmons's avatar
Eric Timmons committed
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359

    #+ATTR_REVEAL: :frag t
    This location can be overriden using the =CLPM_DATA_DIR= environment
    variable.

    #+ATTR_REVEAL: :frag t
    You should let CLPM manage these files and not edit them manually.

** Additional features

*** ASDF Groveler

    When installing a release directly from version control or your file
    system, there is no metadata for CLPM to use in order to determine
    dependencies. CLPM uses the [[https://gitlab.common-lisp.net/clpm/asdf-system-groveler][asdf-system-groveler]] in a separate process to
    extract the needed information.

*** Groveler Sandbox (Experimental)

    Unfortunately, groveling from .asd files may involve loading arbitrary
    code. Therefore, CLPM is capable of using [[https://firejail.wordpress.com/][Firejail]] to sandbox the groveler
    process if it is installed. This feature is very experimental, and the CLPM
    author would love to have feedback on it, as well as ideas for other
    sandboxing methods to use!

*** CLPI metadata generation (Very Experimental)

    CLPM has very experimental support for extracting the release metadata
    needed for CLPI metadata and packaging it as JSON for sending to a web
    service maintaining a CLPI index. The author is currently trialing this in
    house. Reach out if you're interested in this feature/running your own CLPI
    index server.

* Initial Setup

** Installing CLPM

*** Linux

    + Download the latest tarball for your system. Assuming you are running an
      amd64 processor on most GNU/Linux distributions, it is currently located
      at
Eric Timmons's avatar
v0.4.1    
Eric Timmons committed
360
      [[https://files.clpm.dev/clpm/clpm-0.4.1-linux-amd64.tar.gz]].
Eric Timmons's avatar
Eric Timmons committed
361
    + Download the signed digests for this version from
Eric Timmons's avatar
v0.4.1    
Eric Timmons committed
362
      [[https://files.clpm.dev/clpm/clpm-0.4.1.DIGESTS.asc]].
Eric Timmons's avatar
Eric Timmons committed
363
364
365
366
367
368
    + Download the CLPM Signing Key:
      #+begin_src shell
        gpg --recv-keys 0x10327DE761AB977333B1AD7629932AC49F3044CE
      #+end_src
    + Verify the signature on the digests file:
      #+begin_src shell
Eric Timmons's avatar
v0.4.1    
Eric Timmons committed
369
        gpg --decrypt clpm-0.4.1.DIGESTS.asc
Eric Timmons's avatar
Eric Timmons committed
370
371
372
373
374
375
      #+end_src

    #+REVEAL: split:t
    + Ensure the sha512 sum for the downloaded file matches the one from the
      digests file:
      #+begin_src shell
Eric Timmons's avatar
v0.4.1    
Eric Timmons committed
376
        sha512sum clpm-0.4.1-linux-amd64.tar.gz
Eric Timmons's avatar
Eric Timmons committed
377
378
379
      #+end_src
    + Unpack the tarball:
      #+begin_src shell
Eric Timmons's avatar
v0.4.1    
Eric Timmons committed
380
        tar xf clpm-0.4.1-linux-amd64.tar.gz
Eric Timmons's avatar
Eric Timmons committed
381
382
383
      #+end_src
    + Install CLPM:
      #+begin_src shell
Eric Timmons's avatar
v0.4.1    
Eric Timmons committed
384
        cd clpm-0.4.1-linux-amd64 && sudo sh ./install.sh
Eric Timmons's avatar
Eric Timmons committed
385
386
387
388
389
      #+end_src

*** MacOS

    + Download the latest tarball, currently located at
Eric Timmons's avatar
v0.4.1    
Eric Timmons committed
390
      [[https://files.clpm.dev/clpm/clpm-0.4.1-darwin-amd64.tar.gz]]
Eric Timmons's avatar
Eric Timmons committed
391
    + Download the signed digests for this version from
Eric Timmons's avatar
v0.4.1    
Eric Timmons committed
392
      [[https://files.clpm.dev/clpm/clpm-0.4.1.DIGESTS.asc]].
Eric Timmons's avatar
Eric Timmons committed
393
394
395
396
397
398
    + Download the CLPM Signing Key:
      #+begin_src shell
        gpg --recv-keys 0x10327DE761AB977333B1AD7629932AC49F3044CE
      #+end_src
    + Verify the signature on the digests file:
      #+begin_src shell
Eric Timmons's avatar
v0.4.1    
Eric Timmons committed
399
        gpg --decrypt clpm-0.4.1.DIGESTS.asc
Eric Timmons's avatar
Eric Timmons committed
400
401
402
403
404
405
      #+end_src

    #+REVEAL: split:t
    + Ensure the sha512 sum for the downloaded file matches the one from the
      digests file:
      #+begin_src shell
Eric Timmons's avatar
v0.4.1    
Eric Timmons committed
406
        sha512sum clpm-0.4.1-darwin-amd64.tar.gz
Eric Timmons's avatar
Eric Timmons committed
407
408
409
      #+end_src
    + Unpack the tarball:
      #+begin_src shell
Eric Timmons's avatar
v0.4.1    
Eric Timmons committed
410
        tar xf clpm-0.4.1-darwin-amd64.tar.gz
Eric Timmons's avatar
Eric Timmons committed
411
412
413
      #+end_src
    + Install CLPM:
      #+begin_src shell
Eric Timmons's avatar
v0.4.1    
Eric Timmons committed
414
        cd clpm-0.4.1-darwin-amd64 && sudo sh ./install.sh
Eric Timmons's avatar
Eric Timmons committed
415
416
417
418
419
      #+end_src

*** Windows

    + Download the latest installer, currently located at
Eric Timmons's avatar
v0.4.1    
Eric Timmons committed
420
      [[https://files.clpm.dev/clpm/clpm-0.4.1-windows-amd64.msi]].
Eric Timmons's avatar
Eric Timmons committed
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
    + Run the installer.

    NOTE: I do not think gpg is very common on Windows, but if you use it, you
    should be able to figure out what to do based on the instructions for other
    OSes.

** Configuring sources

   Global sources configred in =sources.conf= in CLPM's config directory.

   The demos assume you have Quicklisp configured as a source. Described below
   are two options to do this. See [[*Sources]] for a discussion of pros and cons
   of each.

*** Quicklisp distribution directly

    Add the following to =sources.conf= to use the primary Quicklisp
    distribution directly:

    #+begin_src common-lisp
      ("quicklisp"
       :type :quicklisp
       :url "https://beta.quicklisp.org/dist/quicklisp.txt")
    #+end_src

*** Quicklisp CLPI Mirror

    Alternatively, if you like living on the edge, add the following to
    =sources.conf=:

    #+begin_src common-lisp
      ("quicklisp"
       :type :ql-clpi
       :url "https://quicklisp.common-lisp-project-index.org/")
    #+end_src

** Configuring ASDF

   ASDF needs to be configured to find the =clpm-client= system. Assuming you
   haven't modified your [[https://common-lisp.net/project/asdf/asdf.html#Controlling-where-ASDF-searches-for-systems][ASDF source registry]] too much, place the output of the
   following command at:

   + Linux/MacOS :: =~.config/common-lisp/source-registry.conf.d/20-clpm-client.conf=
464
   + Windows :: =%LOCALAPPDATA%\config\common-lisp\source-registry.conf.d\20-clpm-client.conf=
Eric Timmons's avatar
Eric Timmons committed
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493

   #+begin_src shell
     clpm client source-registry.d
   #+end_src

** Configuring the Client

   Configure the client by adding code to your implementation's init file.

   Use the following to print the default configuration. Add =--help= to see
   options.

   #+begin_src shell
     clpm client rc
   #+end_src

   #+begin_notes
   The CLPM client can be configured by adding forms to your favorite Lisp
   implementation's initilization script. To use the recommended configuration
   of the CLPM client, place the output of =clpm client rc= in your
   initialization script (for example, =~/.sbclrc= for SBCL).

   CLPM ships with several configurations for the client, depending on your
   preferences. See =clpm client rc --help= for more info (and feel free to
   contribute new configurations!)
   #+end_notes

*** Default configuration

Eric Timmons's avatar
v0.4.1    
Eric Timmons committed
494
    As of v0.4.1, the recommended configuration is:
Eric Timmons's avatar
Eric Timmons committed
495
496
497
498
499

    # #+ATTR_REVEAL: :code_attribs data-line-numbers='1-2|5-7|8-9'
    #+begin_src common-lisp
      ;;; Use CLPM with default configuration.
      ;;;
Eric Timmons's avatar
v0.4.1    
Eric Timmons committed
500
      ;;; Generated by CLPM 0.4.1
Eric Timmons's avatar
Eric Timmons committed
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028

      (require "asdf")
      #-clpm-client
      (when (asdf:find-system "clpm-client" nil)
        ;; Load the CLPM client if we can find it.
        (asdf:load-system "clpm-client")
        (when (uiop:symbol-call :clpm-client '#:active-context)
          ;; If started inside a context (i.e., with `clpm exec` or `clpm bundle exec`),
          ;; activate ASDF integration
          (uiop:symbol-call :clpm-client '#:activate-asdf-integration)))
    #+end_src

    #+ATTR_REVEAL: :frag t
    This configuration:

    #+ATTR_REVEAL: :frag (t)
    1. Loads ASDF.
    2. Loads the CLPM client if its .asd file can be found.
    3. Activates ASDF integration if launched via =clpm exec= or =clpm bundle
       exec=.

* Demo: Using Global Contexts from the REPL

  This demo shows you how to use CLPM while interactively developing from the
  REPL in a global context. Before continuing, please ensure you have installed
  CLPM and configured the client.

  Additionally, clone the [[https://gitlab.common-lisp.net/clpm/clpm-demo][clpm-demo]] project somewhere. (Undoing any changes you
  made from other demos).

** Start your Lisp!

   Start your Lisp! You can check the client was loaded and it can find CLPM by
   evaluating:

   #+begin_src common-lisp
     (clpm-client:clpm-version)
   #+end_src

   #+REVEAL: split:t
   [[file:screen-shots/global-repl/version.png]]

** Syncing sources

   If you are using the Quicklisp distribution directly and have not yet synced
   it, do so now.

   This command will likely take a while as it needs to download data for every
   version of the distribution. You do not need to do this step if you are
   using the CLPI mirror.

   #+begin_src common-lisp
     (clpm-client:sync :sources '("quicklisp"))
   #+end_src

   #+REVEAL: split:t
   [[file:screen-shots/global-repl/sync.png]]

** Activating contexts

   Assuming you did not launch your Lisp as a child process of =clpm bundle
   exec= or =clpm exec= (see the CLI demos for more details), the default
   configuration does not activate any context.

   You can activate or switch contexts using =clpm-client:activate-context=.

   If you switch contexts you will get a continuable error. This is just a
   warning that things may break unexpectedly (e.g., you have already loaded
   systems in some context and are now switching to one where they aren't
   installed or a different version is installed.

   #+REVEAL: split:t

   Activate the =demo= context now by evaluating:

   #+begin_src common-lisp
     (clpm-client:activate-context "demo" :activate-asdf-integration t)
   #+end_src

   This queries CLPM for the ASDF configuration for the =demo= context (source
   registry and output translations). Then it configures ASDF in the running
   image with those settings. Last, it hooks into ASDF's system search functions

   #+REVEAL: split:t
   [[file:screen-shots/global-repl/activate.png]]

** Installing systems

   You can install systems or projects into a context using
   =clpm-client:install=.

   The systems are installed into the context specified by
   =clpm-client:*default-context*=. If that is NIL (the default), it installs
   into the active context. If there is no active context, it falls back to the
   context named =default=.

   Before modifying the context, CLPM produces a diff and presents it to you
   for approval. The presentation is controlled by
   =clpm-client:*context-diff-approval-method*=. It can be set to:

   + =:error= :: Default. A condition is signaled (using =error=, so you will
     be dropped into the debugger). The restarts =approve-diff= and
     =reject-diff= will be available.
   + =:yes-or-no-p= :: The diff is printed and you are asked for approval via
     =yes-or-no-p=.
   + =t= :: The diff is silently accepted.

   #+REVEAL: split:t

   Install the CFFI project into the demo context by evaluating:

   #+begin_src common-lisp
     (clpm-client:install :projects "cffi")
   #+end_src

   #+REVEAL: split:t
   [[file:screen-shots/global-repl/install-diff.png]]

   Note how this installs every system that is part of the CFFI project
   (notably pulling in its testing dependencies), instead of only the CFFI
   system and its dependencies.

** Lazily installing systems

   Alternatively, if you have an activated context and the client's ASDF
   integration activated, you can install systems on demand. This works by
   adding a system search function to ASDF, therefore any attempt to find a
   system that is not installed (e.g., using =asdf:find-system=,
   =asdf:load-system=, etc.) will trigger the client.

   #+REVEAL: split:t

   How the client behaves in the presence of a missing system can be configured
   by the variable =clpm-client:*asdf-system-not-found-behavior*=. It can be
   set to:

   + =:error= :: Default. A condition is signaled (using =error=, so you will
     be dropped into the debugger). The restarts =install-and-reload-config=,
     =reresolve-requirements-and-reload-config=,
     =install-without-dependencies-and-reload-config=, and =reload-config= will
     be available.
   + =:install= :: The client will attempt to install the system and its
     dependencies without prompting.
   + =:install-without-deps= :: The client will attempt to install the system
     without its dependencies without prompting.
   + =nil= :: The client does nothing.

   #+REVEAL: split:t

   Note that these settings combine with
   =clpm-client:*context-diff-approval-method*=. Therefore the default behavior
   is to ask if the system should be installed *and* to ask if the diff is
   approved (both using the debugger).

   #+REVEAL: split:t

   Lazily load 1am by evaluating:

   #+begin_src common-lisp
     (asdf:load-system "1am")
   #+end_src

   Invoke the =install-and-reload-config= restart and then the =approve-diff=
   restart.

   #+REVEAL: split:t
   [[file:screen-shots/global-repl/load-error.png]]

   #+REVEAL: split:t
   [[file:screen-shots/global-repl/load-diff.png]]

** Updating

   When new versions of your dependencies are released, you can easily update a
   context using =clpm-client:update=. By default, all projects are considered
   eligible for upgrading (subject to requirements specified via
   =install=). However, only specific systems or projects can be upgraded by
   specifying the =:projects= and/or =:systems= arguments.

   #+REVEAL: split:t

   To update everything in the demo context, run:

   #+begin_src common-lisp
     (clpm-client:update)
   #+end_src

   If you would like to limit the updates to particular systems or projects,
   use the appropriate keywords. For example:

   #+begin_src common-lisp
     (clpm-client:update :projects "1am")
   #+end_src

   This probably won't do anything right now since you already have the latest
   versions!

** Installing an editable system

   CLPM allows you to "install" a system into a context using a pointer to the
   .asd file on your local hard drive. This is great for developing software as
   you can leave the code in a folder not under CLPM's control, but CLPM will
   still examine it to determine and install dependencies.

   #+REVEAL: split:t

   Install the clpm-demo project into the demo context by evaluating:

   #+begin_src common-lisp
     (clpm-client:install :asds '("/path/to/clpm-demo/clpm-demo.asd"
                                  "/path/to/clpm-demo/clpm-demo-test.asd"))
   #+end_src

   Then load the system by evaluating:

   #+begin_src common-lisp
     (asdf:load-system :clpm-demo)
   #+end_src

   #+REVEAL: split:t
   [[file:screen-shots/global-repl/editable-diff.png]]

** Adding a dependency to an editable system

   Look at =clpm-demo-test.asd= and =test.lisp=. We have a test suite defined
   using fiveam, but we forgot to declare a dependency! Add fiveam to the
   =:depends-on= for the =clpm-demo-test= system:

   #+begin_src common-lisp
     (defsystem #:clpm-demo-test
       :version "0.1.0"
       :description "CLPM Demo Test System"
       :license "BSD-2-Clause"
       :depends-on (#:fiveam #:clpm-demo)
       :components
       ((:file "test")))
   #+end_src

   #+REVEAL: split:t

   Now let's run the tests by evaluating:

   #+begin_src common-lisp
     (asdf:test-system :clpm-demo)
   #+end_src

   Uh oh, we don't have fiveam installed! This is the perfect time to use the
   =reresolve-requirements-and-reload-config= restart. It will notice the new
   dependency and ask you to install it. Accept the diff and watch the test
   pass.

   #+REVEAL: split:t
   [[file:screen-shots/global-repl/fiveam-error.png]]

   #+REVEAL: split:t
   [[file:screen-shots/global-repl/fiveam-diff.png]]

   #+REVEAL: split:t
   [[file:screen-shots/global-repl/tests-pass.png]]

* Demo: Using Bundles from the REPL

  This demo shows you how to use CLPM while interactively developing from the
  REPL in an anonymous, project specific context (bundle). Before continuing,
  please ensure you have installed CLPM and configured the client.

  Additionally, clone the [[https://gitlab.common-lisp.net/clpm/clpm-demo][clpm-demo]] project somewhere. (Undoing any changes you
  made from other demos).

** Start your Lisp!

   Start your Lisp! You can check the client was loaded and it can find CLPM by
   evaluating:

   #+begin_src common-lisp
     (clpm-client:clpm-version)
   #+end_src

   #+REVEAL: split:t
   [[file:screen-shots/bundle-repl/version.png]]

** Creating the bundle

   A bundle is a project specific context defined by two files.

   =clpmfile= is generated and maintained by the developer. It contains the
   sources and requirements for the bundle.

   The second, =clpmfile.lock= is generated by CLPM. It contains a copy of the
   information from the =clpmfile=, as well as a list of releases that satisfy
   the requirments. Additionally, it contains a dependency tree. This is not
   currently used by CLPM, and at the moment is purely to provide the developer
   information.

   #+REVEAL: split:t

   Initialize a new clpmfile for the =clpm-demo= project by evaluating:

   #+begin_src common-lisp
     (clpm-client:bundle-init #p"/path/to/clpm-demo/clpmfile"
                              :asds '("clpm-demo.asd" "clpm-demo-test.asd"))
   #+end_src

   #+ATTR_REVEAL: :frag t :frag_idx 1
   This creates a new file named clpmfile at =/path/to/clpm-demo/clpmfile=. It
   should contain the following:

   #+ATTR_REVEAL: :frag t :frag_idx 1
   #+begin_src common-lisp
     ;;; -*- Mode: common-lisp; -*-
     (:api-version "0.3")

     (:source "quicklisp" :url "https://beta.quicklisp.org/dist/quicklisp.txt" :type :quicklisp)

     (:asd "clpm-demo.asd")
     (:asd "clpm-demo-test.asd")
   #+end_src

   #+REVEAL: split:t
   [[file:screen-shots/bundle-repl/init.png]]

** Installing the bundle

   Before you can use the new bundle, you need to install the dependencies! You
   can do this using =clpm-client:install=. As you do not have an active
   context yet, you need to specify the context.

   When specifying a bundle as the context, it is required that you use a
   pathname!

   #+REVEAL: split:t

   Install all the dependencies for the bundle. Don't forget the =#p=!

   #+begin_src common-lisp
     (clpm-client:install :context #p"/path/to/clpm-demo/clpmfile")
   #+end_src

   #+REVEAL: split:t
   [[file:screen-shots/bundle-repl/install.png]]

** Activating the bundle

   Once you have a lock file, you can activate the context specified by the
   bundle using =clpm-client:activate-context=.

   If you switch contexts you will get a continuable error. This is just a
   warning that things may break unexpectedly (e.g., you have already loaded
   systems in some context and are now switching to one where they aren't
   installed or a different version is installed.

   #+REVEAL: split:t

   Activate the bundle and the ASDF integration by evaluating:

   #+begin_src common-lisp
     (clpm-client:activate-context #p"/path/to/clpm-demo/clpmfile"
                                   :activate-asdf-integration t)
   #+end_src

   And load the system:

   #+begin_src common-lisp
     (asdf:load-system :clpm-demo)
   #+end_src

   #+REVEAL: split:t
   [[file:screen-shots/bundle-repl/activate.png]]

** Test the system

   Look at =clpm-demo-test.asd= and =test.lisp=. We have a test suite defined
   using fiveam, but we forgot to declare a dependency! Add fiveam to the
   =:depends-on= for the =clpm-demo-test= system:

   #+begin_src common-lisp
     (defsystem #:clpm-demo-test
       :version "0.1.0"
       :description "CLPM Demo Test System"
       :license "BSD-2-Clause"
       :depends-on (#:fiveam #:clpm-demo)
       :components
       ((:file "test")))
   #+end_src

   #+REVEAL: split:t

   Now run the tests by evaluating:

   #+begin_src common-lisp
     (asdf:test-system :clpm-demo)
   #+end_src

   Uh oh, we don't have fiveam installed! This is the perfect time to use the
   =reresolve-requirements-and-reload-config= restart. It will notice the new
   dependency and ask you to install it. Accept the diff and watch the test
   pass.

   #+REVEAL: split:t
   [[file:screen-shots/bundle-repl/fiveam-error.png]]

   #+REVEAL: split:t
   [[file:screen-shots/bundle-repl/tests-pass.png]]

** Installing projects from VCS

   Now, let's say you decide that there's a feature in Fiveam's git repository
   you want to use, but it hasn't been released to Quicklisp yet. No worries!
   Add the following to the =clpmfile=:

   #+begin_src common-lisp
     (:github "fiveam"
      :path "sionescu/fiveam"
      :branch "master")
   #+end_src

   #+REVEAL: split:t
   And reinstall the context (so it gets the new requirement):

   #+begin_src common-lisp
     (clpm-client:install :context #p"/path/to/clpm-demo/clpmfile")
   #+end_src

   The diff will show you that it is changing from a released version of fiveam
   to a specific commit from the git repo.

   #+REVEAL: split:t
   [[file:screen-shots/bundle-repl/fiveam-git.png]]

   #+REVEAL: split:t
   If you retest =clpm-demo= now, you'll see that it recompiles fiveam using
   files from a different location.

   #+begin_src common-lisp
     (let ((asdf:*compile-file-failure-behaviour* :ignore))
       (asdf:test-system :clpm-demo))
   #+end_src

   The binding for =*compile-file-failure-behavior*= was required at the time
   of writing as FiveAM's master branch had a package definition in variance
   with the one in the version released in Quicklisp.

** Overriding VCS repos

   Last, let's say you discover a bug in fiveam and want to use a local copy of
   fiveam in your bundle while you fix it. This can be accomplished by cloning
   the fiveam repo next to the =clpm-demo= folder and adding the following to
   =clpm-demo/.clpm/bundle.conf=:

   #+begin_src common-lisp
     ;;; -*- mode: common-lisp; -*-

     (version "0.2")

     ((:bundle :local "fiveam")
      "../fiveam/")
   #+end_src

   This tells CLPM that if it encounters a VCS requirement for fiveam, it
   should be satisfied using the repository located at =../fiveam/= (relative
   to the =clpmfile=).

   #+REVEAL: split:t

   #+REVEAL: split:t
   Reinstall the context (so it gets the new override):

   #+begin_src common-lisp
     (clpm-client:install :context #p"/path/to/clpm-demo/clpmfile")
   #+end_src

   In the current diff format, you want see an entry on the diff, but you still
   need to approve it for the change it take effect in the current
   session. This will hopefully be made more clear in v0.4.

   #+REVEAL: split:t
   [[file:screen-shots/bundle-repl/fiveam-override.png]]

   #+REVEAL: split:t

   Using this feature requires great care. If you commit a version of
   =clpmfile.lock= that references a fiveam commit that you have not pushed,
   all of your collaborators will have a bad day and blame it on you!

* Demo: Using CLPM from the CLI

  This demo shows off a subset of CLPM's capabilities that can be accessed
  using its command line interface.

  Most of what was described in previous demos will also work from the CLI and
  now that you know the concepts, the details can hopefully be figured out by
  adding =--help= to any invocation of =clpm=.

  A feature of CLPM that is currently unique to its CLI is the =exec=
  interface. That is described below.

** Executing a process in a global context

   If you would like to use CLPM but not have to deal with the client, you can
   start processes with environment variables set that allow ASDF to be
   configured using CLPM without the client.

   #+REVEAL: split:t
   For example, to run an sbcl process with ASDF configured to use systems
   installed in the =demo= context, run the following command:

   #+begin_src shell
     clpm exec --context=demo -- sbcl
   #+end_src

   The =--= is not required, but highly recommended as it keeps CLPM from
   trying to parse any options given to SBCL as options for CLPM.

   This sets the =CL_SOURCE_REGISTRY= and =ASDF_OUTPUT_TRANSLATIONS=
   environment variables to configure ASDF, along with some others in case you
   choose to load the client in the new process.

** Executing a process in a bundle

   If you would like to execute a command with the environment configured to
   use a CLPM bundle, instead run:

   #+begin_src shell
     clpm bundle exec -- sbcl
   #+end_src

   It defaults to finding =clpmfile= in the current directory, but that can be
   overriden with =-f=.