A separate allegro image exists, why not add allegro to the cl-devel image?
When trying to load an FFI dependent lisp library, e.g. cl-glfw-opengl-sgix_shadow_ambient, with ABCL, one gets error like this:
; Caught SIMPLE-WARNING:
; Please obtain and install maven-3.0.3 or later locally from <http://maven.apache.org/download.html>, then set ABCL-ASDF:*MVN-LIBS-DIRECTORY* to the directory containing maven-core-3.*.jar et. al.
Unhandled SERIOUS-CONDITION of type COMMON-LISP:ERROR is signaled: Class not found: com.sun.jna.Pointer
This is because ABCL uses maven to download the JNA library it uses for FFI.
My current workaround is to have maven downloaded from the said URL to a directory mounted to the docker image as the cl
user home, and then have M2_HOME and M2 variables exported in my main script (located in the same mounted directory)
THIS_DIR=$(realpath "`dirname $0`")
export M2_HOME=$THIS_DIR/apache-maven-3.8.6
export M2=$M2_HOME/bin
# ... run lisp
(Actually any one of the two variables would be enough I believe, although not tested, because I needed both in my environment for some other reason).
Note, maven by default downloads libraries to ~/.m2/, so having home directory mounted helps here - the download happens only once and then reused by the container on next run.
The ABCL code that looks up for Maven installation is here: https://github.com/armedbear/abcl/blob/master/contrib/abcl-asdf/maven.lisp But I don't know which of these functions are required for most crucial things like FFI, and which are optional or used only in specific situations.
It may be good to have in the Docker image Maven already installed, and maybe JNA already downloaded.
$ ecl
ecl: error while loading shared libraries: libffi.so.7: cannot open shared object file: No such file or directory
The web page - https://cl-docker-images.common-lisp.dev/images/development.html - says:
Additionally, the following Lisp utilities are installed:
ASDF v3.3.4: The source code is located in /usr/local/share/common-lisp/source/asdf. The first time you attempt to use ASDF for anything non trivial, it will upgrade itself to this version.
Does it mean the lisp implementations are configured to use this ASDF installation instead of the default ASDF that comes with the lisp impl?
For example, when a user simply runs sbcl
or ccl
does he anyhow get this ASDF v3.3.4 installation, or the default ASDF normally coming with the lisp implementations will be upgraded to that version at some point?
I have tested, SBCL and CCL seem to use different ASDF version, but the wording of this paragraph leaves impression the ASDF version may be overridden for all the implementations.
In parallel, it would be good to go through and see which systems are causing this problem and submit patches upstream. I maintain that it's a bug for a system to use
uiop:define-package
(or any UIOP functionality, but especially macros) without recording UIOP as a system dependency. Does cl-test-grid have any analysis tools that would easily let us find the systems where this bug originates
At the moment I can't think of an automated way, but from the errors in the report, fare-utils is a very often cause - it uses uiop:define-package
without specifying depends-on (:uiop)
.
Since fare-utils does not use the :local-nicknames
feature, the #189 fix will remove the error. However, it's probably more correct to declare UIOP anyways. @frideau, what do you think?
It is my understanding as well, that the situation when at runtime the UIOP is different than was used during compilation, happens because a lower level library, like fare-utils, was first compiled in a context where new UIOP is loaded from quicklisp by being declared elsewhere in the dependency three of the root system that is passed to `(ql:quickload ...); and later, when another library is being loaded that does not have UIOP specified in the dependency tree at all, the default UIOP coming from the Lisp implementation's ADSF is used, but fare-utils is already compiled with the new UIOP.
As @rgoldman correctly mentions, the #189 is not part of the quicklisp-2021-08-07. As we can see in http://beta.quicklisp.org/dist/quicklisp/2021-08-07/releases.txt, the following is used: http://beta.quicklisp.org/archive/uiop/2021-08-07/uiop-3.3.5.tgz
In result, when this package is downloaded, in the quicklisp/dists/quicklisp/software/uiop-3.3.5/package.lisp we have the old
:finally (return `(',package
:nicknames ',nicknames :documentation ',documentation
:use ',(if use-p use '(:common-lisp))
:shadow ',shadow :shadowing-import-from ',shadowing-import-from
:import-from ',import-from :export ',export :intern ',intern
:recycle ',(if recycle-p recycle (cons package nicknames))
:mix ',mix :reexport ',reexport :unintern ',unintern
:local-nicknames ',local-nicknames)))))
I'm not certain how cl-test-grid works, but I assume you create a new process for every system you want to test? If that's the case, you can also fix this without using 3.3.5.1+ by clearing out the fasl cache between each system you test. That's because this bug poisons that cache.
For the record, cl-test-grid first arranges for a clean fasl directory, then creates process for every system, in alphabetical order by project / system, without cleaning the fals directory between process invocations. That's intentional, it allows faster testing and allows to capture compilation order problems like that. (There is no goal to capture them all - that would require trying many compilation orders, but at least one ordering is covered).
If I correctly understand, the problem is that code compiled in presence of new uiop gets the uiop:define-package macro expanded in a way that is incompatible with old uiop, so when this code is loaded with old uiop, the macro-generated code tries to call that old uiop in the new way, which results in an error.
If so, a possible complete solution could be to have the uiop:define-package
generate a code, that at runtime checks what uiop is available and calls corresponding runtime function:
(if (old-uiop-p)
(apply old-function old-args)
(apply new-function new-args))
When testing all libs in quicklisp 2021-08-07, many of them fail with that error: https://common-lisp.net/project/cl-test-grid/ql/quicklisp-2021-08-07-diff2.html
The fix that closed this issue, was it expected as a complete fix or partial?