Skip to content

Draft: Update CI Scripts

Eric Timmons requested to merge etimmons/iterate:clci into master

I'm on a personal quest to make automated testing of CL projects easier. I've been pulling out the Gitlab CI scripts I've accumulated over the years, cleaning them up, documenting them, and packaging them up in a way that's trivial to use. It's not quite ready for prime time yet, but I figured I'd give it a try with a project that's not mine to see how it looks, make sure I haven't been focused too narrowly on my personal projects, and get feedback from others :).

I like the results so far on iterate and it actually gives some interesting results that the current approach misses, so that's nice. It appears that tests actually fail on Clasp and CMUCL. The current Makefile just invokes them incorrectly and somehow fails to exit with a non-zero status. Additionally, tests fail on clisp. Last, the ABCL tests appear flaky; while testing, I saw failures in FINDING.MINIMIZING.1, MINIMIZE.1, and MINIMIZE.2 that were resolved by an immediate retry with no changes to the code https://gitlab.common-lisp.net/etimmons/iterate/-/jobs/21206.

In a nutshell, this CLCI project I'm working on defines a bunch of yml files for Gitlab CI. definitions.gitlab-ci.yml contains a bunch of useful snippets that one could use to build up more complicated pipelines. test-pipeline.gitlab-ci.yml does what it sounds like: tests on a bunch of different implementations. Each test job downloads the latest version of ASDF (specified by the $CLCI_ASDF_VERSION variable), loads it, then loads the file pointed to by $CLCI_TEST_SCRIPT. If the test script errors or explicitly exits with a non-zero code, the test is considered a failure.

Additionally, I have a release-pipeline.gitlab-ci.yml file. This pipeline will run any time a tag is pushed that is named v followed by anything that looks like a version number. It will extract the correct entry from changelog.md, publish the release using Gitlab's API (so it'd show up under "Deployments > Releases" for the project), and announce it. Right now, I've got the ability to announce the release using Slack webhooks, but I would love to add more announcement methods like email (or really anything that's not very project specific).

So, would you be interested in using this? And do you have any feedback on what you see so far?

Edited by Eric Timmons

Merge request reports