|
|
|
<!-- Name: BuildingCmucl -->
|
|
|
|
<!-- Version: 8 -->
|
|
|
|
<!-- Last-Modified: 2014/10/12 09:28:48 -->
|
|
|
|
<!-- Author: rtoy -->
|
|
|
|
# Building CMU CL
|
|
|
|
<!-- [[PageOutline]] -->
|
|
|
|
This document is intended to give you a general overview of the build
|
|
|
|
process (i.e. what needs to be done, in what order, and what is it
|
|
|
|
generally called). It will also tell you how to set up a suitable
|
|
|
|
build environment, how the individual scripts fit into the general
|
|
|
|
scheme of things, and give you a couple of examples.
|
|
|
|
|
|
|
|
## General Requirements
|
|
|
|
|
|
|
|
|
|
|
|
In order to build CMU CL, you will need:
|
|
|
|
```
|
|
|
|
#!html
|
|
|
|
<ol type="a">
|
|
|
|
<li> A working CMU CL binary.
|
|
|
|
<p>There is no way around this requirement!
|
|
|
|
This binary can either be for the platform you want to target, in
|
|
|
|
that case you can either recompile or cross-compile, or for another
|
|
|
|
supported platform, in that case you must cross-compile, obviously.</p>
|
|
|
|
</li>
|
|
|
|
<li> A supported C compiler for the C runtime code.
|
|
|
|
<p>Most of the time, this means GNU gcc, though for some ports it
|
|
|
|
means the vendor-supplied C compiler. The compiler must be
|
|
|
|
available under the name specified by your ports Config file.</p>
|
|
|
|
</li>
|
|
|
|
<li> GNU make.
|
|
|
|
<p>This has to be available either as gmake or make in your <code>PATH</code>, or
|
|
|
|
the <code>MAKE</code> environment variable has to be set to point to the correct
|
|
|
|
binary.</p>
|
|
|
|
</li>
|
|
|
|
<li> The CMU CL source code.
|
|
|
|
<p>Here you can either use one of the release source tarballs, or
|
|
|
|
check out the source code directly from the public CMUCL git
|
|
|
|
repository.</p>
|
|
|
|
</li>
|
|
|
|
</ol>
|
|
|
|
```
|
|
|
|
If you want to build CMU CL's Motif interface/toolkit, you'll need a
|
|
|
|
working version of the Motif libraries, either true-blue OSF/Motif, or
|
|
|
|
[OpenMotif](http://www.openmotif.org), or [LessTif](http://www.lestif.org). The code
|
|
|
|
was developed against 1.2 Motif, though recompilation against 2.x
|
|
|
|
Motif probably works as well.
|
|
|
|
|
|
|
|
## Setting up a build environment
|
|
|
|
|
|
|
|
```
|
|
|
|
#!html
|
|
|
|
<ol>
|
|
|
|
<li> Create a base directory and change to it
|
|
|
|
<pre>
|
|
|
|
</pre>
|
|
|
|
</li>
|
|
|
|
<li> Fetch the sources and put them into the base directory
|
|
|
|
<pre>
|
|
|
|
</pre> <p> or, if you want to use the git sources directly:</p>
|
|
|
|
<pre>
|
|
|
|
</pre> <p>Whatever you do, the sources must be in a directory named <code>src</code>
|
|
|
|
|
|
|
|
</li>
|
|
|
|
</ol>
|
|
|
|
```
|
|
|
|
That's it, you are now ready to build CMU CL.
|
|
|
|
|
|
|
|
## A quick guide for simple builds
|
|
|
|
|
|
|
|
We recommend that you read all of this document, but in case you don't
|
|
|
|
want to do that and in case you know, somehow, that the version of
|
|
|
|
CMUCL you are building from will build the sources you have, here is a
|
|
|
|
quick guide.
|
|
|
|
```
|
|
|
|
#!html
|
|
|
|
<ol type="a">
|
|
|
|
<li> Simple builds
|
|
|
|
<p> Use this to build from a version of CMUCL that is very close to the
|
|
|
|
sources you are trying to build now:</p>
|
|
|
|
<pre>
|
|
|
|
bin/build.sh -C "" -o "<path-to-old-lisp> <options-to-lisp>"
|
|
|
|
</pre>
|
|
|
|
<p>For example, let's say you want to use the 2012-05 snapshot that
|
|
|
|
you stored in the <code>$HOME/2012-05</code> directory. Then
|
|
|
|
<code><path-to-old-lisp></code> would be
|
|
|
|
<code>$HOME/2012-05/bin/lisp</code>. Usually, no options are
|
|
|
|
needed so <code><options-to-lisp><code> is not needed.</p>
|
|
|
|
|
|
|
|
<p>This will build CMUCL 3 times, each time with the result of the
|
|
|
|
previous build. The last time, the additional libraries like CLX,
|
|
|
|
CLM, and Hemlock are built. The final result will be in the
|
|
|
|
directory build-4.</p>
|
|
|
|
|
|
|
|
<p>This script basically runs create-target.sh, build-world.sh,
|
|
|
|
load-world.sh three times. See below for descriptions of these
|
|
|
|
scripts.</p>
|
|
|
|
</li>
|
|
|
|
<li> Slightly more complicated builds
|
|
|
|
<p>For slightly more complicated builds, you may need to use some
|
|
|
|
bootstrap files. See below for more information about these
|
|
|
|
bootstrap files. </p>
|
|
|
|
|
|
|
|
<p>For these, you can use this:</p>
|
|
|
|
<pre>
|
|
|
|
bin/build.sh -C "" -o "<old-lisp>" -B boot1.lisp -B boot2.lisp
|
|
|
|
</pre>
|
|
|
|
<p>The bootstrap files listed with the -B option (as many as needed)
|
|
|
|
are loaded in order, so be sure to get them right.</p>
|
|
|
|
|
|
|
|
<p>As in a) above, three builds are done, and the result is in the
|
|
|
|
directory build-4.</p>
|
|
|
|
</li>
|
|
|
|
<li> More complicated builds
|
|
|
|
<p>If you have more complicated builds, this script probably will not
|
|
|
|
work, and definitely does not handle cross-compiles. In this case,
|
|
|
|
you will have to invoke the individual scripts by hand, as
|
|
|
|
described below.</p>
|
|
|
|
</li>
|
|
|
|
</ol>
|
|
|
|
```
|
|
|
|
|
|
|
|
How do you know which of the three options above apply? The easiest
|
|
|
|
way is to look in `src/bootfiles/<version>/*` for boot files. If the
|
|
|
|
file date of a boot file is later than the version of CMUCL you are
|
|
|
|
building from, then you need to use b) or c) above. You may need to
|
|
|
|
read the bootfiles for additional instructions, if any.
|
|
|
|
|
|
|
|
If there are no bootfiles, then you can use a) above.
|
|
|
|
|
|
|
|
The `build.sh` script supports other options, and `bin/build.sh -?`
|
|
|
|
will give a quick summary. Read `bin/build.sh` for more
|
|
|
|
information.
|
|
|
|
|
|
|
|
## A general outline of the build process
|
|
|
|
|
|
|
|
Building CMU CL can happen in one of two ways: Normal recompilation,
|
|
|
|
and cross-compilation. We'll first look at normal recompilation:
|
|
|
|
|
|
|
|
The recompilation process basically consists of 4 phases/parts:
|
|
|
|
```
|
|
|
|
#!html
|
|
|
|
<ol type="a">
|
|
|
|
<li> Compiling the lisp files that make up the standard kernel.
|
|
|
|
<p>This happens in your current CMU CL process, using your current
|
|
|
|
CMU CL's normal file compiler. This phase currently consists of 3
|
|
|
|
sub-phases, namely those controlled by <code>src/tools/worldcom.lisp</code>,
|
|
|
|
which compiles all the runtime files, <code>src/tools/comcom.lisp</code>, which
|
|
|
|
compiles the compiler (including your chosen backend), and finally
|
|
|
|
src/tools/pclcom.lisp, which compiles PCL, CMU CL's CLOS
|
|
|
|
implementation. The whole phase is often called "world-compile",
|
|
|
|
or "compiling up a world", based on the name of the first
|
|
|
|
sub-phase.</p>
|
|
|
|
</li>
|
|
|
|
<li> Building a new kernel.core file out of the so created files
|
|
|
|
<p>This process, which is generally called genesis, and which is
|
|
|
|
controlled by <code>src/tools/worldbuild.lisp</code>, uses the newly compiled
|
|
|
|
files in order to build a new, basic core file, which is then used
|
|
|
|
by the last phase to create a fully functional normal core file.
|
|
|
|
It does this by "loading" the compiled files into an in-core
|
|
|
|
representation of a new core file, which is then dumped out to
|
|
|
|
disk, together with lots of fixups that need to happen once the new
|
|
|
|
core is started.</p>
|
|
|
|
|
|
|
|
<p>As part of this process, it also creates the file <code>internals.h</code>,
|
|
|
|
which contains information about the general memory layout of the
|
|
|
|
new core and its basic types, their type tags, and the location of
|
|
|
|
several important constants and other variables, that are needed by
|
|
|
|
the C runtime code to work with the given core.</p>
|
|
|
|
|
|
|
|
<p>So going through genesis is needed to create internals.h, which is
|
|
|
|
needed to compile the C runtime code (i.e. the "lisp" binary).
|
|
|
|
However there is a slight circularity here, since genesis needs as
|
|
|
|
one of its inputs the file target:lisp/lisp.nm, which contains the
|
|
|
|
(slightly pre-treated) output of running nm on the new lisp
|
|
|
|
binary. Genesis uses this information to fixup the addresses of C
|
|
|
|
runtime support functions for calls from Lisp code.</p>
|
|
|
|
|
|
|
|
<p>However the circularity isn't complete, since genesis can work with
|
|
|
|
an empty/bogus lisp.nm file. While the kernel.core it then
|
|
|
|
produces is unusable, it will create a usable internals.h file,
|
|
|
|
which can be used to recompile the C runtime code, producing a
|
|
|
|
usable lisp.nm file, which in turn can be used to restart genesis,
|
|
|
|
producing a working kernel.core file.</p>
|
|
|
|
|
|
|
|
<p>Genesis also checks whether the newly produced internals.h file
|
|
|
|
differs from a pre-existing internals.h file (this might be caused
|
|
|
|
by an empty internals.h file if you are rebuilding for the first
|
|
|
|
time, or by changes in the lisp sources that cause differences in
|
|
|
|
the memory layout of the kernel.core), and informs you of this, so
|
|
|
|
that you can recompile the C runtime code, and restart genesis.</p>
|
|
|
|
|
|
|
|
<p>If it doesn't inform you of this, you can skip directly to the last
|
|
|
|
phase d).</p>
|
|
|
|
|
|
|
|
</li>
|
|
|
|
<li> Recompiling the C runtime code, producing the "lisp" binary file
|
|
|
|
<p>This step is only needed if you haven't yet got a suitable lisp
|
|
|
|
binary, or if the internals.h file has changed during genesis (of
|
|
|
|
which genesis informs you), or when you made changes to the C
|
|
|
|
sources that you want to take effect.</p>
|
|
|
|
|
|
|
|
<p>Recompiling the C runtime code is controlled by a GNU Makefile, and
|
|
|
|
your target's Config file. It depends on a correct internals.h
|
|
|
|
file as produced by genesis.</p>
|
|
|
|
|
|
|
|
<p>Note that whenever you recompile the runtime code, for whatever
|
|
|
|
reason, you must redo phase b). Note that if you make changes to
|
|
|
|
the C sources and recompile because of this, you can do that before
|
|
|
|
Phase b), so that you don't have to perform that phase twice.</p>
|
|
|
|
</li>
|
|
|
|
|
|
|
|
<li> Populating the kernel.core, and dumping a new lisp.core file.
|
|
|
|
<p>In this phase, which is controlled by src/tools/worldload.lisp, and
|
|
|
|
hence often called world-load, the kernel.core file is started up
|
|
|
|
using the (possibly new) lisp binary, the remaining files which
|
|
|
|
were compiled in phase a) are loaded into it, and a new lisp.core
|
|
|
|
file is dumped out.</p>
|
|
|
|
</li>
|
|
|
|
</ol>
|
|
|
|
```
|
|
|
|
We're not quite done yet. This produces just a basic lisp.core.
|
|
|
|
To complete the build so that you something similar to what the
|
|
|
|
releases of CMUCL do, there are a few more steps:
|
|
|
|
```
|
|
|
|
#!html
|
|
|
|
<ol type="a">
|
|
|
|
<li> Build the utilities like Gray streams, simple streams, CLX, CLM,
|
|
|
|
and Hemlock. Use the bin/build-utils.sh script for this, as
|
|
|
|
described below
|
|
|
|
</li>
|
|
|
|
<li> Create tarfiles using the bin/make-dist.sh script, as
|
|
|
|
explained below.
|
|
|
|
</li>
|
|
|
|
</ol>
|
|
|
|
```
|
|
|
|
With these tarfiles, you can install them anywhere. The contents of
|
|
|
|
the tarfiles will be the same as the snapshots and releases of CMUCL.
|
|
|
|
|
|
|
|
When cross-compiling, there is additional phase at the beginning, and
|
|
|
|
some of the phases happen with different hosts/platforms. The initial
|
|
|
|
phase is setting up and compiling the cross-compilation backend, using
|
|
|
|
your current compiler. The new backend is then loaded, and all
|
|
|
|
compilation in phase a) happens using this compiler backend. The
|
|
|
|
creation of the kernel.core file in phase b) happens as usual, while
|
|
|
|
phase c) of course happens on the target platform (if that differs
|
|
|
|
from the host platform), as does the final phase d). Another major
|
|
|
|
difference is that you can't compile PCL using the cross-compiler, so
|
|
|
|
one usually does a normal rebuild using the cross-compiled core on the
|
|
|
|
target platform to get a full CMU CL core.
|
|
|
|
|
|
|
|
So, now you know all about CMU CL compilation, how does that map onto
|
|
|
|
the scripts included with this little text?
|
|
|
|
|
|
|
|
## Overview of the included build scripts
|
|
|
|
|
|
|
|
`bin/build.sh [-123obvuBCU?]`::
|
|
|
|
This is the main build script. It essentially calls the other build
|
|
|
|
scripts described below in the proper sequence to build cmucl from an
|
|
|
|
existing binary of cmucl.
|
|
|
|
|
|
|
|
`bin/create-target.sh target-directory [lisp-variant [motif-variant]]`::
|
|
|
|
This script creates a new target directory, which is a shadow of the
|
|
|
|
source directory, that will contain all the files that are created by
|
|
|
|
the build process. Thus, each target's files are completely separate
|
|
|
|
from the src directory, which could, in fact, be read-only. Hence you
|
|
|
|
can simultaneously build CMUCL for different targets from the same
|
|
|
|
source directory.
|
|
|
|
|
|
|
|
The first argument is the name of the target directory to create. The
|
|
|
|
remaining arguments are optional. If they are not given, the script
|
|
|
|
tries to determine the lisp variant and motif variant from the system
|
|
|
|
the script is running on.
|
|
|
|
|
|
|
|
The lisp-variant (i.e. the suffix of the src/lisp/Config.* to use as
|
|
|
|
the target's Config file), and optionally the motif-variant (again the
|
|
|
|
suffix of the src/motif/server/Config.* file to use as the Config file
|
|
|
|
for the target's CMUCL/Motif server code). If the lisp-variant is
|
|
|
|
given but the motif-variant is not, the motif-variant is determined
|
|
|
|
from the lisp-variant.
|
|
|
|
|
|
|
|
The script will generate the target directory tree, link the relevant
|
|
|
|
Config files, and generate place-holder files for various files, in
|
|
|
|
order to ensure proper operation of the other build-scripts. It also
|
|
|
|
creates a sample setenv.lisp file in the target directory, which is
|
|
|
|
used by the build and load processes to set up the correct list of
|
|
|
|
*features* for your target lisp core.
|
|
|
|
|
|
|
|
IMPORTANT: You will normally NOT have to modify the sample setenv.lisp
|
|
|
|
file, if you are building from a binary that has the desired features.
|
|
|
|
In fact, the sample has all code commented out, If you want to add or
|
|
|
|
remove features, you need to include code that puts at least a minimal
|
|
|
|
set of features onto the list (use PUSHNEW and/or REMOVE). You can
|
|
|
|
use the current set of *features* of your lisp as a first guide. The
|
|
|
|
sample setenv.lisp includes a set of features that should work for the
|
|
|
|
intended configuration. Note also that some adding or removing some
|
|
|
|
features may require a cross-compile instead of a normal compile.
|
|
|
|
|
|
|
|
`bin/clean-target.sh [-l] target-directory [more dirs]`::
|
|
|
|
Cleans the given target directory, so that all created files will be
|
|
|
|
removed. This is useful to force recompilation. If the -l flag is
|
|
|
|
given, then the C runtime is also removed, including all the lisp
|
|
|
|
executable, any lisp cores, all object files, lisp.nm, internals.h,
|
|
|
|
and the config file.
|
|
|
|
|
|
|
|
`bin/build-world.sh target-directory [build-binary] [build-flags...]`::
|
|
|
|
Starts a complete world build for the given target, using the lisp
|
|
|
|
binary/core specified as a build host. The recompilation step will
|
|
|
|
only recompile changed files, or files for which the fasl files are
|
|
|
|
missing. It will also not recompile the C runtime code (the lisp
|
|
|
|
binary). If a (re)compilation of that code is needed, the genesis
|
|
|
|
step of the world build will inform you of that fact. In that case,
|
|
|
|
you'll have to use the rebuild-lisp.sh script, and then restart the
|
|
|
|
world build process with build-world.sh
|
|
|
|
|
|
|
|
`bin/rebuild-lisp.sh target-directory`::
|
|
|
|
This script will force a complete recompilation of the C runtime code
|
|
|
|
of CMU CL (aka the lisp executable). Doing this will necessitate
|
|
|
|
building a new kernel.core file, using build-world.sh.
|
|
|
|
|
|
|
|
`bin/load-world.sh target-directory version`::
|
|
|
|
This will finish the CMU CL rebuilding process, by loading the
|
|
|
|
remaining compiled files generated in the world build process into the
|
|
|
|
kernel.core file, that also resulted from that process, creating the
|
|
|
|
final lisp.core file.
|
|
|
|
|
|
|
|
You have to pass the version string as a second argument. The dumped
|
|
|
|
core will anounce itself using that string. Please don't use a string
|
|
|
|
consisting of an official release name only, (e.g. "18d"), since those
|
|
|
|
are reserved for official release builds. Including the build-date in
|
|
|
|
ISO8601 format is often a good idea, e.g. "18d+ 2002-05-06" for a
|
|
|
|
binary that is based on sources current on the 6th May, 2002, which is
|
|
|
|
post the 18d release.
|
|
|
|
|
|
|
|
`bin/build-utils.sh target-directory`::
|
|
|
|
This script will build auxiliary libraries packaged with CMU CL,
|
|
|
|
including CLX, CMUCL/Motif, the Motif debugger, inspector, and control
|
|
|
|
panel, and the Hemlock editor. It will use the lisp executable and
|
|
|
|
core of the given target.
|
|
|
|
|
|
|
|
`bin/make-dist.sh [-bg] [-G group] [-O owner] target-directory version arch os`::
|
|
|
|
This script creates both main and extra distribution tarballs from the
|
|
|
|
given target directory, using the make-main-dist.sh and
|
|
|
|
make-extra-dist.sh scripts. The result will be two tar files. One
|
|
|
|
contains the main distribution including the runtime and lisp.core
|
|
|
|
with PCL (CLOS); the second contains the extra libraries such as
|
|
|
|
Gray-streams, simple-streams, CLX, CLM, and Hemlock.
|
|
|
|
|
|
|
|
Some options that are available:
|
|
|
|
|
|
|
|
-b Use bzip2 compression
|
|
|
|
-g Use gzip compression
|
|
|
|
-G group Group to use
|
|
|
|
-O owner Owner to use
|
|
|
|
|
|
|
|
If you specify both -b and -g, you will get two sets of tarfiles. The
|
|
|
|
-G and -O options will attempt to set the owner and group of the files
|
|
|
|
when building the tarfiles. This way, when you extract the tarfiles,
|
|
|
|
the owner and group will be set as specified. You may need to be root
|
|
|
|
to do this because many Unix systems don't normally let you change the
|
|
|
|
owner and group of a file.
|
|
|
|
|
|
|
|
The remaining arguments used to create the name of the tarfiles. The
|
|
|
|
names will have the form:
|
|
|
|
```
|
|
|
|
cmucl-<version>-<arch>-<os>.tar.bz2
|
|
|
|
cmucl-<version>-<arch>-<os>.extras.tar.bz2
|
|
|
|
```
|
|
|
|
Of course, the "bz2" will be "gz" if you specified gzip compression
|
|
|
|
instead of bzip.
|
|
|
|
|
|
|
|
/bin/make-main-dist.sh target-directory version arch os::
|
|
|
|
This is script is not normally invoked by the user; make-dist will do
|
|
|
|
it appropriately.
|
|
|
|
|
|
|
|
This script creates a main distribution tarball (both in gzipped and
|
|
|
|
bzipped variants) from the given target directory. This will include
|
|
|
|
all the stuff that is normally included in official release tarballs
|
|
|
|
such as lisp.core and the PCL libraries, including Gray streams and
|
|
|
|
simple streams.
|
|
|
|
|
|
|
|
This is intended to be run from `make-dist.sh`.
|
|
|
|
|
|
|
|
`bin/make-extra-dist.sh target-directory version arch os`::
|
|
|
|
This is script is not normally invoked by the user; make-dist will do
|
|
|
|
it appropriately.
|
|
|
|
|
|
|
|
This script creates an extra distribution tarball (both in gzipped and
|
|
|
|
bzipped variants) from the given target directory. This will include
|
|
|
|
all the stuff that is normally included in official extra release
|
|
|
|
tarballs, i.e. the auxiliary libraries such as CLX, CLM, and Hemlock.
|
|
|
|
|
|
|
|
|
|
|
|
`cross-build-world.sh target-directory cross-directory cross-script [build-binary] [build-flags...]`::
|
|
|
|
This is a script that can be used instead of build-world.sh for
|
|
|
|
cross-compiling CMUCL. In addition to the arguments of build-world.sh
|
|
|
|
it takes two further required arguments: The name of a directory that
|
|
|
|
will contain the cross-compiler backend (the directory is created if
|
|
|
|
it doesn't exist, and must not be the same as the target-directory),
|
|
|
|
and the name of a Lisp cross-compilation script, which is responsible
|
|
|
|
for setting up, compiling, and loading the cross-compiler backend.
|
|
|
|
The latter argument is needed because each host/target combination of
|
|
|
|
platform's needs slightly different code to produce a working
|
|
|
|
cross-compiler.
|
|
|
|
|
|
|
|
We include a number of working examples of cross-compiler scripts in
|
|
|
|
the cross-scripts directory. You'll have to edit the features section
|
|
|
|
of the given scripts, to specify the features that should be removed
|
|
|
|
from the current set of features in the host lisp, and those that
|
|
|
|
should be added, so that the backend features are correct for the
|
|
|
|
intended target.
|
|
|
|
|
|
|
|
You can look at Eric Marsden's collection of build scripts for the
|
|
|
|
basis of more cross-compiler scripts.
|
|
|
|
|
|
|
|
## Step-by-Step Example of recompiling CMUCL for OpenBSD
|
|
|
|
|
|
|
|
Set up everything as described in the setup section above. Then
|
|
|
|
execute:
|
|
|
|
|
|
|
|
```
|
|
|
|
# Create a new target directory structure/config for OpenBSD:
|
|
|
|
bin/create-target.sh openbsd OpenBSD_gencgc OpenBSD
|
|
|
|
|
|
|
|
# edit openbsd/setenv.lisp to contain what we want:
|
|
|
|
cat <<EOF > openbsd/setenv.lisp
|
|
|
|
;;; Put code to massage *features* list here...
|
|
|
|
|
|
|
|
(in-package :user)
|
|
|
|
|
|
|
|
(pushnew :openbsd *features*)
|
|
|
|
(pushnew :bsd *features*)
|
|
|
|
(pushnew :i486 *features*)
|
|
|
|
(pushnew :mp *features*)
|
|
|
|
(pushnew :hash-new *features*)
|
|
|
|
(pushnew :random-mt19937 *features*)
|
|
|
|
(pushnew :conservative-float-type *features*)
|
|
|
|
(pushnew :gencgc *features*)
|
|
|
|
|
|
|
|
;;; Version tags
|
|
|
|
|
|
|
|
(pushnew :cmu18d *features*)
|
|
|
|
(pushnew :cmu18 *features*)
|
|
|
|
(setf *features* (remove :cmu17 *features*))
|
|
|
|
(setf *features* (remove :cmu18c *features*))
|
|
|
|
EOF
|
|
|
|
|
|
|
|
# Recompile the lisp world, and dump a new kernel.core:
|
|
|
|
bin/build-world.sh openbsd lisp # Or whatever you need to invoke your
|
|
|
|
|
|
|
|
# If build-world tells you (as it will the first time) that:
|
|
|
|
# "The C header file has changed. Be sure to re-compile the startup
|
|
|
|
# code."
|
|
|
|
# You 'll need to start rebuild-lisp.sh to do that, and then reinvoke
|
|
|
|
# build-world.sh:
|
|
|
|
|
|
|
|
# Recompile lisp binary itself:
|
|
|
|
bin/rebuild-lisp.sh openbsd
|
|
|
|
|
|
|
|
# Restart build-world.sh now:
|
|
|
|
bin/build-world.sh openbsd lisp
|
|
|
|
|
|
|
|
# Now we populate the kernel.core with further compiled files,
|
|
|
|
# and dump the final lisp.core file:
|
|
|
|
|
|
|
|
bin/load-world.sh openbsd "18d+ 2002-05-06"
|
|
|
|
|
|
|
|
# The second argument above is the version number that the built
|
|
|
|
# core will announce. Please always put the build-date and some
|
|
|
|
# other information in there, to make it possible to differentiate
|
|
|
|
# those builds from official builds, which only contain the release.
|
|
|
|
```
|
|
|
|
|
|
|
|
Now you should have a new lisp.core, which you can start with
|
|
|
|
```
|
|
|
|
./openbsd/lisp/lisp -core ./openbsd/lisp/lisp.core -noinit -nositeinit
|
|
|
|
```
|
|
|
|
|
|
|
|
## Compiling sources that contain disruptive changes
|
|
|
|
|
|
|
|
The above instructions should always work as-is for recompiling CMU CL
|
|
|
|
using matching binaries and source files. They also work quite often
|
|
|
|
when recompiling newer sources. However, every so often, some change
|
|
|
|
to the CMU CL sources necessitates some form of bootstrapping, so that
|
|
|
|
binaries built from earlier sources can compile the sources containing
|
|
|
|
that change. There are two forms of boostrapping that can be
|
|
|
|
required:
|
|
|
|
```
|
|
|
|
#!html
|
|
|
|
<ol style="list-style-type:lower-alpha">
|
|
|
|
<li> Bootfiles
|
|
|
|
<p>The maintainers try to make bootfiles available, that allow going
|
|
|
|
from an old release to the next release. These are located in the
|
|
|
|
src/bootfiles/<old-release>/ directory of the CMU CL sources.</p>
|
|
|
|
|
|
|
|
<p>I.e. if you have binaries that match release 18d, then you'll need
|
|
|
|
to use all the bootfiles in src/bootfiles/18d/ in order to go to
|
|
|
|
the next release (or current sources, if no release has been made
|
|
|
|
yet). If you already used some of the bootstrap files to compile
|
|
|
|
your current lisp, you obviously don't need to use those to get to
|
|
|
|
later versions.</p>
|
|
|
|
|
|
|
|
<p>You can use the bootfiles by concatenating them into a file called
|
|
|
|
bootstrap.lisp in the target directory (i.e. target:bootstrap.lisp)
|
|
|
|
in the order they are numbered. Be sure to remove the bootstrap
|
|
|
|
file once it is no longer needed.</p>
|
|
|
|
|
|
|
|
<p>Alternatively, the bootstrap file can just "load" the individual
|
|
|
|
bootfiles as needed.</p>
|
|
|
|
</li>
|
|
|
|
<li> Cross-compiling
|
|
|
|
<p>Under some circumstances, bootstrap code will not be sufficient,
|
|
|
|
and a cross-compilation is needed. In that case you will have to
|
|
|
|
use cross-build-world.sh, instead of build-world.sh. Please read
|
|
|
|
the instructions of that script for details of the more complex
|
|
|
|
procedure.</p>
|
|
|
|
|
|
|
|
<p><< This isn't really true anymore, and we should place a more
|
|
|
|
|
|
|
|
<p>When cross-compiling, there are two sorts of bootscripts that can be
|
|
|
|
used: Those that want to be executed prior to compiling and loading
|
|
|
|
the cross-compiler, which should be placed in the file called
|
|
|
|
target:cross-bootstrap.lisp, and those that should happen after the
|
|
|
|
cross-compiler has been compiled and loaded, just prior to compiling
|
|
|
|
the target, which should be placed in target:bootstrap.lisp, just
|
|
|
|
like when doing a normal recompile.</p>
|
|
|
|
|
|
|
|
<p>Additionally, sometimes customized cross-compiler setup scripts
|
|
|
|
(to be used in place of e.g. cross-x86-x86.lisp) are required,
|
|
|
|
which are also placed in one of the bootfiles/*/* files. In those
|
|
|
|
cases follow the instructions provided in that file, possibly merging
|
|
|
|
the changed contents thereof with your normal cross-script.</p>
|
|
|
|
</li>
|
|
|
|
</ol>
|
|
|
|
```
|
|
|
|
## Step-by-Step Example of Cross-Compiling
|
|
|
|
|
|
|
|
This gives a step-by-step example of cross-compiling a sparc-v8 build
|
|
|
|
using a sparc-v9 build. (For some unknown reason, you can't just
|
|
|
|
remove the :sparc-v9 feature and add :sparc-v8.)
|
|
|
|
|
|
|
|
So, first get a recent sparc-v9 build. It's best to get a version
|
|
|
|
that is up-to-date with the sources. Otherwise, you may also need to
|
|
|
|
add a bootstrap file to get any bootfiles to make your lisp
|
|
|
|
up-to-date with the current sources.
|
|
|
|
```
|
|
|
|
#!html
|
|
|
|
<ol>
|
|
|
|
<li> Select a directory for the cross-compiler and compiled target:
|
|
|
|
|
|
|
|
<p>Create a cross-compiler directory to hold the cross-compiler
|
|
|
|
and a target directory to hold the result:</p>
|
|
|
|
<pre>
|
|
|
|
bin/create-target.sh xcross
|
|
|
|
bin/create-target.sh xtarget
|
|
|
|
</pre>
|
|
|
|
</li>
|
|
|
|
<li> Adjust cross-compilation script
|
|
|
|
|
|
|
|
<p>Copy the src/tools/cross-scripts/cross-sparc-sparc.lisp to
|
|
|
|
xtarget/cross.lisp. Edit it appropriately. In this case, it
|
|
|
|
should look something like:</p>
|
|
|
|
<pre>
|
|
|
|
(c::new-backend "SPARC"
|
|
|
|
;; Features to add here
|
|
|
|
'(:sparc :sparc-v8
|
|
|
|
:complex-fp-vops
|
|
|
|
:linkage-table
|
|
|
|
:gencgc
|
|
|
|
:stack-checking
|
|
|
|
:relative-package-names
|
|
|
|
:conservative-float-type
|
|
|
|
:hash-new :random-mt19937
|
|
|
|
:cmu :cmu19 :cmu19a
|
|
|
|
)
|
|
|
|
;; Features to remove from current *features* here
|
|
|
|
'(:sparc-v9 :sparc-v7 :x86 :x86-bootstrap :alpha :osf1 :mips
|
|
|
|
:propagate-fun-type :propagate-float-type :constrain-float-type
|
|
|
|
:openbsd :freebsd :glibc2 :linux :pentium
|
|
|
|
:long-float :new-random :small))
|
|
|
|
|
|
|
|
(setf *features* (remove :sparc-v9 *features*))
|
|
|
|
(pushnew :sparc-v8 *features*)
|
|
|
|
</pre>
|
|
|
|
It's important to add frob *features* here as well as in the
|
|
|
|
new-backend. If you don't adjust *features*, they won't be
|
|
|
|
set appropriately in the result.
|
|
|
|
</li>
|
|
|
|
<li> Build the cross compiler and target
|
|
|
|
<p>Now compile the result:</p>
|
|
|
|
<pre>
|
|
|
|
bin/cross-build-world.sh xtarget xcross xtarget/cross.lisp [v9 binary]
|
|
|
|
</pre>
|
|
|
|
|
|
|
|
<li> Rebuild the lisp files:
|
|
|
|
|
|
|
|
<p>When this finishes, you need to compile the C code:</p>
|
|
|
|
<pre>
|
|
|
|
bin/rebuild-lisp.sh xtarget
|
|
|
|
</pre>
|
|
|
|
<p>At this point, you may want to run cross-build-world.sh again
|
|
|
|
to generate a new kernel.core. It shouldn't build anything;
|
|
|
|
just loads everything and creates a kernel.core.</p>
|
|
|
|
|
|
|
|
<li> Build the world:
|
|
|
|
|
|
|
|
<p>With the new kernel.core, we need to create a lisp.core:</p>
|
|
|
|
<pre>
|
|
|
|
bin/load-world.sh xtarget "new lisp"
|
|
|
|
</pre>
|
|
|
|
<p>Test the result with</p>
|
|
|
|
<pre>
|
|
|
|
xtarget/lisp/lisp -noinit
|
|
|
|
</pre>
|
|
|
|
```
|
|
|
|
However, this lisp will be missing some functionality like PCL. You
|
|
|
|
probably now want to use the compiler to rebuild everything once
|
|
|
|
again. Just follow the directions for a normal build, and use
|
|
|
|
xtarget/lisp/lisp as your compiler. Be sure to use create-target.sh
|
|
|
|
to create a new directory where the result can go.
|
|
|
|
|
|
|
|
## Cross-Platform Cross-Compile
|
|
|
|
|
|
|
|
A cross-platform cross-compile is very similar to a normal
|
|
|
|
cross-compile, and the basic steps are the same. For the sake of
|
|
|
|
concreteness, assume we are on ppc/darwin and want to cross-compile
|
|
|
|
to x86/linux.
|
|
|
|
|
|
|
|
To simplify things, we assume that both platforms have access to the
|
|
|
|
same file system, via NFS or something else.
|
|
|
|
```
|
|
|
|
#!html
|
|
|
|
<ol>
|
|
|
|
<li> As above, we need to create directories for the cross-compiler and
|
|
|
|
compiled target. We assume we are on ppc/darwin. So, when running
|
|
|
|
create-target.sh we need to specify the target:
|
|
|
|
<pre>
|
|
|
|
</pre>
|
|
|
|
<li> Adjust the cross-compilation script. An example for ppc/darwin to
|
|
|
|
x86/linux is in src/tools/cross-scripts/cross-ppc-x86.lisp.
|
|
|
|
|
|
|
|
<li> Build the cross compiler and target, as above, using the specified
|
|
|
|
cross-compile script:
|
|
|
|
<pre>
|
|
|
|
</pre>
|
|
|
|
where cross.lisp is the cross-compile script from 2) above.
|
|
|
|
|
|
|
|
<li> Everything has now been compiled for the x86/linux target. We need
|
|
|
|
to compile the C code for x86 and create a lisp.core from the
|
|
|
|
kernel.core. This is where it's useful to have both platforms be
|
|
|
|
able to access the same file system. If not, you will need to copy
|
|
|
|
all of the generated files from ppc/darwin to x86/linux. Basically
|
|
|
|
everything in xtarget needs to be copied.
|
|
|
|
|
|
|
|
<p>Note carefully that you may have to edit lisp/internals.h and/or
|
|
|
|
lisp/internals.inc to have the correct features. This is a known
|
|
|
|
bug in the generation of these files during cross-compilation.</p>
|
|
|
|
|
|
|
|
Compile the lisp code:
|
|
|
|
<pre>
|
|
|
|
</pre>
|
|
|
|
<li> Now run load-world.sh to create the desired lisp.core from lisp and
|
|
|
|
kernel.core. As above, PCL has not been compiled, so select
|
|
|
|
restart 3 (return nil from pclload) to create lisp.core
|
|
|
|
<pre>
|
|
|
|
</pre>
|
|
|
|
</ol>
|
|
|
|
```
|
|
|
|
At this point, you will have a shiny new lisp on the new platform.
|
|
|
|
Since it's missing PCL, you will need to do at least one normal build
|
|
|
|
to get PCL included. This is also a good check to see if everything
|
|
|
|
was compiled properly. A full set of builds via build.sh might be
|
|
|
|
good at this point too.
|
|
|
|
|
|
|
|
Some of the details for each command may have changed; You can get
|
|
|
|
help for each command by using the -h argument.
|
|
|
|
|
|
|
|
In particular steps 3, 4, and 5 can be combined into one by using the
|
|
|
|
-c, -r, and -l options for cross-build-world.sh. The -c option cleans
|
|
|
|
out the targe and cross directories; -r does step 4; and -l does step
|
|
|
|
5. |