Software installation specification from precompiled binaries
Benoît Papillault, 2004-11-23
Goals
Software installation in the Source Mage GNU/Linux distribution is
done by using the cast command which download and compile
software from its source code. This option is not realistic when doing
installation CD-ROM. Thus we will analyze what cast does
to try to reproduce it by using files present under
/var/cache/sorcery .
Differents types of spells
There exist currently two types of spells : BUILD_API=1
and BUILD_API=2 . This is a shell variable indicating
spell structure. It can be defined at 3 locations (we take the glibc
spell from the stable grimoire as an example here):
-
At the grimoire level: /var/lib/sorcery/codex/stable/API_VERSION
-
At the section level: /var/lib/sorcery/codex/stable/libs/API_VERSION
-
At the spell level: /var/lib/sorcery/codex/stable/libs/glibc/DETAILS
Absence of this variable is treated like BUILD_API=1 .
What cast does (BUILD_API=1)
We will detail only what is relevant for the next paragraphs. All the
compilation is watched by the installwatch tool, next the
POST_INSTALL file is executed if it exists.
Once this is done, the binary cache in /var/cache/sorcery
as well as the md5sum file is built. It means that the
POST_INSTALL must not modify those files. Otherwise, the
md5sum checking done by cleanse will fail.
What cast does (BUILD_API=2)
The process is identical, except that the POST_INSTALL
script is replaced by the FINAL script.
Installation from precompiled binaries
The installation is done in 4 steps:
-
dependencies installation using the same process recursively. Thos
dependencies are listed in the
DEPENDS file from the
grimoire with the depends and
optional_depends functions.
Warning: sorcery does not allow for optional dependencies
tracking. For instnance, openssl is an optional
dependency of irssi . If you compiled
irssi and selected openssl , the
resulting binary will work only if openssl is
installed. So, if you later install irssi from the
binary cache, sorcery will not install
openssl . However, those dependencies (optional or
not) are recorded in /var/state/sorcery/depends .
-
binary file extraction from the archive in
/var/cache/sorcery . This archive could be downloaded
from local support (existing system, mounted partition, CD-ROM,
nfs mounts) or network support (http, ftp, ...).
When using a network suport, we need a way to get the list of
available archives. The easiest way is to create a file
ls containing the result of the ls
command.
-
updating sorcery files:
/var/state/sorcery/packages
/var/state/sorcery/depends
/var/cache/sorcery/$SPELL-$VERSION-$HOST.tar.bz2
(optional)
-
launching a post-installation script: POST_INSTALL for BUILD_API=1
and FINAL for BUILD_API=2. This script, as well as the BUILD_API
determination, depends on the grimoire. Thus, there is a potential
risk if the grimoire does not match the grimoire used at compile
time.
File format and location
The various files needed to install software from precompiled binaries
may exist in various forms:
-
on an installed system: (what the resurect functionality of
cast does approximately)
-
installed version:
/var/state/sorcery/packages
-
dependencies:
/var/state/sorcery/depends
-
precompiled binaries (optionel):
/var/cache/sorcery/${SPELL}-${VERSION}-${HOST}.tar.bz2
file list:
/var/log/sorcery/install/${SPELL}-${VERSION}
-
post-installation script:
-
POST_INSTALL if BUILD_API=1
-
FINAL if BUILD_API=2
-
in a remote (ftp/http/rsync/cvs/scp) or local (partition, nfs)
system. This system will be made of a unique directory containing
all the precompiled binaries as well as a
ls file. It
will be symbolicaly accessible through the URL ${BASE}
which will be used for the description.
-
precompiled binaries list in the
${BASE}/ls file.
-
${BASE}/${SPELL}_${VERSION}_${HOST}.tar.bz2 containing:
- the list of files to install, for instance:
usr/bin/which
usr/share/info/which.info
usr/share/man/man1/which.1
var/log/sorcery/compile/which-2.16.bz2
var/log/sorcery/md5sum/which-2.16
var/log/sorcery/install/which-2.16
install/depends will be created from the
matching line in /var/state/sorcery/depends . The
whole line must be copied to be later restored in the system to
install. For instance:
irssi:glib2:on:required::
irssi:ncurses:on:required::
irssi:openssl:on:optional::--disable-ssl
irssi:perl:off:optional:--with-perl=module:--with-perl=no
install/postinst is a copy of
POST_INSTALL or FINAL .
-
on a ISO system. This is an hybrid system where we try to gain disk
space. We will find software in both versions: either already
installed, either in tar.bz2 form in the directory
/install .
Tools
In order to use the system presented here, we need two tools: a tool
to create tar.bz2 file containing an install directory
(spell-binary-make) and a tool to install them (spell-binary-install).
spell-binary-make
-
Usage: [BASE=/install] [INSTALL_ROOT=/mnt/root] spell-binary-make spell
-
Requirement: the spell must be already installed, as well as all its
dependencies.
-
Output: a file ${SPELL}_${VERSION}_${HOST}.tar.bz2 in the ${BASE}
directory for the spell and all its dependencies.
-
Note: the spell name can be modified in some cases. The caracter _
is used as delimitor and thus caracters _ are replaced by - in spell
names.
-
BASE is the output directory containing ${SPELL}_${VERSION}_${HOST}.tar.bz2
and INSTALL_ROOT is the source system's root directory. Default values:
BASE=. et INSTALL_ROOT=/
spell-binary-install
-
Usage: [BASE=/install] [INSTALL_ROOT=/mnt/root] spell-binary-install spell
-
BASE is the input directory containing
${SPELL}_${VERSION}_${HOST}.tar.bz2 (it can be a URL) and
INSTALL_ROOT is the destination system's root directory. Default
values: BASE=. et INSTALL_ROOT=/
-
If no ${SPELL}_${VERSION}_${HOST}.tar.bz2 file exist, an
installation is attempted from the running system. This mode is used
by the ISO.
Implementation
Both tools are implemented by bash shell scripts and can
be downloaded:
This implementation has some restrictions:
- It is now possible to make a package with a spell that has no
cache file
- The use of
spell-binary-make with something
different than INSTALL_ROOT=/ can lead to erroneous behaviour
- The use of
spell-binary-install with an URL using
http/ftp is possible in this implementation
- The use of
spell-binary-install with something
different than INSTALL_ROOT=/ can produce bug in the
post-installation script
- Alien files are not installed
Possible use
ISO size reduction
This system avoids to have precompiled binaries if a spell is already
installed on the ISO itself.
Example:
BASE=/install INSTALL_ROOT=/mnt/root spell-binary-install irssi
Installation from another support or network location
This system allows what is commonly called netinstall and
will allow to create an ISO without any precompiled binaries. This can
also be used to update precompiled binaries without the need for
releasing a new ISO, thus reducing the cost of updates and
maintainance.
Example:
BASE=http://download.sourcemage.dtdm.net/install INSTALL=/mnt/root spell-binary-install ircssi
In this example, the openssl dependency will be installed
by using the same parameters, and thus will use the same base URL.
Common errors
-
POST_INSTALL or FINAL scripts must not use
source files present in ${SOURCE_DIRECTORY} . This
should be done in BUILD ou INSTALL
scripts.
-
POST_INSTALL or FINAL scripts must not
modify files installed by BUILD or INSTALL
scripts. This should be done in BUILD or
INSTALL scripts themselves.
-
POST_INSTALL or FINAL scripts must
correctly use the ${INSTALL_ROOT} variable. This later
is the system's root directory and is normally set to
/ . However, in the case of installing from a CD-ROM, it
will be set to /mnt/root .
Links
|