Friday 23 July 2010

LAP - Actual Build - Part 5, Sed & Utils

Now we have Glibc, Binutils and GCC finally compiled and working we can get on with the installation of the rest of the system.  Let's start with … ah.  Sed.

tar -xjvf /sources/sed-4.2.1.tar.bz2 
cd sed-4.2.1
./configure --prefix=/usr --bindir=/bin --htmldir=/usr/share/doc/sed-4.2.1

The [htmldir] just specifies where the documentation should be installed to.

make $CORES_TO_USE
make html
make check
====================================================== 
All 65 tests behaved as expected (4 expected failures) 
====================================================== 

The test report is comforting.  Last thing we want is something going wrong with something as dangerous and potentially lethal as sed.

make install
make -C doc install-html

The last command is just a funnily expressed way of installing the documentation.

cd ..
rm -rvf sed-4.2.1

The next package is pkg-config.  This is a one trick pony bit of software that reports what libraries you have installed for the purposes of other packages that you are compiling.

tar -xzvf /sources/pkg-config-0.23.tar.gz
cd pkg-config-0.23
./configure --prefix=/usr
make $CORES_TO_USE
make check

================== 
All 6 tests passed 
==================

Well, that's good news.

make install
cd ..
rm -rvf pkg-config-0.23

Next up is ncurses, which was part of the Toolchain.

tar -xzvf /sources/ncurses-5.7.tar.gz 
cd ncurses-5.7
./configure --prefix=/usr --with-shared --without-debug --enable-widec

This time, we do want the ada compiler so we do not disable it.  The [enable-widec] installs what are called wide character libraries.  Some how I know that these are not just fat letters, but I cannot shake the impression.  No idea what we use chubby characters for.

make $CORES_TO_USE
make install

The installation defaults to the /usr/lib directory, but we want out libraries to be in the /lib folder.  So we move the libraries and correct a symbolic link:

mv -v /usr/lib/libncursesw.so.5* /lib
ln -sfv ../../lib/libncursesw.so.5 /usr/lib/libncursesw.so

As we said, we have installed fat letters.  Some programs expect to find thin letters.  We fool those programs by the following, complicated looking, means:

for lib in ncurses form panel menu ; do rm -vf /usr/lib/lib${lib}.so ; echo "INPUT(-l${lib}w)" >/usr/lib/lib${lib}.so ; ln -sfv lib${lib}w.a /usr/lib/lib${lib}.a ; done
ln -sfv libncurses++w.a /usr/lib/libncurses++.a

Frankly, it would actually be easier to just install the thin letters as well, but what do I know.  The first part of that replaces various files (with ncurses, form, panel and menu in the filenames) with very similar files but with a [w] added to the filenames referred to.  It then sets up symbolic links to those replacement files.  The second command just makes an additional symbolic link - you'll see the [w] added to the filename.

rm -vf /usr/lib/libcursesw.so
echo "INPUT(-lncursesw)" >/usr/lib/libcursesw.so
ln -sfv libncurses.so /usr/lib/libcurses.so
ln -sfv libncursesw.a /usr/lib/libcursesw.a
ln -sfv libncurses.a /usr/lib/libcurses.a

OK.  Getting strange now.  We first delete the libcursesw.so file that we JUST made using the last command.  Ah.  No we don't.  The previous one was libncurses, but this is just libcurses.  My bad (eyesight).  So it really just does the same kind of thing as the last commands, but for curses instead of ncurses.  Presumably ncurses is an updated or replacement version of curses.  So if a program goes looking for curses, and we fool it into using ncurses instead, it shouldn't break.  We now just install the documentation:

mkdir -v       /usr/share/doc/ncurses-5.7
cp -v -R doc/* /usr/share/doc/ncurses-5.7

Now, ordinarily, we would just clean up and move on.  However, brilliantly the book points out at this point that we CAN actually install the thin letters if we want to.  Now, every package that we are going to build from source will use the fat letters, but the book warns that we may, in theory, come across some bit of software that wants thin letters.  Let's just install this to be on the safe side.

make distclean

This cleans up the source folder to put it back before the last configure script ran.

./configure --prefix=/usr --with-shared --without-normal --without-debug --without-cxx-binding

The [without-cxx-binding] stops it from building a c++ binding and demo.  Whatever that means.  The configure script [--help] does not even know what [--without-normal] means.  So I am at a complete fucking loss.

make $CORES_TO_USE sources libs

I think that that command only builds from the code that it finds in the sources and libs directories.  Presumably this is on the basis that we have already installed the rest of the software already.

cp -av lib/lib*.so.5* /usr/lib

And tidy.

cd ..
rm -rvf ncurses-5.7

A number of the packages we are going to install are collections of useful/essential utilities grouped in rough themes.  The next package we are going to install is a set of general utilities called Util-linux-ng.  Other ones are binutils, e2fsprogs and coreutils packages.

tar -xjvf /sources/util-linux-ng-2.17.tar.bz2
cd util-linux-ng-2.17

Earlier on I mentioned the Filesystem Hierarchy Standards.  These state that the adjtime program should be in a different directory than this package defaults to.  You just know this means sed is going to be unchained don't you?

sed -e 's@etc/adjtime@var/lib/hwclock/adjtime@g' -i $(grep -rl '/etc/adjtime' .)
mkdir -pv /var/lib/hwclock

You were right.

./configure --enable-arch --enable-partx --enable-write

The three options enable some optional programs to be installed.

make $CORES_TO_USE
make install
cd ..
rm -rvf util-linux-ng-2.17

The E2fsprogs package contains utilities for dealing with the Linux standard disk formats ext2, ext3 and ext4.  Things like format, resize, check and so on.

tar -xzvf /sources/e2fsprogs-1.41.10.tar.gz
cd e2fsprogs-1.41.10
mkdir -v build
cd build
../configure --prefix=/usr --with-root-prefix="" --enable-elf-shlibs --disable-libblkid --disable-libuuid --disable-uuidd --disable-fsck

The [with-root-prefix=""] tells it not to install essential programs into /usr just in case we want to mount that from a separate partition.  [enable-elf-shlibs] is more point ear shit, or shared libraries that other programs may want to use.  The programs which are [disable]d have already been installed from the Util-Linux-NG package.

make $CORES_TO_USE
make $CORES_TO_USE check

105 tests succeeded    0 tests failed 

Good stuff.

make install
make install-libs

The second command installs static libraries which are not installed by the classic command.  We need to write enable all the stuff we just installed so we can remove the debugging symbols in due course.

chmod -v u+w /usr/lib/{libcom_err,libe2p,libext2fs,libss}.a

For some reason the standard installation does not properly tell the system that it has been installed.  This seems to be handled by a file called [dir] in the [usr/share/info] folder.  Apparently programs come with .info files which includes information that is then added to the [dir] file.  With this program, the .info is compressed.  So we need to uncompress it, and then add the .info information to the [dir] file like so:

gunzip -v /usr/share/info/libext2fs.info.gz
install-info --dir-file=/usr/share/info/dir /usr/share/info/libext2fs.info

We then install the documentation manually and clean up:

makeinfo -o      doc/com_err.info ../lib/et/com_err.texinfo
install -v -m644 doc/com_err.info /usr/share/info
install-info --dir-file=/usr/share/info/dir /usr/share/info/com_err.info
cd ../../
rm -rvf e2fsprogs-1.41.10

The next package completes our trio of utility packages.  It is coreutils and includes stuff like 'echo', 'ls', 'ln' and 'mkdir'.

tar -xzvf /sources/coreutils-8.4.tar.gz
cd coreutils-8.4

We first of all need to run a patch file to fix a bug with the 'uname' program that is in the coretuils package.

case `uname -m` in
 i?86 | x86_64) patch -Np1 -i /sources/coreutils-8.4-uname-1.patch ;;
esac

This gave me the following messages:

patching file src/uname.c
Hunk #2 succeeded at 314 with fuzz 2.
Hunk #3 succeeded at 441 with fuzz 1.
Hunk #4 succeeded at 449 with fuzz 2.

We need another patch now because, according to the book, "POSIX requires that programs from Coreutils recognize character boundaries correctly even in multibyte locales.".  Right.  I presume the multibyte locales aren't some sort of neighbourhood restaurant.  I know what nearly every word in that sentence means, but when they are added together, forget it.  Not a clue.  Let's just apply the patch.  Something to do with localisation nonsense probably.

patch -Np1 -i /sources/coreutils-8.4-i18n-1.patch

I won't repeat the output, because it was quite detailed, but it seemed to work without an error.

./configure --prefix=/usr --enable-no-install-program=kill,uptime

The 'kill' and 'uptime' programs will be installed by other packages in due course, so we tell it not to [install] them now.

make $CORES_TO_USE

Now lets run the tests:

make che.....  What the fu....?  OK, this is going to be painful.  First of all we do the tests that are designed to be run as the root user:

make NON_ROOT_USERNAME=nobody check-root

======================
All 14 tests passed
(7 tests were not run)
======================

Well, that looks good.  We now need to run some tests as a non-root user.  This is presumably to check that some privileges bullshit works properly in pissing me off.  We do not want to use any of the users currently in /etc/group so we will have to add another one:

echo "dummy:x:1000:nobody" >> /etc/group

If you remember, the [>>] bit means that the bit in quotes is added to the end of /etc/group.  This has the effect of adding another group called [dummy], giving it the group number [1000], and adding the user [nobody] to it.  We then make all the files in the coreutils folder belong to [nobody] so that user can run the checks.

chown -Rv nobody .

Watch out for the [.] at the end of that command - that actually is the thing that will be changed.  Here it means the existing diretory, and the [R] option means include all of the subdirectories and their contents as well. 

Now we run the tests.  The actual tests are the same ones we ran in the Toolchain install [make RUN_EXPENSIVE_TESTS=yes check] but we have to wrap them in a new shell started as the nobody user.  Remember in the Toolchain install of coreutils we copied the su program as 'su-tools'?  Now it is time to use it:

su-tools nobody -s /bin/bash -c "make $CORES_TO_USE RUN_EXPENSIVE_TESTS=yes check"

The [s] option specifies the shell to use.  Remember that [/bin/bash] is still a symbolic link to /tools/bin/bash.  The [c] option tells the new shell to run the command in quotes.  After all that the result was:

====================== 
All 198 tests passed 
(4 tests were not run) 
====================== 

Which is good.  Now we just need to get rid of the temporary [nobody] user.  To do we will use ... ah ... sed:

sed -i '/dummy/d' /etc/group

Actually, that's not bad.  Its not the usual level of insanity anyway.  It also manages to achieve the goal of removing the line in question from the [/etc/group] file, but I have no idea how.

make install

A lot of the default install locations are non-fhs compliant, so we will just move them around:

mv -v /usr/bin/{cat,chgrp,chmod,chown,cp,date,dd,df,echo} /bin
mv -v /usr/bin/{false,ln,ls,mkdir,mknod,mv,pwd,rm} /bin
mv -v /usr/bin/{rmdir,stty,sync,true,uname} /bin
mv -v /usr/bin/chroot /usr/sbin

We need to make one more change of location just in case we are using a separate /usr partition:

mv -v /usr/bin/{head,sleep,nice} /bin
cd ..
rm -rvf coreutils-8.4

No comments:

Post a Comment