Thursday 22 July 2010

LAP - Actual Build - Part 4, Binutils & GCC

OK, having done that housekeeping stuff we are back to building the final packages.  The first in this batch is zlib which contains compression and decompressions tools that other programs are going to need to rely on.

tar -xjvf /sources/zlib-1.2.3.tar.bz2
cd zlib-1.2.3
./configure --prefix=/usr --shared --libdir=/lib

We are telling it that it needs to put its libraries in the directory /lib.  I presume that's it anyway, the book is helpfully silent.  The [./configure --help] command is no better.  It points out that

[--libdir=LIBDIR]

Brilliant.  That's cleared that mystery up, thanks Columbo.

make $CORES_TO_USE
make check

The entire test output is worth repeating here:

hello world 
zlib version 1.2.3 = 0x1230, compile flags = 0x55 
uncompress(): hello, hello! 
gzread(): hello, hello! 
gzgets() after gzseek:  hello! 
inflate(): hello, hello! 
large_inflate(): OK 
after inflateSync(): hello, hello! 
inflate with dictionary: hello, hello! 
        *** zlib test OK *** 

That's just pleasant to look at.  Well done Mr Zlib programmer.  Its a fuck of a lot better than screeds and screeds of unreadable crap followed by 'Error'.  GCC, I am looking at you.

make install
rm -v /lib/libz.so
ln -sfv ../../lib/libz.so.1.2.3 /usr/lib/libz.so

Not entirely sure what is going on here.  The output of the last command was:

`/usr/lib/libz.so' -> `../../lib/libz.so.1.2.3'

What I think is going on here is that zlib has installed part of itself to [/lib/libz.so].  The LFS system says that file should be in [/usr/lib] so we have deleted the symbolic link and made a new one to the correct location. Why we have to have the [../..] nonsense before the path is beyond me.

We now have to make a static library.  In my experience all libraries are static apart from the yellow community ones on wheels that drive to old folks homes.  What do I know though?  Well certainly now what a fucking static library is for a start.  To do this, we have to clean up the source folder:

make clean

And then run a new configure:
./configure --prefix=/usr

Hmmm.  We have omitted the [--shared] and [libdir] stuff.  Presumably that relates to non-static libraries.

make $CORES_TO_USE
make check

All OK again.

make install

Finally we need to do some permissions bollocks:

chmod -v 644 /usr/lib/libz.a
cd ..
rm -rvf zlib-1.2.3

Next up is the final Bintuils.  We did not need the zlib to be installed before it to make the toolchain, so my guess would be that zlib is only needed for the test suite.

tar -jxvf /sources/binutils-2.20.tar.bz2
cd binutils-2.20

First the book tells us to test to make sure the PTYs are working by running:

expect -c "spawn ls"

The only time I expect spawn is when I have a pool of copulating frogs, and I have no idea what PTYs are.  Still best do what one is told.  The output is:

spawn ls 

Well that is what I was told to look for, so I guess we are OK to go ahead.  Apparently bintutils installs an out of date file, which a later package is going to install properly, so we will prevent it from working now:

rm -fv etc/standards.info
sed -i.bak '/^INFO/s/standards.info //' etc/Makefile.in

shhh, it's sed, just pretend like we haven't seen it and it might not notice us...

mkdir -v ../binutils-build
cd ../binutils-build
../binutils-2.20/configure --prefix=/usr --enable-shared

This time around the configure command looks much simpler.  We have got rid of the [disable] stuff, and we are enabling [shared] – whatever that is.  Strangely, we have to put the install info into the [make] command:

make $CORES_TO_USE tooldir=/usr

You should not miss the check this time.  Really.  They even write the warning in RED in the book, so they must be serious.

make $CORES_TO_USE check

# of expected passes        558 
# of expected failures        8 

That looks fairly promising.  The install command also needs the install path specified:

make tooldir=/usr install

Lastly, we need to install a particular library that some stuff needs but otherwise would not be installed:

cp -v ../binutils-2.20/include/libiberty.h /usr/include
cd ..
rm -rvf binutils-2.20 binutils-build

The next program is GMP, which is one of the hardcore maths programs that we had to install into the GCC source directory to get it to build in the Toolchain install.  When we are building the final GCC as opposed to the temporary one, we need to install both this and the MPFR packages properly, first.

tar -xjvf /sources/gmp-5.0.0.tar.bz2
cd gmp-5.0.0
./configure --prefix=/usr --enable-cxx --enable-mpbsd

[enable-cxx] apparently tells the compiler to compile this [in/with compatibility for] the c++ language.  The [enable-mpbsd] tells the compiler to build the Berkeley MP compatibility library.  Ah, well, that's clear then.  Oh, just one question... What the fuck is the Berkeley MP compatibility library?  No idea.

make $CORES_TO_USE
make $CORES_TO_USE check 2>&1 | tee gmp-check-log
awk '/tests passed/{total+=$2} ; END{print total}' gmp-check-log

At first blush this seems to be as scary as a typical 'sed' command, but it actually makes a bit of sense.  Firstly, [awk] is a [what is awk?] and what it is doing is searching through the log file that the previous command created.  It then [print]s the [total] amount of times the words [tests passed] are mentioned.  It should be 162.  It is:

162

Woot.

make install
mkdir -v /usr/share/doc/gmp-5.0.0
cp    -v doc/{isa_abi_headache,configuration} doc/*.html /usr/share/doc/gmp-5.0.0

This manually copies the documentation for the software to the appropriate location.
cd ..
rm -rvf  gmp-5.0.0

The other hardcore maths package is MPFR.  I still have only a very vague sense of what this does.  By way of example, the first entry on Wikipedia's list of this programs features is that it supports 'signed zeros'.  Yes, that's right; it means as in +0 or -0.  I have done maths.  I got an award in my school for being best at maths.  I've read Simon Singh on Fermat's Last Theorem.  I have even downloaded and tried to read Andrew Wiles solution to said last theorem (effect was much the same as trying to manually check the fucking SPECS file).  In all of that I have never, never, heard of 0 being positive or negative.  This, therefore, has the sniff of sweaty male undercarriage.

Still GCC needs it, and is going to violently eject the playthings from the perambulator unless we install it, so here we go:

tar -xjvf /sources/mpfr-2.4.2.tar.bz2
cd mpfr-2.4.2
./configure --prefix=/usr --enable-thread-safe
make $CORES_TO_USE
make $CORES_TO_USE check

==================== 
All 148 tests passed 
==================== 

Now, that result is not bad either.  You do get screeds and screeds of crap, but at least the final message is a warm and comforting - “Hey, you know what?  It all works.”

make install
make html

That command builds the manual files for the software - presumably in HTML format.  We need to install them manually, once we have made a directory for them in [/usr/share/doc]:

mkdir -pv /usr/share/doc/mpfr-2.4.2
find . -name \*.html -type f -exec cp -v \{} /usr/share/doc/mpfr-2.4.2 \;

This is a bit trickier than the [cp] command used for GMP.  The [find] command does two things.  First it searches for all [html] files that have been created in the current directory tree, and then it [c]o[p]ies them to the directory we just made.  Instead of just finding stuff the find command needs to be told what to do when it finds it.  To just display the results of the find you would add [-print] instead of the whole bit including and after [-exec] here.

cd ..
rm -rvf mpfr-2.4.2

The next package is File.  This is a small utility which takes as an input a file, and tries to tell you what kind of file it is.

tar -xzvf /sources/file-5.04.tar.gz
cd file-5.04
./configure --prefix=/usr
make $CORES_TO_USE
make $CORES_TO_USE check

No obvious errors appeared.

make install
cd ..
rm -rvf file-5.04

OK.  Deep breath time.  We are now going to compile our final version of GCC. I have been having problems compiling this in [/tmp]. I suspect it is just running out of RAM. My 2Gb machine says it has around 200Mb left available, but the [make] step ends with errors, and the [make check] does not complete properly either. To test further, I loaded the Keys up on my 4Gb machine, and it compiled fine from [/tmp] with the usual muddy flap errors. I am therefore putting this down to insufficient memory. I did manage to get it to compile in /tmp, but I had to close Firefox to free up a few hundred Mb. Remember to use a separate terminal window to run:
~/repeat.sh 60 ~/clearc.sh
Despite my best efforts at cache clearing, the make ... check test just will not run from Ramdisk on only 2Gb of memory. It exhausts all the available memory and crashes the live CD. Not much fun. The safest option is to start this install from ...
cd /sources
... although at the end you can return to [/tmp] to build the next packages. From perusing the book this does appear to be the single largest install we are going to do. A halfway house here could be to unpack the source code to /sources/ but to create the build directory on /tmp/. I leave the appropriate mechanism as a homework exercise to the dedicated student.

tar -xjvf /sources/gcc-4.4.3.tar.bz2
cd gcc-4.4.3
sed -i 's/install_to_$(INSTALL_DEST) //' libiberty/Makefile.in

Fucking sed again.  This is, apparently, preventing something called libiberty being installed.  We already have this from bintutils.

case `uname -m` in
  i?86) sed -i 's/^T_CFLAGS =$/& -fomit-frame-pointer/' gcc/Makefile.in ;;
esac

This is a slightly different version of the Toolchain build command to disable the [-fomit-frame-pointer] option in the build.  This stops the painful bootstrapping happening.  What I am not clear about is why this sets up a loop, while in the Toolchain build it was a one off command.  Not sure why there is a different approach.

sed -i 's@\./fixinc\.sh@-c true@' gcc/Makefile.in

This is another nearly the same as the Toolchain command.  This stops a script attempting to fix mistakes in GCC or Glibc.

sed -i 's/getline/get_line/' libiberty/testsuite/test-demangle.c

That command prevents an error turning up when the test suite was run.  We didn't need to do this for the Toolchain, because we did not really pay much attention to the test outputs.

mkdir -v ../gcc-build
cd ../gcc-build
../gcc-4.4.3/configure --prefix=/usr --libexecdir=/usr/lib --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-languages=c,c++ --disable-multilib --disable-bootstrap

We leave out the options from the Toolchain version which are specific to /tools, which turns out to be quite a bit.  We are also leaving out the [libstdcxx-pch] stuff, presumably because we will have a use for it this time around.  The rest is either from the Toolchain build, or what are rapidly becoming standard options for the final build.

time make $CORES_TO_USE
time make $CORES_TO_USE -k check

The check command will take a LOOOOOONG time.  On my netbook it takes 3-4 HOURS.  On my Quad Core i5 750 it takes less than 20 minutes. If I use the Ramdisk option it takes on 13 minutes.

../gcc-4.4.3/contrib/test_summary

That The only unexpected failures reported were:

FAIL: libmudflap.c++/pass41-frag.cxx execution test 
FAIL: libmudflap.c++/pass41-frag.cxx (-static) execution test 
FAIL: libmudflap.c++/pass41-frag.cxx (-O2) execution test 
FAIL: libmudflap.c++/pass41-frag.cxx (-O3) execution test 

The book confirms that these [libmudflap] tests are known to be problematic.  As are the mud flaps on my car, because I had to fit them myself.  They came with a completely stupid locking mehanism.  Probably no connection to these errors though, when I come to think about it.

make install

We have installed this software to the /usr folder.  Some software apparently expects to find the [cpp] program in the /lib folder.  Also some software looks for [cc] instead of [gcc].  We just symbolically link away both of these problems:

ln -sv ../usr/bin/cpp /lib
ln -sv gcc /usr/bin/cc

OK, we are now going to run some more long winded checks to make absolutely sure that the brand new GCC is being used by the system.

echo 'main(){}' > dummy.c
cc dummy.c -v -Wl,--verbose &> dummy.log
readelf -l a.out | grep ': /lib'

Hello, I am a test.  You may remember me from such tests as when we had just installed Glibc.  My output was:

      [Requesting program interpreter: /lib/ld-linux.so.2]

Good so far.  Next we check for the start files again.

grep -o '/usr/lib.*/crt[1in].*succeeded' dummy.log

/usr/lib/gcc/i686-pc-linux-gnu/4.4.3/../../../crt1.o succeeded
/usr/lib/gcc/i686-pc-linux-gnu/4.4.3/../../../crti.o succeeded
/usr/lib/gcc/i686-pc-linux-gnu/4.4.3/../../../crtn.o succeeded

This is, frankly, bizzare.  When we last ran this test, the response was just [/usr/lib/crt1.o] not all this palaver about i686.  But the curious thing is if you read through the path properly, when you add in all the [../] parent-directory-commands it works out to be exactly the same thing anyway.  Odd, and completely unexplained.

Next we check for the header files.  Without really knowing why.

grep -B4 '^ /usr/include' dummy.log

 /usr/local/include
 /usr/lib/gcc/i686-pc-linux-gnu/4.4.3/include
 /usr/lib/gcc/i686-pc-linux-gnu/4.4.3/include-fixed
 /usr/include

The installation of GCC has added the first three lines to this output.  As this what the book tells me to look for, I am happy, if bemused.  Now to check that the correct search paths are being used:

grep 'SEARCH.*/usr/lib' dummy.log |sed 's|; |\n|g'

SEARCH_DIR("/usr/i686-pc-linux-gnu/lib")
SEARCH_DIR("/usr/local/lib")
SEARCH_DIR("/lib")
SEARCH_DIR("/usr/lib");

If you compare this with the result from earlier, you will see that the /tools reference has been replaced with the first two entries here.  We also want to check that we are using the correct libc – namely the one we installed a little earlier:

grep "/lib.*/libc.so.6 " dummy.log

attempt to open /lib/libc.so.6 succeeded

Again, that is exactly what we are looking for.  Finally check for that speed dater again:

grep found dummy.log

found ld-linux.so.2 at /lib/ld-linux.so.2

It has found it where it was supposed to be.  OK, all of these checks are fairly tedious, but we want to be as sure as possible that nothing got fucked up in the installation of either Glibc or GCC.  These are both massive parts of the Linux operating system that we are building, and if they did get screwed up or put in the wrong place or whatever, then the whole rest of the build would be compromised.  Worst case scenario is that you wouldn't notice until you finished and tried to boot.  By making sure these packages are working fine, you can be sure that when it won't boot it is something else that is fucked up instead.

The only thing that remains now is to tidy up:

rm -v dummy.c a.out dummy.log
cd ..
rm -rvf gcc-4.4.3 gcc-build

No comments:

Post a Comment