Monday 19 July 2010

LAP - Actual Build - Part 1, Prepare for Chroot

OK, welcome back class.  We have our (hopefully) functioning Toolchain, and we are ready to move on to the next stage, which is the actual building of the final system.  Remember everything in the /tools directory that we have just made will be deleted once we are finished with the installation.  Everything we have done so far is temporary.

What were are now going to do is to use the chroot program.  This takes a little bit of explanation.  You will be aware by now that there in the Linux directory structure the bottom-most level is called '/'.  This is the root directory.  All other directories and files (including all hardware – remember it is treated as files) branch from this root directory.

The chroot command literally [ch]anges the [root] directory to be something else.  So if you made a directory called, for instance funbags:

mkdir /funbags

Oooooh, for fucks sake:

sudo mkdir /funbags

You can then use the command:

sudo chroot /funbags

and effectively /funbags is the new /.  What actually happens if you try this is that you get the message:

chroot: cannot run command `/bin/bash': No such file or directory

Now, of course the command /bin/bash exists – that is what is generating the flashing cursor after all.  But remember /funbags is now /.  Is there a /funbags/bin/bash?  No because we just made /funbags and it is empty.  Lets try a little trick:

sudo mkdir /funbags/bin
cd /funbags/bin
sudo ln -sv /bin/bash ./bash

What we have done there is to make a 'bin' directory in /funbags, change into that new directory and create a symbolic link to the real /bin/bash file.  Now lets try to chroot again:

sudo chroot /funbags

chroot: cannot run command `/bin/bash': Too many levels of symbolic links

Ah.  Another error – but note it is a different one.  It has found our new /bin/bash (which remember is actually /funbags/bin/bash) but it is now complaining about the (not so) clever symbolic link.

In other words, the chroot command EXPECTS the directory that you chroot into to have all the proper programs and directories that a working installation of Linux would have.  Why do we want to chroot at all?  Well, when we install the final Linux programs we want them to think that they are installed in their final working places.  If they knew that they were actually being installed in /media/lfs/[whatever] then they will get very, very upset once you try to boot from the USB Key directly.  Because then /media/lfs will become '/'. 

Hang on though, isn't /media/lfs becoming '/' when booted directly, just the chroot effect in reverse?  Why yes, yes it is.  So what we do is chroot INTO /media/lfs, and carry out the installation.  Because of the chroot command, as far as any of the software that is being installed is concerned, it is being installed to '/', so when we reboot to the USB Key directly, it should not notice any difference.

As we discovered with the /funbags test above, we need to ensure that the /media/lfs directory has all the files and directories that are needed to run Linux.  At the moment it has /tools, and /sources (the latter should be empty).  Not good enough is it?  From now, I am assuming that you have exited the lfs user mode from earlier, and you are once again logged in as the LiveCD user.  You could well be doing this after a reboot, in which case all you actually need to do is to mount the USB Key.  You no longer need to apt-get install the build software because we will be using the Toolchain from now on.

To get started, we need to manually create the required directories.

sudo mkdir -v /media/lfs/{dev,proc,sys}

That command makes the directories /media/lfs/dev /media/lfs/proc and /media/lfs/sys, just in an efficient way.  The dev directory will eventually contain files which related to all devices on the PC, disks, processors, mice, etc etc.  The proc directory is eventually going to contain files which correspond to every running [proc]ess on the PC.  The sys folder is just used as a point in the filesystem for the kernel (once it is up and running) to mount a virtual filesystem called sysfs.  This is, to a reasonable approximation, an advanced version of /dev/.

mknod -m 600 /media/lfs/dev/console c 5 1

mknod: `/media/lfs/dev/console': Permission denied 

Sigh.

sudo mknod -m 600 /media/lfs/dev/console c 5 1
sudo mknod -m 666 /media/lfs/dev/null c 1 3

These two commands [m]a[k]e [nod]es in the /dev/ directory we just made.  I think these are both special types of file – we dealt with /dev/null when talking about the bash configuration files.  The three digit number is, unsurprisingly, permissions bullshit.  The 'c' tells the command thse are special files.  I have no idea what the trailing numbers are about – the mknod manual page says they are Major and Minor variables.  Absolutely no indication of what that actually means though.

We will eventually build a piece of software that will automatically manage all of our hardware for us (udev), but for now the best option is to just make a copy of the existing machine's /dev/ directory.  That way, when we chroot into /funbags ... sorry /media/lfs the console will still be able to see all of the hardware.  To do this we run a special mount command:

mount -v --bind /dev /media/lfs/dev

mount: only root can do that

Sigh. 

sudo mount -v --bind /dev /media/lfs/dev

The --bind option tells the mount command to make a mirror of the existing /dev/ directory – so any changes are replicated instantly at /media/lfs/dev

We then also need to share some of the existing virtual filesystems.  These (including the sys one I referred to earlier) are held in RAM while the machine is operating and have a link to the '/' filesystem so they can interact with the rest of the machine.

sudo mount -vt devpts devpts /media/lfs/dev/pts
sudo mount -vt tmpfs shm /media/lfs/dev/shm
sudo mount -vt proc proc /media/lfs/proc
sudo mount -vt sysfs sysfs /media/lfs/sys

The [t] option in the mount command this time tells it that what we are trying to mount is a special type of filesystem.  It is the same [t] as we use to tell the system that the USB Key is in format 'ext3'.  So the first command tells it to mount a filesystem of [t]ype [devpts] called [devpts] at location [media/lfs/dev/pts].  I have covered sys and proc.  What are devpts and shm?  Well ... oh god it is so depressing ... just read explanation of devpts ... so boring ... have lost the will to live ...   clinging to sanity by finger nails ... something about lots of little devices ... in /dev .. take up too much space ... conjoin all in one big virtual file ... automanage that .. saves hassle ... *GASP* *GASP*, PHEEEeeew.  I don't know if I can bear shm ... lets see ... reading through half closed eyes just in case ... Oh! It's a Ramdisk.  Well, that's simple.

The system is now ready to use the chroot command, so lets do it:

sudo chroot "/media/lfs" /tools/bin/env -i HOME=/root TERM="$TERM" PS1='\u:\w\$ ' PATH=/bin:/usr/bin:/sbin:/usr/sbin:/tools/bin CORES_TO_USE=-j2 /tools/bin/bash --login +h

Well, that looks more complication than chroot /knockers or whatever we were trying above.  It is easily broken down though, and we have seen all of this before – remember the bash config files?  Instead of the 'env' command on the host system being used, here we tell the system to use the 'env' command that we build into the Toolchain [/tools/bin/env].  We set HOME, TERM and PS1.  The path is going to look a bit different now.  We make sure that /tools is at the end of the path for the same type of reason as we put it at the beginning before.  We now want /tools to be the last place that the system looks for a command.  This means that as soon as we have built each package now, it takes over from the /tools version immediately.  We use the [+h] to again prevent the system from memorising where to find programs.  Finally we direct the system to /tools/bin/bash because, of course, there is no /bin/bash yet.  I get the prompt:

I have no name!:/# 

Which looks a little alarming.  However, the point here is that there are no users or groups set up and the machine is unnamed.  It literally does not know who or where it is.  Much like a student at 6:30am on a Monday morning.

The absolute final step we have to take now, before we start building stuff again, is to create the rest of the filesystem.  This is the stuff that was not strictly essential just to get the chroot command to work, but will be needed when we come to start installing software.

mkdir -pv /{bin,boot,etc/opt,home,lib,mnt,opt}
mkdir -pv /{media/{floppy,cdrom},sbin,srv,var}
install -dv -m 0750 /root
install -dv -m 1777 /tmp /var/tmp
mkdir -pv /usr/{,local/}{bin,include,lib,sbin,src}
mkdir -pv /usr/{,local/}share/{doc,info,locale,man}
mkdir -v  /usr/{,local/}share/{misc,terminfo,zoneinfo}
mkdir -pv /usr/{,local/}share/man/man{1..8}
for dir in /usr /usr/local; do
  ln -sv share/{man,doc,info} $dir
done
case $(uname -m) in
 x86_64) ln -sv lib /lib64 && ln -sv lib /usr/lib64 ;;
esac
mkdir -v /var/{lock,log,mail,run,spool}
mkdir -pv /var/{opt,cache,lib/{misc,locate},local}

This all looks much more complicated that it actually is.  There are actually only three commands being run here, the mkdir, install and ln commands.  I have no particular desire to rehash the directory structure of Linux here, so I will just direct you to the Filesystem Hierarchy Standard which helpfully explains it all.  There are a couple of tricky bits.  The /root directory is the home of the root user.  You do not want anyone to even ENTER that directory let alone change its contents.  The [install] command essentially makes the directory and then changes its security features in one step, efficiently.  The same is true of the [install] command for the [/tmp] directory.  This has a sticky bit attached to it, which turns out to mean that everyone can write to the directory, but you cannot overwrite someone else's data.  Lastly, some of the directories need to have symbolic links created and there are a couple of loops for that purpose.

Some of the software that we are going to install assumes that some other important programs are installed in their normal places – NOT /tools.  Therefore we need some more symbolic links to fool this software into working before we install the final versions of these important programs:

ln -sv /tools/bin/{bash,cat,echo,pwd,stty} /bin
ln -sv /tools/bin/perl /usr/bin
ln -sv /tools/lib/libgcc_s.so{,.1} /usr/lib
ln -sv /tools/lib/libstdc++.so{,.6} /usr/lib
ln -sv bash /bin/sh

The first line makes a link to the bash program – don't panic though – we won't get our /jubblies error about symbolic links this time, because we are linking one file to another from inside the chroot environment.  The second to fourth links relate to the C, C++ and Perl compilers/interpreters.  The last link is a bit sneaky.  Technically, sh is a different shell program to Bash, although they both perform the same role – giving you a flashing cursor.  I feel sure there is more to a shell program, but I just can't put my finger on what that might be.  What this command does is redirect any program looking for sh to bash.  As I said, sneaky.

We now need to create some important files before we can proceed.  The first is simple:

touch /etc/mtab

mtab, which I had no idea about at an earlier stage, turns out to be a file which keeps a list of all mounted filesystems in it.  So typing [cat /etc/mtab] or [mount] should give you access to the same information.  We do not need to put anything in mtab now, just the fact that it exists is enough.  Next we need to cure the system's identity crisis.  First of all lets make a file with all the user names in it.  That would be the /etc/passwd file:

cat > /etc/passwd << "EOF"
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/dev/null:/bin/false
nobody:x:99:99:Unprivileged User:/dev/null:/bin/false
EOF

The syntax of that file is username:password:usernumber:groupnumber:description:home folder: logon shell program.  'x' is just a stand in password, and passwords are eventually stored encrypted so worry not about security.

Next we need to define our initial groups in /etc/group:

cat > /etc/group << "EOF"
root:x:0:
bin:x:1:
sys:x:2:
kmem:x:3:
tty:x:4:
tape:x:5:
daemon:x:6:
floppy:x:7:
disk:x:8:
lp:x:9:
dialout:x:10:
audio:x:11:
video:x:12:
utmp:x:13:
usb:x:14:
cdrom:x:15:
mail:x:34:
nogroup:x:99:
EOF

I am not going to comment on these, as they are a standard part of a *nix architecture.  There are plenty of places you can read up on these, starting with the Linux Standard Base.

Now to use these files, we need to restart the shell:

exec /tools/bin/bash --login +h

Note we are still specifying +h.  The restarted shell should be able to look up what its name is, and the prompt will change accordingly. The penultimate step to take is to set up some log files which some programs expect to exist.  These are in the 'var' directory and are created as follows:

touch /var/run/utmp /var/log/{btmp,lastlog,wtmp}
chgrp -v utmp /var/run/utmp /var/log/lastlog
chmod -v 664 /var/run/utmp /var/log/lastlog

We change the group ownership here to utmp, so we need to create these after we create the /etc/group file.

Lastly, if we still want to proceed to use the Live Key's [/tmp] directory as a handy ramdisk, we are going to have to mount it to the new filesystem we have just made. Remember if we do not do this, then /tmp from the [chroot] [env]ironment is just a directory on the Amiga Key. So we need to exit from the chroot ...

exit

... and to mount the Live [/tmp] folder to the [/tmp] folder on the Amiga Key we run:

sudo mount -v --bind /tmp /media/lfs/tmp

We are again using the --bind option to mirror the directory.  Lastly log back in to the chroot:

sudo chroot "/media/lfs" /tools/bin/env -i HOME=/root TERM="$TERM" PS1='\u:\w\$ ' PATH=/bin:/usr/bin:/sbin:/usr/sbin:/tools/bin CORES_TO_USE=-j2 /tools/bin/bash --login +h

Most the stuff that we had put in the .bash_profile and .bashrc files we can just specify as [env]ironmental variables for the [chroot] command. Handy. Note that when you run this now, you get straight to the prompt:

root:/#

So there we go.  We are now running as the root user in the faked root of a brand new filesystem.  Only the very basic structure is set up, and it is just waiting to be filled up with a functional linux system.  The only part of the existing host system that we are now using is the running kernel.  All is calm.

Let's properly fuck this up.

To summarise what you would need to do to get back to this stage from a reboot of the LiveCD, run the following commands:

umount -v /media/amiga
sudo mkdir /media/lfs
sudo mount -v -t ext3 /dev/disk/by-label/amiga /media/lfs
sudo mount -v --bind /dev /media/lfs/dev
sudo mount -vt devpts devpts /media/lfs/dev/pts
sudo mount -vt tmpfs shm /media/lfs/dev/shm
sudo mount -vt proc proc /media/lfs/proc
sudo mount -vt sysfs sysfs /media/lfs/sys
sudo mount -v --bind /tmp /media/lfs/tmp
sudo chroot "/media/lfs" /tools/bin/env -i HOME=/root TERM="$TERM" PS1='\u:\w\$ ' PATH=/bin:/usr/bin:/sbin:/usr/sbin:/tools/bin CORES_TO_USE=-j2 /tools/bin/bash --login +h
cd /tmp

And that should do it. Remember, if you do not have enough memory to use the [/tmp] as a ramdisk, in the following posts, cd into [/sources] not [/tmp] before unpacking and building.

No comments:

Post a Comment