Tardis Beginner Tutorials/11

From TARDIS Project

Tutorial 11: Configuring and compiling your own software from source

For a typical windows user, having to compile software you download yourself just to use it may appear rather daunting. You are probably used to being able to download complete binaries ready to run, or software that comes on nice cds that autorun when you put them in the drive, come up with a cute friendly installer wizard with big buttons to click that walks you through the toughest decisions you have to make - where to install it on your disk, and whether you want a desktop shortcut. There are no two ways to install something, and whether the program runs and how is irrelevant because it's just a bog-standard unoptimised generic windows binary. It is not optimised for your computer's architecture, and it will run slowly. If there is a bug in it, you cannot fix it, even if you knew how, because the software is closed-source. You cannot decide what features you want enabled when you install it - you get it all or nothing. This is the sissy way, and we are not sissies. Linux lets you do things properly. In this tutorial we will compile a piece of free software from source and make it run. You will be amazed how easy it is - you'll be compiling your own kernels on your home linux box in no time. Let's break the process down into 6 steps:

1. Finding and downloading the software

For this tutorial we will get a nice console game of tetris. The idea is that it is small, will be easy to configure and fast to compile, and will not have (m)any dependencies to meet - and at the end of it you will get something that actually does something, albeit not especially useful. There is an old and trusted console tetris game included in the bsd-games suite, which is what we will use. A swift google session reveals that the source can be obtained from http://www.ibiblio.org/pub/Linux/games/bsd-games-2.17.tar.gz. I advise you to select and copy the link to save having to type it out. Now what we need to do is download it to our homedir - in the last tutorial i showed you how to use wget, and you should have the appropriate environment variables already set to be able to download things through the uni http proxy. Remember you can paste by just right-clicking in the putty window. Type:

cd ~
wget http://www.ibiblio.org/pub/Linux/games/bsd-games-2.17.tar.gz

That simple. But what to do with this mysterious .tar.gz file??

2. Extracting the downloaded package

The extension means two things. It means it has been "tarballed", and that it has been compressed with gzip. Gzip is just like winzip or any other compression utility - it simply takes a file and makes it physically smaller by using certain mathematical algorithms. However, unlike winzip et al, it only works with an individual file. So in order to compress multiple files together, we can combine them into one large file (which we can extract the contents of later) first, called a "tarball". This is what GNU tar does. This program also supports piping its input or output through various compression programs such as gzip, so there is no need to compress and decompress separately. Let's decompress our package (remember you can use tab completion):

tar -xvvzf bsd-games-2.17.tar.gz

The commandline options are to extract, be very verbose, that the tarball is a gzipped file, and that we should read input from the file by the following name. If you are confused, check man tar (although tar is one of those programs i mentioned earlier that has a much more detailed info page than man page). You could also have achieved the same effect manually by typing gunzip bsd-games-2.17.tar.gz followed by tar -xvvf bsd-games-2.17.tar. It is also worth mentioning that the .tar.gz extension is just one convention, and you will often see files with the shorthand .tgz extension meaning the same thing. You will also see files with extensions such as .tar.bz, for which you should use the -j flag instead of -z.

Ok, now we have the expanded source tree in our homedir - but what to do with it?

3. Configuring the software before compilation

Type ls in your homedir. You should now see your original source file, and a directory by the same name (without the extension).

cd bsd-games-2.17

You will see several directories with various source files in them, several README type files named in caps to make them stand out, and a configure script. When installing a new package you should less README and less INSTALL as well as any other helpful looking files you might see - they usually explain in great detail all the ins and outs of installing the package in question.

The first thing we do is to configure the program prior to compiling it, so make knows what you want out of it. Before you do this you need to know something that varies between users - what user group you belong to. Type groups - this will most likely say "student" or some equivalent. You will need to know this to set the right file permissions later. Type ./configure and you will be faced with a series of prompts about how you want the software configured. You can generally press return to use the default answers for all the questions, except the ones about where the software is going to be installed, and who it will be owned by. We are only compiling tetris from this package, so when a prompt comes up to ask what packages to build (not what NOT to build), simply type "tetris". Since you are installing software as a user rather than root, you do not have access to write to the usual places software is kept to make it available for all users. Therefore you will have to install it inside your own homedir. For all prompts that ask you for a location, simply give it the same one: "~/games". Then for the prompts that ask who what should be owned by, instead of "root" put your username for the user, and the group as your group (that you found out ealier). When the process is complete, the configurator will generate a makefile and exit. This is a file that is read by make that basically tells it what to do and how.

4. Compiling the software

It is now time to compile the software - the source code is in c, and will be compiled by gcc, but we do not need to worry about this. We could easily compile a single source .c file by typing gcc [SOURCE FILENAME] -o [NAME OF BINARY FILE TO MAKE], but software composed of many source files generally tends to be packaged to be built with make. Now we have created a makefile, all we need to do to start the compilation process is to type make - do that now! The process should not take long at all, as we are only compiling tetris.

5. Installing the compiled binaries

So we compiled the compilation, but nothing much seems to have changed. The usable tetris binary has been created, but it is not in a convenient place and it is surrounded by a lot of chaff in the form of no longer needed source files. We can run tetris right now by typing cd tetris and then ./tetris to run it - but let's do this properly. Make sure you are in the ~/bsd-games-2.17 directory and type make install. This will create the appropriate directories in your homedir and set the appropriate permissions for the files based on how we configured it. If you did not set these values correctly in the configurator, this step will fail because you will most likely not have the permissions necessary to write anywhere other than your homedir. If this step succeeds, you will have the ~/games/ directory that you created which will contain the tetris binary (highlighted green if your ls has colour support enabled) that you can run by typing either its full path or ./tetris while in its directory. The ./ is necessary, and just means "in the current directory", as otherwise executables have to be in your PATH variable in the environment to be executed just by typing their name. Don't worry about this too much for now though.

If for some reason the install step failed, you can move the relevant binary file elsewhere manually as it is a simple program that needs nothing but itself to run (no runtime libraries etc). Often you will have difficulty using make install on a certain package if you are trying to install as user, because a lot of software wants to be installed as root to get full functionality and be accessable to all users. If you are having trouble installing such a package, you can ask an admin to install it for you as root on the system. For the current tutorial, if something went wrong with the make install step you can simply move the "~/bsd-games-2.17/tetris/tetris" binary to ~/ and run it from there. You can always go back and run ./configure again and correct your mistakes, if you don't mind answering all the questions again, then make install again - unless you changed any major options you will not need to recompile. All the make install script does is locate the binaries and supporting files in appropriate and convenient locations automatically and change their permissions correctly.

6. Cleaning up the remaining unneeded files to make space

Now we have our tetris binary created and moved somewhere new, such as ~/ or ~/games, we can remove the remaining source files as we no longer need them.

cd ~
rm bsd-games-2.17.tar.gz -f
rm bsd-games-2.17 -rf

As mentioned earlier, be exceedingly careful when combining -r and -f, as the latter forces a delete, leaving no room for error.

Now you can play tetris that you compiled yourself, on your tardis shell account, by going to its directory and typing ./tetris. Congratulations! You have now completed my beginner tutorials. This does not mean there is nothing left to learn however! My tutorials have tried to keep it extremely brief, and there is a lot you have to learn for yourself from man pages and maybe other internet tutorials. For example, it is very important to know about pipes, that i have barely mentioned - you should google for "bash pipes tutorial" to find out more. Nevertheless, now you should be able to do most everyday tasks from your shell account without difficulty. If you have completed all these tutorials in order, i want to thank you for taking the time to do so. Please contact me on irc or by email as described on the index page to suggest any improvements, no matter how minor - if you were lost at any point, please tell me so that i can clarify for future readers!

Next: Cheatsheet