Re: Tutorial

["Yann E. MORIN" <yann dot morin dot 1998 at anciens dot enib dot fr>]

Francesco, All,

On Monday 17 January 2011 14:48:04 Francesco Turco wrote:
> This message is not strictly related to crosstool-ng but I hope I'm not
> off-topic here.

It is totally on-topic here. :-)

The list is not dedicated to crosstool-NG, and whatever implies using
gcc in a cross-compiler setup is welcome on the list.

> I've written a basic tutorial about building a GCC cross-compiler using
> uClibc as the target C library. You can find it here:
> http://fturco.org/wiki/doku.php?id=ubuntu:cross-compiler
> 
> Since this is a very new document, it still misses lots of informations
> and it's not very well tested. Please report me any problems with it.
> 
> My goal is not to create another automatic tool such as buildroot or
> crosstool-ng. They already do great things. I'd like instead to share my
> (very little) knowledge about cross-compilers and teach new people how to
> accomplish this task and learn something at the same time.
> 
> What do you think?

I did not look in details, but here are a few comments:

You do not need to configure the kernel to install the headers

In your example, the gcc pass-1 is not required, as the uClibc headers
do not need a cross compiler when using LinuxThreads. If you were to 
use NPTL, then you need the gcc pass 1, and you'd need to also install
the uClibc "start files" (which you are missing), see below.

I like that you added the non-parallel troubleshooting. That is important
to note. :-)

Installing the pass 1 & 2 compilers in the final prefix is dubious. It
/might/ break in some cases, especially when doing a canadian-cross.

As you stated that it is not intended as  "another automatic tool" but
rather a mean to "teach new people", I would have first introduced the
dependencies that exist between all the components. For example, why one
needs to build gcc three times. Or why one needs to install the kernel
headers.

All this might sound obvious to you and us, but to it will most probably
sound like black magic to newcomers (at least it did to me when I started
building toolchains).

So in your case, I would articulate the tutorial in a different manner.
Rather than a step-by-step procedure that applies to one case (that is,
building an i686 toolchain on a x86_64 host), I'd do start with what the
user eventually wants, a cross-compiler. Fell free to use the following,
adapt it and fix typos/oversights:

---8<---

A cross-compiler is in fact a collection of different tools set up to
tightly work together. The tools are arranged in a way that they are
chained, in a kind of cascade, where the output from one becomes the
input to another one, to ultimately produce the actual binary code that
runs on a machine. So, we call this arrangement a "toolchain". When
a toolchain is meant to generate code for a machine different from the
machine it runs on, this is called a cross-toolchain.

The components that play a role in the toolchain are first and foremost
the compiler itself. The compiler turns source code (in C, C++, whatever)
into assembly code. The compiler of choice is the GNU compiler collection,
well known as 'gcc'.

The assembly code is interpreted by the assembler to generate object code.
This is done by the binary utilities, such as the GNU 'binutils'.

Once the different object code files have been generated, they got to get
aggregated together to form the final executable binary. This is called
linking, and is achieved with the use of a linker. The GNU 'binutils' also
come with a linker.

So far, we get a complete toolchain that is capable of turning source code
into actual executable code. Depending on the Operating System, or the lack
thereof, running on the target, we also need the C library. The C library
provides a standard abstraction layer that performs basic tasks (such as
allocating memory, printing output on a terminal, managing file access...).
There are many C libraries, each targetted to different systems. For the
Linux /desktop/, there is glibc or eglibc, for embeded Linux, you have a
choice of eglibc or uClibc, while for system without an Operating System,
you may use newlib, dietlibc, or even none at all. There a few other C
libraries, but they are not as widely used, and/or are targetted to very
specific needs (eg. klibc is a very small subset of the C library aimed at
building contrained initial ramdisks).

Under Linux, the C library needs to know the API to the kernel to decide
what features are present, and if needed, what emulation to include for
missing features. That API is provided by the kernel headers. Note: this
is Linux-specific (and potentially a very few others), the C library on
other OSes do not need the kernel headers.

So far, all major components have been covered, but yet there is a specific
order they need to be built. Here we see what the dependencies are, starting
with the compiler we want to ultimately use. We call that compiler the
'final compiler'.

  - the final compiler needs the C library, to know how to use it,
but:
  - building the C library requires a compiler

A needs B which needs A. This is the classic chicken'n'egg problem... This
is solved by building a stripped-down compiler that does not need the C
library, but is capable of building it. We call it a bootstrap, initial, or
core compiler. So here is the new dependency list:

  - the final compiler needs the C library, to know how to use it,
  - building the C library requires a core compiler
but:
  - the core compiler needs the C library headers and start files, to know
    how to use the C library

B needs C whixh needs B. Chicken'n'egg, again. To solve this one, we will
need to build a C library that will only install its headers and start
files. The start files are a very few files that gcc needs to be able to
turn on thread local storage (TLS) on an NPTL system. So now we have:

  - the final compiler needs the C library, to know how to use it,
  - building the C library requires a core compiler
  - the core compiler needs the C library headers and start files, to know
    how to use the C library
but:
  - building the start files require a compiler

Geez... C needs D which needs C, yet again. So we need to build a yet
simpler compiler, that does not need the headers and does need the start
files. This compiler is also a bootstrap, initial or ocre compiler. In order
to differentiate the two core compilers, let's call that one "core pass 1",
and the former one "core pass 2". The dependency list becomes:

  - the final compiler needs the C library, to know how to use it,
  - building the C library requires a core compiler
  - the core pass 2 compiler needs the C library headers and start files,
    to know how to use the C library
  - building the start files requires a compiler
  - we need a core pass 1 compiler

And as we said earlier, the C library also requires the kernel headers.
There is no requirement for the kernel headers, so end of story in this
case:

  - the final compiler needs the C library, to know how to use it,
  - building the C library requires a core compiler
  - the core pass 2 compiler needs the C library headers and start files,
    to know how to use the C library
  - building the start files requires a compiler and the kernel headers
  - we need a core pass 1 compiler

We need to add a few new requirements. The moment we compile code for the
target, we need the assembler and the linker. Such code is, of course,
built from the C library, so we need to build the binutils before the C
library start files, and the complete C library itself. Also, some code
in gcc will turn to run on the target as well. Luckily, there is no
requirement for the binutils. So, our dependency chain is as follows:

  - the final compiler needs the C library, to know how to use it, and the
    binutils
  - building the C library requires a core pass 2 compiler and the binutils
  - the core pass 2 compiler needs the C library headers and start files,
    to know how to use the C library, and the binutils
  - building the start files requires a compiler, the kernel headers and the
    binutils
  - the core pass 1 compiler needs the binutils

Which turns in this order to build the components:

  1 binutils
  2 core pass 1 compiler
  3 kernel headers
  4 C library headers and start files
  5 core pass 2 compiler
  6 complete C library
  7 final compiler

Yes! :-) But are we done yet? 

In fact, no, there are still missing dependencies. As far as the tools
themselves are involved, we do not need anything else.

But gcc has a few pre-requisites. It relies on a few external libraries to
perform some non-trivial tasks (such as handling complex numbers in
constants...). There are a few options to build those libraries. First, one
may think to rely on a Linux distribution to provide those libraries. Alas,
they were not widely available until very, very recently. So, if the distro
is not too recent, chances are that we will have to build those libraries
(which we do below). The affected libraries are:

  - the GNU Multiple Precision Arithmetic Library, GMP
  - the C library for multiple-precision floating-point computations with
    correct rounding, MPFR
  - the C library for the arithmetic of complex numbers, MPC

The dependencies for those liraries are:

  - MPC requires GMP and MPFR
  - MPFR requires GMP
  - GMP has no pre-requisite

So, the build order becomes:

  1 GMP
  2 MPFR
  3 MPC
  4 binutils
  5 core pass 1 compiler
  6 kernel headers
  7 C library headers and start files
  8 core pass 2 compiler
  9 complete C library
 10 final compiler

Yes! Or yet some more?

This is now sufficient to build a functional toolchain. So if you've had
enough for now, you can stop here. Or if you are curious, you can continue
reading.

gcc can also make use of a few other external libraries. These additional,
optional libraries are used to enable advanced features in gcc, such as
loop optimisation (GRAPHITE) and Link Time Optimisation (LTO). If you want
to use these, you'll need three additional libraries:

  - the Parma Polyhedra Library, PPL
  - the Chunky Loop Generator, using the PPL backend, CLooG/PPL
  - the ELF object file access library, libelf

The depencies for those libraries are:

  - PPL requires GMP
  - CLooG/PPL requires GMP and PPL
  - libelf has no pre-requisites

The list now looks like (optional libs with a *):

  1 GMP
  2 MPFR
  3 MPC
  4 PPL *
  5 CLooG/PPL *
  6 libelf *
  7 binutils
  8 core pass 1 compiler
  9 kernel headers
 10 C library headers and start files
 11 core pass 2 compiler
 12 complete C library
 13 final compiler

This list is now complete! Wouhou! :-)

---8<---

And once you have stated the above, you can carry on with an example. I would
suggest doing a true cross-compiler example. i686 on x86_64 is not really
representative, while building an ARM cross-compiler on x86_64 is.

Hope this helps! :-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

--
For unsubscribe information see http://sourceware.org/lists.html#faq