toolchaintechnotes.xml 12 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221
  1. <?xml version="1.0" encoding="ISO-8859-1"?>
  2. <!DOCTYPE sect1 PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
  3. "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
  4. <!ENTITY % general-entities SYSTEM "../general.ent">
  5. %general-entities;
  6. ]>
  7. <sect1 id="ch-tools-toolchaintechnotes">
  8. <?dbhtml filename="toolchaintechnotes.html"?>
  9. <title>Toolchain Technical Notes</title>
  10. <para>This section explains some of the rationale and technical details
  11. behind the overall build method. It is not essential to immediately
  12. understand everything in this section. Most of this information will be
  13. clearer after performing an actual build. This section can be referred
  14. to at any time during the process.</para>
  15. <para>The overall goal of <xref linkend="chapter-temporary-tools"/> is to
  16. produce a temporary area that contains a known-good set of tools that can be
  17. isolated from the host system. By using <command>chroot</command>, the
  18. commands in the remaining chapters will be contained within that environment,
  19. ensuring a clean, trouble-free build of the target LFS system. The build
  20. process has been designed to minimize the risks for new readers and to provide
  21. the most educational value at the same time.</para>
  22. <important>
  23. <para>Before continuing, be aware of the name of the working platform,
  24. often referred to as the target triplet. A simple way to determine the
  25. name of the target triplet is to run the <command>config.guess</command>
  26. script that comes with the source for many packages. Unpack the Binutils
  27. sources and run the script: <userinput>./config.guess</userinput> and note
  28. the output. For example, for a modern 32-bit Intel processor the
  29. output will likely be <emphasis>i686-pc-linux-gnu</emphasis>.</para>
  30. <para>Also be aware of the name of the platform's dynamic linker, often
  31. referred to as the dynamic loader (not to be confused with the standard
  32. linker <command>ld</command> that is part of Binutils). The dynamic linker
  33. provided by Glibc finds and loads the shared libraries needed by a program,
  34. prepares the program to run, and then runs it. The name of the dynamic
  35. linker for a 32-bit Intel machine will be
  36. <filename class="libraryfile">ld-linux.so.2</filename>.
  37. A sure-fire way to determine the name of the dynamic linker is to
  38. inspect a random binary from the host system by running:
  39. <userinput>readelf -l &lt;name of binary&gt; | grep interpreter</userinput>
  40. and noting the output. The authoritative reference covering all platforms
  41. is in the <filename>shlib-versions</filename> file in the root of the Glibc
  42. source tree.</para>
  43. </important>
  44. <para>Some key technical points of how the <xref
  45. linkend="chapter-temporary-tools"/> build method works:</para>
  46. <itemizedlist>
  47. <listitem>
  48. <para>Slightly adjusting the name of the working platform, by changing the
  49. &quot;vendor&quot; field target triplet by way of the
  50. <envar>LFS_TGT</envar> variable, ensures that the first build of Binutils
  51. and GCC produces a compatible cross-linker and cross-compiler. Instead of
  52. producing binaries for another architecture, the cross-linker and
  53. cross-compiler will produce binaries compatible with the current
  54. hardware.</para>
  55. </listitem>
  56. <listitem>
  57. <para> The temporary libraries are cross-compiled. Because a
  58. cross-compiler by its nature cannot rely on anything from its host
  59. system, this method removes potential contamination of the target
  60. system by lessening the chance of headers or libraries from the host
  61. being incorporated into the new tools. Cross-compilation also allows for
  62. the possibility of building both 32-bit and 64-bit libraries on 64-bit
  63. capable hardware.</para>
  64. </listitem>
  65. <listitem>
  66. <para>Careful manipulation of <command>gcc</command>'s
  67. <filename>specs</filename> file tells the compiler which target dynamic
  68. linker will be used</para>
  69. </listitem>
  70. </itemizedlist>
  71. <para>Binutils is installed first because the <command>configure</command>
  72. runs of both GCC and Glibc perform various feature tests on the assembler
  73. and linker to determine which software features to enable or disable. This
  74. is more important than one might first realize. An incorrectly configured
  75. GCC or Glibc can result in a subtly broken toolchain, where the impact of
  76. such breakage might not show up until near the end of the build of an
  77. entire distribution. A test suite failure will usually highlight this error
  78. before too much additional work is performed.</para>
  79. <para>Binutils installs its assembler and linker in two locations,
  80. <filename class="directory">/tools/bin</filename> and <filename
  81. class="directory">/tools/$LFS_TGT/bin</filename>. The tools in one
  82. location are hard linked to the other. An important facet of the linker is
  83. its library search order. Detailed information can be obtained from
  84. <command>ld</command> by passing it the <parameter>--verbose</parameter>
  85. flag. For example, an <userinput>ld --verbose | grep SEARCH</userinput>
  86. will illustrate the current search paths and their order. It shows which
  87. files are linked by <command>ld</command> by compiling a dummy program and
  88. passing the <parameter>--verbose</parameter> switch to the linker. For example,
  89. <userinput>gcc dummy.c -Wl,--verbose 2&gt;&amp;1 | grep succeeded</userinput>
  90. will show all the files successfully opened during the linking.</para>
  91. <para>The next package installed is GCC. An example of what can be
  92. seen during its run of <command>configure</command> is:</para>
  93. <screen><computeroutput>checking what assembler to use... /tools/i686-lfs-linux-gnu/bin/as
  94. checking what linker to use... /tools/i686-lfs-linux-gnu/bin/ld</computeroutput></screen>
  95. <para>This is important for the reasons mentioned above. It also demonstrates
  96. that GCC's configure script does not search the PATH directories to find which
  97. tools to use. However, during the actual operation of <command>gcc</command>
  98. itself, the same search paths are not necessarily used. To find out which
  99. standard linker <command>gcc</command> will use, run:
  100. <userinput>gcc -print-prog-name=ld</userinput>.</para>
  101. <para>Detailed information can be obtained from <command>gcc</command> by
  102. passing it the <parameter>-v</parameter> command line option while compiling
  103. a dummy program. For example, <userinput>gcc -v dummy.c</userinput> will show
  104. detailed information about the preprocessor, compilation, and assembly stages,
  105. including <command>gcc</command>'s included search paths and their order.</para>
  106. <para>The next package installed is Glibc. The most important considerations
  107. for building Glibc are the compiler, binary tools, and kernel headers. The
  108. compiler is generally not an issue since Glibc will always use the compiler
  109. relating to the <parameter>--host</parameter> parameter passed to its
  110. configure script, e.g. in our case,
  111. <command>i686-lfs-linux-gnu-gcc</command>. The binary tools and kernel
  112. headers can be a bit more complicated. Therefore, take no risks and use the
  113. available configure switches to enforce the correct selections. After the run
  114. of <command>configure</command>, check the contents of the
  115. <filename>config.make</filename> file in the <filename
  116. class="directory">glibc-build</filename> directory for all important details.
  117. Note the use of <parameter>CC="i686-lfs-gnu-gcc"</parameter> to control which
  118. binary tools are used and the use of the <parameter>-nostdinc</parameter> and
  119. <parameter>-isystem</parameter> flags to control the compiler's include
  120. search path. These items highlight an important aspect of the Glibc
  121. package&mdash;it is very self-sufficient in terms of its build machinery and
  122. generally does not rely on toolchain defaults.</para>
  123. <para>After the Glibc installation, change <command>gcc</command>'s specs file
  124. to point to the new dynamic linker in <filename
  125. class="directory">/tools/lib</filename>. This last step is vital in ensuring
  126. that searching and linking take place only within the <filename
  127. class="directory">/tools</filename> prefix. A hard-wired
  128. path to a dynamic linker is embedded into every Executable and Link Format
  129. (ELF)-shared executable. This can be inspected by running:
  130. <userinput>readelf -l &lt;name of binary&gt; | grep interpreter</userinput>.
  131. Amending <command>gcc</command>'s specs file ensures that every program
  132. compiled from here through the end of this chapter will use the new dynamic
  133. linker in <filename class="directory">/tools/lib</filename>.</para>
  134. <para>For the second pass of GCC, its sources also need to be modified
  135. to tell GCC to use the new dynamic linker. Failure to do
  136. so will result in the GCC programs themselves having the name of the
  137. dynamic linker from the host system's <filename
  138. class="directory">/lib</filename> directory embedded into them, which
  139. would defeat the goal of getting away from the host.</para>
  140. <para>During the second pass of Binutils, we are able to utilize the
  141. <parameter>--with-lib-path</parameter> configure switch to control
  142. <command>ld</command>'s library search path. From this point onwards,
  143. the core toolchain is self-contained and self-hosted. The remainder of
  144. the <xref linkend="chapter-temporary-tools"/> packages all build against
  145. the new Glibc in <filename class="directory">/tools</filename>.</para>
  146. <para>Upon entering the chroot environment in <xref
  147. linkend="chapter-building-system"/>, the first major package to be
  148. installed is Glibc, due to its self-sufficient nature mentioned above.
  149. Once this Glibc is installed into <filename
  150. class="directory">/usr</filename>, we will perform a quick changeover of the
  151. toolchain defaults, and then proceed in building the rest of the target
  152. LFS system.</para>
  153. <!-- FIXME: Removed as part of the fix for bug 1061 - we no longer build pass1
  154. packages statically, therefore this explanation isn't required
  155. <sect2>
  156. <title>Notes on Static Linking</title>
  157. <para>Besides their specific task, most programs have to perform many
  158. common and sometimes trivial operations. These include allocating
  159. memory, searching directories, reading and writing files, string
  160. handling, pattern matching, arithmetic, and other tasks. Instead of
  161. obliging each program to reinvent the wheel, the GNU system provides
  162. all these basic functions in ready-made libraries. The major library
  163. on any Linux system is Glibc.</para>
  164. <para>There are two primary ways of linking the functions from a
  165. library to a program that uses them&mdash;statically or dynamically. When
  166. a program is linked statically, the code of the used functions is
  167. included in the executable, resulting in a rather bulky program. When
  168. a program is dynamically linked, it includes a reference to the
  169. dynamic linker, the name of the library, and the name of the function,
  170. resulting in a much smaller executable. A third option is to use the
  171. programming interface of the dynamic linker (see <filename>dlopen(3)</filename>
  172. for more information).</para>
  173. <para>Dynamic linking is the default on Linux and has three major
  174. advantages over static linking. First, only one copy of the executable
  175. library code is needed on the hard disk, instead of having multiple
  176. copies of the same code included in several programs, thus saving
  177. disk space. Second, when several programs use the same library
  178. function at the same time, only one copy of the function's code is
  179. required in core, thus saving memory space. Third, when a library
  180. function gets a bug fixed or is otherwise improved, only the one
  181. library needs to be recompiled instead of recompiling all programs
  182. that make use of the improved function.</para>
  183. <para>If dynamic linking has several advantages, why then do we
  184. statically link the first two packages in this chapter? The reasons
  185. are threefold&mdash;historical, educational, and technical. The
  186. historical reason is that earlier versions of LFS statically linked
  187. every program in this chapter. Educationally, knowing the difference
  188. between static and dynamic linking is useful. The technical benefit is
  189. a gained element of independence from the host, meaning that those
  190. programs can be used independently of the host system. However, it is
  191. worth noting that an overall successful LFS build can still be
  192. achieved when the first two packages are built dynamically.</para>
  193. </sect2>-->
  194. </sect1>