This page is a copy of the
[Archive.org](https://web.archive.org/web/20201202024834/https://web.archive.org/web/20120922073347/http://www.acsu.buffalo.edu/~charngda/elf.html)
copy of the now no longer availabel
[http://www.acsu.buffalo.edu/\~charngda/elf.html](https://web.archive.org/web/20201202024834/http://www.acsu.buffalo.edu/~charngda/elf.html).
It is kept here online as a reference only.

------------------------------------------------------------------------

Acronyms relevant to Executable and Linkable Format (ELF)
---------------------------------------------------------

<table data-border="">
<colgroup>
<col style="width: 50%" />
<col style="width: 50%" />
</colgroup>
<tbody>
<tr class="odd">
<td><a href="https://web.archive.org/web/20201202024834/http://www.wikipedia.org/wiki/Application_binary_interface">ABI</a></td>
<td>Application binary interface</td>
</tr>
<tr class="even">
<td><a href="https://web.archive.org/web/20201202024834/http://www.wikipedia.org/wiki/A.out">a.out</a></td>
<td>Assembler output file format</td>
</tr>
<tr class="odd">
<td><a href="https://web.archive.org/web/20201202024834/http://www.wikipedia.org/wiki/.bss">BSS</a></td>
<td>Block started by symbol. The uninitialized data segment containing statically-allocated variables.</td>
</tr>
<tr class="even">
<td><a href="https://web.archive.org/web/20201202024834/http://www.wikipedia.org/wiki/COFF">COFF</a></td>
<td>Common object file format</td>
</tr>
<tr class="odd">
<td>DTV</td>
<td>Dynamic thread vector (for TLS)</td>
</tr>
<tr class="even">
<td><a href="https://web.archive.org/web/20201202024834/http://www.wikipedia.org/wiki/DWARF">DWARF</a></td>
<td>A standardized debugging data format</td>
</tr>
<tr class="odd">
<td>GD</td>
<td>Global Dynamic (dynamic TLS) One of the <a href="https://web.archive.org/web/20201202024834/http://download.oracle.com/docs/cd/E19963-01/html/819-0690/chapter8-1.html">Thread-Local Storage access models</a>.</td>
</tr>
<tr class="even">
<td>GOT</td>
<td>Global offset table</td>
</tr>
<tr class="odd">
<td>IE</td>
<td>Initial Executable (static TLS with assigned offsets) One of the <a href="https://web.archive.org/web/20201202024834/http://download.oracle.com/docs/cd/E19963-01/html/819-0690/chapter8-1.html">Thread-Local Storage access models</a>.</td>
</tr>
<tr class="even">
<td>LD</td>
<td>Local Dynamic (dynamic TLS of local symbols) One of the <a href="https://web.archive.org/web/20201202024834/http://download.oracle.com/docs/cd/E19963-01/html/819-0690/chapter8-1.html">Thread-Local Storage access models</a>.</td>
</tr>
<tr class="odd">
<td>LE</td>
<td>Local Executable (static TLS) One of the <a href="https://web.archive.org/web/20201202024834/http://download.oracle.com/docs/cd/E19963-01/html/819-0690/chapter8-1.html">Thread-Local Storage access models</a>.</td>
</tr>
<tr class="even">
<td><a href="https://web.archive.org/web/20201202024834/http://www.wikipedia.org/wiki/Mach-O">Mach-O</a></td>
<td>Mach object file format</td>
</tr>
<tr class="odd">
<td>PC</td>
<td>Program counter. On x86, this is the same as IP (Instruction Pointer) register.</td>
</tr>
<tr class="even">
<td><a href="https://web.archive.org/web/20201202024834/http://www.wikipedia.org/wiki/Portable_Executable">PE</a></td>
<td>Portable executable</td>
</tr>
<tr class="odd">
<td>PHT</td>
<td>Program header table</td>
</tr>
<tr class="even">
<td><a href="https://web.archive.org/web/20201202024834/http://www.wikipedia.org/wiki/Position_independent_code">PIC</a></td>
<td>Position independent code</td>
</tr>
<tr class="odd">
<td><a href="https://web.archive.org/web/20201202024834/http://www.wikipedia.org/wiki/Position_independent_code">PIE</a></td>
<td>Position independent executable</td>
</tr>
<tr class="even">
<td>PLT</td>
<td>Procedure linkage table</td>
</tr>
<tr class="odd">
<td>REL<br />
RELA</td>
<td>Relocation</td>
</tr>
<tr class="even">
<td>RVA</td>
<td>Relative virtual address</td>
</tr>
<tr class="odd">
<td>SHF</td>
<td>Section header flag</td>
</tr>
<tr class="even">
<td>SHT</td>
<td>Section header table</td>
</tr>
<tr class="odd">
<td>SO</td>
<td>Shared object (another name for dynamic link library)</td>
</tr>
<tr class="even">
<td>VMA</td>
<td>Virtual memory area/address</td>
</tr>
</tbody>
</table>

Useful books and references
---------------------------

[ELF man
page](https://web.archive.org/web/20201202024834/http://manpages.courier-mta.org/htmlman5/elf.5.html)

[System V Application Binary
Interface](https://web.archive.org/web/20201202024834/http://www.sco.com/developers/gabi/latest/contents.html)

[AMD64 System V Application Binary
Interface](https://web.archive.org/web/20201202024834/http://www.x86-64.org/documentation/abi.pdf)

[The gen on function calling
conventions](https://web.archive.org/web/20201202024834/http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/function-calling-conventions.html)

Section II of [Linux Standard Base 4.0 Core
Specification](https://web.archive.org/web/20201202024834/http://refspecs.freestandards.org/LSB_4.0.0/LSB-Core-generic/LSB-Core-generic/book1.html)

*Self-Service Linux: Mastering the Art of Problem Determination* by Mark
Wilding and Dan Behman

[Solaris Linker and Libraries
Guide](https://web.archive.org/web/20201202024834/http://download.oracle.com/docs/cd/E19963-01/html/819-0690/)

[Linkers and
Loaders](https://web.archive.org/web/20201202024834/http://www.iecc.com/linker)
by John Levine

[Understanding Linux ELF RTLD
internals](https://web.archive.org/web/20201202024834/http://s.eresi-project.org/inc/articles/elf-rtld.txt)
by mayhem (this article gives you an idea how the runtime linker `ld.so`
works)

[`ld.so` man
page](https://web.archive.org/web/20201202024834/http://manpages.courier-mta.org/htmlman8/ld.so.8.html)

[Prelink](https://web.archive.org/web/20201202024834/http://www.wikipedia.org/wiki/Prelink)
by Jakub Jelinek (and [prelink man
page](https://web.archive.org/web/20201202024834/http://linux.die.net/man/8/prelink))

Executable and Linkable Format
------------------------------

An ELF executable binary contains at least two kinds of headers: ELF
file header (see `struct Elf32_Ehdr`/`struct Elf64_Ehdr` in
`/usr/include/elf.h`) and one or more Program Headers (see
`struct Elf32_Phdr`/`struct Elf64_Phdr` in `/usr/include/elf.h`)

Usually there is another kind of header called Section Header, which
describe attributes of an ELF section (e.g. `.text`, `.data`, `.bss`,
etc) The Section Header is described by
`struct Elf32_Shdr`/`struct Elf64_Shdr` in `/usr/include/elf.h`

The Program Headers are used during execution (ELF's "**execution
view**"); it tells the kernel or the runtime linker `ld.so` what to load
into memory and how to find dynamic linking information.

The Section Headers are used during compile-time linking (ELF's
"**linking view**"); it tells the link editor `ld` how to resolve
symbols, and how to group similar byte streams from different ELF binary
objects.

Conceptually, the two ELF's "views" are as follows (borrowed from Shaun
Clowes's *Fixing/Making Holes in Binaries* slides):

                  +-----------------+
             +----| ELF File Header |----+
             |    +-----------------+    |
             v                           v
     +-----------------+      +-----------------+
     | Program Headers |      | Section Headers |
     +-----------------+      +-----------------+
          ||                               ||
          ||                               ||
          ||                               ||
          ||   +------------------------+  ||
          +--> | Contents (Byte Stream) |<--+
               +------------------------+

In reality, the layout of a typical ELF executable binary on a disk file
is like this:

        +-------------------------------+
        | ELF File Header               |
        +-------------------------------+
        | Program Header for segment #1 |
        +-------------------------------+
        | Program Header for segment #2 |
        +-------------------------------+
        | ...                           |
        +-------------------------------+
        | Contents (Byte Stream)        |
        | ...                           |
        +-------------------------------+
        | Section Header for section #1 |
        +-------------------------------+
        | Section Header for section #2 |
        +-------------------------------+
        | ...                           |
        +-------------------------------+
        | ".shstrtab" section           |
        +-------------------------------+
        | ".symtab"   section           |
        +-------------------------------+
        | ".strtab"   section           |
        +-------------------------------+

The ELF File Header contains the file offsets of the first Program
Header, the first Section Header, and `.shstrtab` section which contains
the section names (a series of NULL-terminated strings)

The ELF File Header also contains the number of Program Headers and the
number of Section Headers.

Each Program Header describes a "segment": It contains the permissions
(Readable, Writeable, or Executable) , offset of the "segment" (which is
just a byte stream) into the file, and the size of the "segment". The
following table shows the purposes of special segments. Some information
can be found in GNU Binutil's source file
[`include/elf/common.h`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=binutils.git;a=blob_plain;f=include/elf/common.h):

<table data-border="">
<colgroup>
<col style="width: 50%" />
<col style="width: 50%" />
</colgroup>
<thead>
<tr class="header">
<th>ELF Segment</th>
<th>Purpose</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td><code>DYNAMIC</code></td>
<td>For dynamic binaries, this segment hold dynamic linking information and is usually the same as <code>.dynamic</code> section in ELF's linking view. See paragraph below.</td>
</tr>
<tr class="even">
<td><code>GNU_EH_FRAME</code></td>
<td>Frame unwind information (EH = Exception Handling). This segment is usually the same as <code>.eh_frame_hdr</code> section in ELF's linking view.</td>
</tr>
<tr class="odd">
<td><code>GNU_RELRO</code></td>
<td>This segment indicates the memory region which should be made Read-Only after relocation is done. This segment usually appears in a dynamic link library and it contains <code>.ctors</code>, <code>.dtors</code>, <code>.dynamic</code>, <code>.got</code> sections. See paragraph below.</td>
</tr>
<tr class="even">
<td><code>GNU_STACK</code></td>
<td>The permission flag of this segment indicates whether the <a href="https://web.archive.org/web/20201202024834/http://www.gentoo.org/proj/en/hardened/gnu-stack.xml">stack is executable or not</a>. This segment does not have any content; it is just an indicator.</td>
</tr>
<tr class="odd">
<td><code>INTERP</code></td>
<td>For dynamic binaries, this holds the full pathname of runtime linker <code>ld.so</code>
<p>This segement is the same as <code>.interp</code> section in ELF's linking view.</p></td>
</tr>
<tr class="even">
<td><code>LOAD</code></td>
<td><strong>Loadable program segment. Only segments of this type are loaded into memory during execution.</strong></td>
</tr>
<tr class="odd">
<td><code>NOTE</code></td>
<td>Auxiliary information.
<p>For core dumps, this segment contains the status of the process (when the core dump is created), such as the signal (the process received and caused it to dump core), pending &amp; held signals, process ID, parent process ID, user ID, nice value, cumulative user &amp; system time, values of registers (including the program counter!)</p>
<p>For more info, see <code>struct elf_prstatus</code> and <code>struct elf_prpsinfo</code> in Linux kernel source file <a href="https://web.archive.org/web/20201202024834/http://lxr.linux.no/linux/include/linux/elfcore.h"><code>include/linux/elfcore.h</code></a> and <code>struct user_regs_struct</code> in <a href="https://web.archive.org/web/20201202024834/http://lxr.linux.no/linux/arch/x86/include/asm/user_64.h"><code>arch/x86/include/asm/user_64.h</code></a></p></td>
</tr>
<tr class="even">
<td><code>TLS</code></td>
<td>Thread-Local Storage</td>
</tr>
</tbody>
</table>

Likewise, each Section Header contains the file offset of its
corresponding "content" and the size of the "content". The following
table shows the purposes of some special sections. Most information here
comes from [LSB
specification](https://web.archive.org/web/20201202024834/http://refspecs.freestandards.org/LSB_4.0.0/LSB-Core-generic/LSB-Core-generic/specialsections.html).
Some information can be found in GNU Binutil's source file
[`bfd/elf.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=binutils.git;a=blob_plain;f=bfd/elf.c)
(look for `bfd_elf_special_section`) and
[`bfd/elflink.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=binutils.git;a=blob_plain;f=bfd/elflink.c)
(look for double-quoted section names such as `".got.plt"`)

<table data-border="">
<colgroup>
<col style="width: 50%" />
<col style="width: 50%" />
</colgroup>
<thead>
<tr class="header">
<th>ELF Section</th>
<th>Purpose</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td><code>.bss</code></td>
<td>Uninitialized global data ("Block Started by Symbol").
<p>Depending on the compilers, uninitialized global variables could be stored in a nameness section called <code>COMMON</code> (named after Fortran 77's "common blocks".) To wit, consider the following code:</p>
<pre><code>    int globalVar;
    static int globalStaticVar;
    void dummy() {
       static int localStaticVar;
    }
    </code></pre>
Compile with <code>gcc -c</code>, then on x86_64, the resulting object file has the following structure:
<pre><code>    $ objdump -t foo.o

    SYMBOL TABLE:
     ....
    0000000000000000 l     O .bss   0000000000000004 globalStaticVar
    0000000000000004 l     O .bss   0000000000000004 localStaticVar.1619
     ....
    0000000000000004       O *COM*  0000000000000004 globalVar
    </code></pre>
so only the file-scope and local-scope global variables are in the <code>.bss</code> section.
<p>If one wants <code>globalVar</code> to reside in the <code>.bss</code> section, use the <code>-fno-common</code> compiler command-line option. Using <code>-fno-common</code> is encouraged, as the following example shows:</p>
<pre><code>    $ cat foo.c
    int globalVar;
    $ cat bar.c
    double globalVar;
    int main(){}
    $ gcc foo.c bar.c
    </code></pre>
Not only there is no error message about redefinition of the same symbol in both source files (notice we did not use the <code>extern</code> keyword here), there is no complaint about their different data types and sizes either. However, if one uses <code>-fno-common</code>, the compiler will complain:
<pre><code>    /tmp/ccM71JR7.o:(.bss+0x0): multiple definition of `globalVar&#39;
    /tmp/ccIbS5MO.o:(.bss+0x0): first defined here
    ld: Warning: size of symbol `globalVar&#39; changed from 8 in /tmp/ccIbS5MO.o to 4 in /tmp/ccM71JR7.o
    </code></pre></td>
</tr>
<tr class="even">
<td><code>.comment</code></td>
<td>A series of NULL-terminated strings containing compiler information.</td>
</tr>
<tr class="odd">
<td><code>.ctors</code></td>
<td><strong>Pointers</strong> to functions which are marked as <code>__attribute__ ((constructor))</code> as well as static C++ objects' constructors. They will be used by <code>__libc_global_ctors</code> function.
<p>See paragraphs below.</p></td>
</tr>
<tr class="even">
<td><code>.data</code></td>
<td>Initialized data.</td>
</tr>
<tr class="odd">
<td><code>.data.rel.ro</code></td>
<td>Similar to <code>.data</code> section, but this section should be made Read-Only after relocation is done.</td>
</tr>
<tr class="even">
<td><code>.debug_XXX</code></td>
<td>Debugging information (for the programs which are compiled with <code>-g</code> option) which is in the DWARF 2.0 format.
<p>See <a href="https://web.archive.org/web/20201202024834/http://dwarfstd.org/">here</a> for DWARF debugging format.</p></td>
</tr>
<tr class="odd">
<td><code>.dtors</code></td>
<td><strong>Pointers</strong> to functions which are marked as <code>__attribute__ ((destructor))</code> as well as static C++ objects' destructors.
<p>See paragraphs below.</p></td>
</tr>
<tr class="even">
<td><code>.dynamic</code></td>
<td>For dynamic binaries, this section holds dynamic linking information used by <code>ld.so</code>. See paragraphs below.</td>
</tr>
<tr class="odd">
<td><code>.dynstr</code></td>
<td>NULL-terminated strings of names of symbols in <code>.dynsym</code> section.
<p>One can use commands such as <code>readelf -p .dynstr a.out</code> to see these strings.</p></td>
</tr>
<tr class="even">
<td><code>.dynsym</code></td>
<td><strong>Runtime</strong>/Dynamic symbol table. For dynamic binaries, this section is the symbol table of globally visible symbols. For example, if a dynamic link library wants to export its symbols, these symbols will be stored here. On the other hand, if a dynamic executable binary uses symbols from a dynamic link library, then these symbols are stored here too.
<p>The symbol names (as NULL-terminated strings) are stored in <code>.dynstr</code> section.</p></td>
</tr>
<tr class="odd">
<td><code>.eh_frame</code><br />
<code>.eh_frame_hdr</code></td>
<td>Frame unwind information (EH = Exception Handling).
<p>See <a href="https://web.archive.org/web/20201202024834/http://refspecs.freestandards.org/LSB_4.0.0/LSB-Core-generic/LSB-Core-generic/ehframechpt.html">here</a> for details.</p>
<p>To see the content of <code>.eh_frame</code> section, use</p>
<pre><code>readelf --debug-dump=frames-interp a.out</code></pre></td>
</tr>
<tr class="even">
<td><code>.fini</code></td>
<td>Code which will be executed when program exits normally. See paragraphs below.</td>
</tr>
<tr class="odd">
<td><code>.fini_array</code></td>
<td><strong>Pointers</strong> to functions which will be executed when program exits normally. See paragraphs below.</td>
</tr>
<tr class="even">
<td><code>.GCC.command.line</code></td>
<td>A series of NULL-terminated strings containing GCC command-line (that is used to compile the code) options.
<p>This feature is supported since GCC 4.5 and the program must be compiled with <code>-frecord-gcc-switches</code> option.</p></td>
</tr>
<tr class="odd">
<td><code>.gnu.hash</code></td>
<td>GNU's extension to hash table for symbols.
<p>See <a href="https://web.archive.org/web/20201202024834/http://blogs.sun.com/ali/entry/gnu_hash_elf_sections">here</a> for its structure and the hash algorithm.</p>
<p>The link editor <code>ld</code> calls <code>bfd_elf_gnu_hash</code> in in GNU Binutil's source file <a href="https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=binutils.git;a=blob_plain;f=bfd/elf.c"><code>bfd/elf.c</code></a> to compute the hash value.</p>
<p>The runtime linker <code>ld.so</code> calls <code>do_lookup_x</code> in <a href="https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/dl-lookup.c"><code>elf/dl-lookup.c</code></a> to do the symbol look-up. The hash computing function here is <code>dl_new_hash</code>.</p></td>
</tr>
<tr class="even">
<td><code>.gnu.linkonceXXX</code></td>
<td>GNU's extension. It means only a single copy of the section will be used in linking. This is used to by g++. g++ will emit each template expansion in its own section. The symbols will be defined as weak, so that multiple definitions are permitted.</td>
</tr>
<tr class="odd">
<td><code>.gnu.version</code></td>
<td>Versions of symbols.
<p>See <a href="https://web.archive.org/web/20201202024834/http://refspecs.freestandards.org/LSB_4.0.0/LSB-Core-generic/LSB-Core-generic/symversion.html">here</a>, <a href="https://web.archive.org/web/20201202024834/http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/4/html/Using_ld_the_GNU_Linker/version.html">here</a>, <a href="https://web.archive.org/web/20201202024834/http://download.oracle.com/docs/cd/E19963-01/html/819-0690/appendixb-45356.html">here</a>, and <a href="https://web.archive.org/web/20201202024834/http://people.redhat.com/drepper/symbol-versioning">here</a> for details of symbol versioning.</p></td>
</tr>
<tr class="even">
<td><code>.gnu.version_d</code></td>
<td>Version definitions of symbols.</td>
</tr>
<tr class="odd">
<td><code>.gnu.version_r</code></td>
<td>Version references (version needs) of symbols.</td>
</tr>
<tr class="even">
<td><code>.got</code></td>
<td>For dynamic binaries, this Global Offset Table holds the addresses of variables which are relocated upon loading. See paragraphs below.</td>
</tr>
<tr class="odd">
<td><code>.got.plt</code></td>
<td>For dynamic binaries, this Global Offset Table holds the addresses of functions in dynamic libraries. They are used by trampoline code in <code>.plt</code> section. If <code>.got.plt</code> section is present, it contains at least three entries, which have special meanings. See paragraphs below.</td>
</tr>
<tr class="even">
<td><code>.hash</code></td>
<td>Hash table for symbols.
<p>See <a href="https://web.archive.org/web/20201202024834/http://www.sco.com/developers/gabi/latest/ch5.dynamic.html#hash">here</a> for its structure and the hash algorithm.</p>
<p>The link editor <code>ld</code> calls <code>bfd_elf_hash</code> in in GNU Binutil's source file <a href="https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=binutils.git;a=blob_plain;f=bfd/elf.c"><code>bfd/elf.c</code></a> to compute the hash value.</p>
<p>The runtime linker <code>ld.so</code> calls <code>do_lookup_x</code> in <a href="https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/dl-lookup.c"><code>elf/dl-lookup.c</code></a> to do the symbol look-up. The hash computing function here is <code>_dl_elf_hash</code>.</p></td>
</tr>
<tr class="odd">
<td><code>.init</code></td>
<td>Code which will be executed when program initializes. See paragraphs below.</td>
</tr>
<tr class="even">
<td><code>.init_array</code></td>
<td><strong>Pointers</strong> to functions which will be executed when program starts. See paragraphs below.</td>
</tr>
<tr class="odd">
<td><code>.interp</code></td>
<td>For dynamic binaries, this holds the full pathname of runtime linker <code>ld.so</code></td>
</tr>
<tr class="even">
<td><code>.jcr</code></td>
<td>Java class registration information.
<p>Like <code>.ctors</code> section, it contains a list of addresses which will be used by <code>_Jv_RegisterClasses</code> function in CRT (C Runtime) startup files (see <a href="https://web.archive.org/web/20201202024834/http://gcc.gnu.org/viewcvs/trunk/gcc/crtstuff.c?view=markup"><code>gcc/crtstuff.c</code></a> in GCC's source tree)</p></td>
</tr>
<tr class="odd">
<td><code>.note.ABI-tag</code></td>
<td>This Linux-specific section is structured as a <a href="https://web.archive.org/web/20201202024834/http://www.sco.com/developers/gabi/latest/ch5.pheader.html#note_section">note</a> section in ELF specification. Its content is mandated <a href="https://web.archive.org/web/20201202024834/http://refspecs.freestandards.org/LSB_4.0.0/LSB-Core-generic/LSB-Core-generic/noteabitag.html">here</a>.</td>
</tr>
<tr class="even">
<td><code>.note.gnu.build-id</code></td>
<td>A unique build ID. See <a href="https://web.archive.org/web/20201202024834/http://fedoraproject.org/wiki/RolandMcGrath/BuildID">here</a> and <a href="https://web.archive.org/web/20201202024834/http://fedoraproject.org/wiki/Releases/FeatureBuildId">here</a></td>
</tr>
<tr class="odd">
<td><code>.note.GNU-stack</code></td>
<td>See <a href="https://web.archive.org/web/20201202024834/http://www.airs.com/blog/archives/518">here</a></td>
</tr>
<tr class="even">
<td><code>.nvFatBinSegment</code></td>
<td>This segment contains information of nVidia's CUDA fat binary container. Its format is described by <code>struct __cudaFatCudaBinaryRec</code> in <code>__cudaFatFormat.h</code></td>
</tr>
<tr class="odd">
<td><code>.plt</code></td>
<td>For dynamic binaries, this Procedure Linkage Table holds the trampoline/linkage code. See paragraphs below.</td>
</tr>
<tr class="even">
<td><code>.preinit_array</code></td>
<td>Similar to <code>.init_array</code> section. See paragraphs below.</td>
</tr>
<tr class="odd">
<td><code>.rela.dyn</code></td>
<td><strong>Runtime</strong>/Dynamic relocation table.
<p>For dynamic binaries, this relocation table holds information of variables which must be relocated upon loading. Each entry in this table is a <code>struct Elf64_Rela</code> (see <code>/usr/include/elf.h</code>) which has only three members:</p>
<ul>
<li><code>offset</code> (the variable's [usually position-independent] virtual memory address which holds the "patched" value during the relocation process)</li>
<li><code>info</code> (Index into <code>.dynsym</code> section and Relocation Type)</li>
<li><code>addend</code></li>
</ul>
See paragraphs below for details about runtime relocation.</td>
</tr>
<tr class="even">
<td><code>.rela.plt</code></td>
<td><strong>Runtime</strong>/Dynamic relocation table.
<p>This relocation table is similar to the one in <code>.rela.dyn</code> section; the difference is this one is for functions, not variables.</p>
<p>The relocation type of entries in this table is <code>R_386_JMP_SLOT</code> or <code>R_X86_64_JUMP_SLOT</code> and the "offset" refers to memory addresses which are inside <code>.got.plt</code> section.</p>
<p>Simply put, this table holds information to relocate entries in <code>.got.plt</code> section.</p></td>
</tr>
<tr class="odd">
<td><code>.rel.text</code><br />
<code>.rela.text</code></td>
<td><strong>Compile-time</strong>/Static relocation table.
<p>For programs compiled with <code>-c</code> option, this section provides information to the link editor <code>ld</code> where and how to "patch" executable code in <code>.text</code> section.</p>
<p>The difference between <code>.rel.text</code> and <code>.rela.text</code> is entries in the former does not have <code>addend</code> member. (Compare <code>struct Elf64_Rel</code> with <code>struct Elf64_Rela</code> in <code>/usr/include/elf.h</code>) Instead, the addend is taken from the memory location described by <code>offset</code> member.</p>
<p>Whether to use <code>.rel</code> or <code>.rela</code> is platform-dependent. For x86_32, it is <code>.rel</code> and for x86_64, <code>.rela</code></p></td>
</tr>
<tr class="even">
<td><code>.rel.XXX</code><br />
<code>.rela.XXX</code></td>
<td>Compile-time/Static relocation table for other sections. For example, <code>.rela.init_array</code> is the relocation table for <code>.init_array</code> section.</td>
</tr>
<tr class="odd">
<td><code>.rodata</code></td>
<td>Read-only data.</td>
</tr>
<tr class="even">
<td><code>.shstrtab</code></td>
<td>NULL-terminated strings of section names.
<p>One can use commands such as <code>readelf -p .shstrtab a.out</code> to see these strings.</p></td>
</tr>
<tr class="odd">
<td><code>.strtab</code></td>
<td>NULL-terminated strings of names of symbols in <code>.symtab</code> section.
<p>One can use commands such as <code>readelf -p .strtab a.out</code> to see these strings.</p></td>
</tr>
<tr class="even">
<td><code>.symtab</code></td>
<td><strong>Compile-time</strong>/Static symbol table.
<p>This is the main symbol table used in compile-time linking or runtime debugging.</p>
<p>The symbol names (as NULL-terminated strings) are stored in <code>.strtab</code> section.</p>
<p>Both <code>.symtab</code> and <code>.symtab</code> can be stripped away by the <code>strip</code> command.</p></td>
</tr>
<tr class="odd">
<td><code>.tbss</code></td>
<td>Similar to <code>.bss</code> section, but for <em>Thread-Local data</em>. See paragraphs below.</td>
</tr>
<tr class="even">
<td><code>.tdata</code></td>
<td>Similar to <code>.data</code> section, but for <em>Thread-Local data</em>. See paragraphs below.</td>
</tr>
<tr class="odd">
<td><code>.text</code></td>
<td>User's executable code</td>
</tr>
</tbody>
</table>

How is an executable binary in Linux being executed ?
-----------------------------------------------------

First, the operating system must recognize executable binaries. For
example, `zcat /proc/config.gz | grep CONFIG_BINFMT_ELF` can show
whether the Linux kernel is compiled to support ELF executable binary
format (if `/proc/config.gz` does not exist, try
`` /lib/modules/`uname -r`/build/.config ``)

When the shell makes an `execvc` system call to run an executable
binary, the Linux kernel responds as follows (see
[here](https://web.archive.org/web/20201202024834/http://asm.sourceforge.net/articles/startup.html)
and
[here](https://web.archive.org/web/20201202024834/http://s.eresi-project.org/inc/articles/elf-rtld.txt)
for more details) in sequence:

1.  [`sys_execve`](https://web.archive.org/web/20201202024834/http://lxr.linux.no/linux/arch/x86/kernel/process.c#L301)
    function (in `arch/x86/kernel/process.c`) handles the `execvc`
    system call from user space. It calls `do_execve` function.

2.  `do_execve` function (in `fs/exec.c`) opens the executable binary
    file and does some preparation. It calls `search_binary_handler`
    function.

3.  [`search_binary_handler`](https://web.archive.org/web/20201202024834/http://lxr.linux.no/linux/fs/exec.c#L1240)
    function (in `fs/exec.c`) finds out the type of executable binary
    and calls the corresponding handler, which in our case, is
    `load_elf_binary` function.

4.  [`load_elf_binary`](https://web.archive.org/web/20201202024834/http://lxr.linux.no/linux/fs/binfmt_elf.c#L564)
    (in `fs/binfmt_elf.c`) loads the user's executable binary file into
    memory. It allocates memory segments and zeros out the BSS section
    by calling the `padzero` function.

    `load_elf_binary` also examines whether the user's executable binary
    contains an `INTERP` segment or not.

5.  If the executable binary is dynamically linked, then the compiler
    will usually creates an `INTERP` segment (which is usually the same
    as `.interp` section in ELF's "linking view"), which contains the
    full pathname of an "interpreter", usually is the Glibc runtime
    linker
    [ld.so](https://web.archive.org/web/20201202024834/http://linux.die.net/man/8/ld-linux).

    To see this, use command `readelf -p .interp a.out`

    According to [AMD64 System V Application Binary
    Interface](https://web.archive.org/web/20201202024834/http://www.x86-64.org/documentation/abi.pdf),
    the only valid interpreter for programs conforming to AMD64 ABI is
    `/lib/ld64.so.1` and on Linux, GCC usually uses
    `/lib64/ld-linux-x86-64.so.2` or `/lib/ld-linux-x86-64.so.2`
    instead:

        $ gcc -dumpspecs
        ....

        *link:
        ...
          %{!m32:%{!dynamic-linker:-dynamic-linker %{muclibc:%{mglibc:%e-mglibc and -muclibc used
        together}/lib/ld64-uClibc.so.0;:/lib/ld-linux-x86-64.so.2}}}}
        ...

    To change the runtime linker, compile the program using something
    like

        gcc foo.c -Wl,-I/my/own/ld.so

    The [System V Application Binary
    Interface](https://web.archive.org/web/20201202024834/http://www.sco.com/developers/gabi/latest/ch5.dynamic.html)
    specifies, the operating system, instead of running the user's
    executable binary, should run this "interpreter". This interpreter
    should complete the binding of user's executable binary to its
    dependencies.

6.  Thus, if the ELF executable binary file contains an `INTERP`
    segment, `load_elf_binary` will call
    [`load_elf_interp`](https://web.archive.org/web/20201202024834/http://lxr.linux.no/linux/fs/binfmt_elf.c#L383)
    function to load the image of this interpreter as well.

7.  Finally, `load_elf_binary` calls `start_thread` (in
    `arch/x86/kernel/process_64.c`) and passes control to either the
    interpreter or the user program.

What about `ld.so` ?
--------------------

`ld.so` is the runtime linker/loader (the compile-time linker `ld` is
formally called "link editor") for dynamic executables. It provides the
[following
services](https://web.archive.org/web/20201202024834/http://download.oracle.com/docs/cd/E19963-01/html/819-0690/chapter3-1.html):

-   Analyzes the user's executable binary's `DYNAMIC` segment and
    determines what dependencies are required. (See below)
-   Locates and loads these dependencies, analyzes their `DYNAMIC`
    segments to determine if more dependencies are required.
-   Performs any necessary relocations to bind these objects.
-   Calls any initialization functions (see below) provided by these
    dependencies.
-   Passes control to user's executable binary.

Compile your own `ld.so`
------------------------

The internal working of `ld.so` is complex, so you might want to compile
and experiment your own `ld.so`. The source code of `ld.so` can be found
in Glibc. The main files are
[`elf/rtld.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/rtld.c),
[`elf/dl-reloc.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/dl-reloc.c),
and
[`sysdeps/x86_64/dl-machine.h`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=sysdeps/x86_64/dl-machine.h).

[This
link](https://web.archive.org/web/20201202024834/http://www.linuxfromscratch.org/lfs/view/development/chapter05/glibc.html)
provides general tips for building Glibc. Glibc's own
[INSTALL](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=INSTALL)
and
[FAQ](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=FAQ)
documents are useful too.

To compile Glibc (`ld.so` cannot be compiled independently) download and
unpack Glibc source tarball.

-   Make sure the version of Glibc you downloaded is the same as the
    system's current one.

-   Make sure the environmental variable `LD_RUN_PATH` is not set.

-   Read the
    [INSTALL](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=INSTALL)
    and make sure all necessary tool chains (Make, Binutils, etc) are
    up-to-date.

-   Make sure the file system you are doing the compilation is case
    sensitive, or you will see [weird
    errors](https://web.archive.org/web/20201202024834/http://crossgcc.rts-software.org/doku.php?id=i386linuxgccformac)
    like

        /scratch/elf/librtld.os: In function `process_envvars':
        /tmp/glibc-2.x.y/elf/rtld.c:2718: undefined reference to `__open'
        ...

-   `ld.so` should be compiled with the optimization flag on (`-O2` is
    the default). Failing to do so will end up with weird errors (see
    Question 1.23 in
    [FAQ](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=FAQ))

-   Suppose Glibc is unpacked at

        /tmp/glibc-2.x.y/

    Then edit `/tmp/glibc-2.x.y/Makefile.in`: Un-comment the line

        # PARALLELMFLAGS = -j 4

    and change 4 to an appropriate number.

-   Since we are only interested in `ld.so` and not the whole Glibc, we
    only want to build the essential source files needed by `ld.so`. To
    do so, edit `/tmp/glibc-2.x.y/Makeconfig`: Find the line started
    with

        all-subdirs = csu assert ctype locale intl catgets math setjmp signal \
          ...

    and change it to

        all-subdirs = csu elf gmon io misc posix setjmp signal stdlib string time

-   Find a scratch directory, say `/scratch`. Then

        $ cd /scratch
        $ /tmp/glibc-2.x.y/configure --prefix=/scratch --disable-profile
        $ gmake

-   Since we are not building the entire Glibc, when the `gmake` stops
    (probably with some errors), check if `/scratch/elf/ld.so` exists or
    not.

-   `ld.so` is a static binary, which means it has its own
    implementation of standard C routines (e.g. `memcpy`, `strcmp`, etc)
    It has its own `printf`-like routine called `_dl_debug_printf`.

    [`_dl_debug_printf`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/dl-misc.c)
    is not the full-blown `printf` and has very limited capabilities.
    For example, to print the address, one would need to use

        _dl_debug_printf("0x%0*lx\n", (int)sizeof (void*)*2, &foo);

How does `ld.so` work ?
-----------------------

`ld.so`, by its nature, cannot be a dynamic executable itself. The entry
point of `ld.so` is `_start` defined in the macro `RTLD_START` (in
[`sysdeps/x86_64/dl-machine.h`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=sysdeps/x86_64/dl-machine.h)).
`_start` is placed at the beginning of `.text` section, and the default
`ld` script specifies "Entry point address" (in ELF header, use
`readelf -h ld.so|grep Entry` command to see) to be the address of
`_start` (use `ld -verbose | grep ENTRY` command to see). One can set
the entry point to a different address at compile time by [`-e`
option](https://web.archive.org/web/20201202024834/http://sourceware.org/binutils/docs/ld/Entry-Point.html))
so `ld.so` is executed from here. The very first thing it does is to
call `_dl_start` of `elf/rtld.c`. To see this, run gdb on some ELF
executable binary, and do

    (gdb) break _dl_start
    Function "_dl_start" not defined.
    Make breakpoint pending on future shared library load? (y or [n]) y
    Breakpoint 1 (_dl_start) pending.
    (gdb) run
    Starting program: a.out

    Breakpoint 1, 0x0000003433e00fa0 in _dl_start () from /lib64/ld-linux-x86-64.so.2
    (gdb) bt
    #0  0x0000003433e00fa0 in _dl_start () from /lib64/ld-linux-x86-64.so.2
    #1  0x0000003433e00a78 in _start () from /lib64/ld-linux-x86-64.so.2
    #2  0x0000000000000001 in ?? ()
    #3  0x00007fffffffe4f2 in ?? ()
    #4  0x0000000000000000 in ?? ()
    ...
    (gdb) x/10i $pc
       0x3433e00a70 <_start>:       mov    %rsp,%rdi
       0x3433e00a73 <_start+3>:     callq  0x3433e00fa0 <_dl_start>
       0x3433e00a78 <_dl_start_user>:       mov    %rax,%r12
       0x3433e00a7b <_dl_start_user+3>:     mov    0x21b30b(%rip),%eax        # 0x343401bd8c <_dl_skip_args>
    ...

At this breakpoint, we can use `pmap` to see the memory map of a.out,
which would look like this:

    0000000000400000      8K r-x--  a.out
    0000000000601000      4K rw---  a.out
    0000003433e00000    112K r-x--  /lib64/ld-2.5.so
    000000343401b000      8K rw---  /lib64/ld-2.5.so
    00007ffffffea000     84K rw---    [ stack ]
    ffffffffff600000   8192K -----    [ anon ]
     total             8408K

The memory segment of `/lib64/ld-2.5.so` indeed starts at 3433e00000
(page aligned) and this can be verified by running
`readelf -t /lib64/ld-2.5.so`.

If we put another breakpoint at `main` and continue, then when it stops,
the memory map would change to this:

    0000000000400000      8K r-x--  a.out
    0000000000601000      4K rw---  a.out
    0000003433e00000    112K r-x--  /lib64/ld-2.5.so
    000000343401b000      4K r----  /lib64/ld-2.5.so
    000000343401c000      4K rw---  /lib64/ld-2.5.so
    0000003434200000   1336K r-x--  /lib64/libc-2.5.so     <-- The first "LOAD" segment, which contains .text and .rodata sections
    000000343434e000   2044K -----  /lib64/libc-2.5.so     <-- "Hole"
    000000343454d000     16K r----  /lib64/libc-2.5.so     <-- Relocation (GNU_RELRO) info  -+---- The second "LOAD" segment
    0000003434551000      4K rw---  /lib64/libc-2.5.so     <-- .got.plt .data sections      -+
    0000003434552000     20K rw---    [ anon ]             <-- The remaining zero-filled sections (e.g. .bss)
    0000003434e00000     88K r-x--  /lib64/libpthread-2.5.so     <-- The first "LOAD" segment, which contains .text and .rodata sections
    0000003434e16000   2044K -----  /lib64/libpthread-2.5.so     <-- "Hole"
    0000003435015000      4K r----  /lib64/libpthread-2.5.so     <-- Relocation (GNU_RELRO) info  -+---- The second "LOAD" segment
    0000003435016000      4K rw---  /lib64/libpthread-2.5.so     <-- .got.plt .data sections      -+
    0000003435017000     16K rw---    [ anon ]                   <-- The remaining zero-filled sections (e.g. .bss)
    00002aaaaaaab000      4K rw---    [ anon ]
    00002aaaaaac6000     12K rw---    [ anon ]
    00007ffffffea000     84K rw---    [ stack ]
    ffffffffff600000   8192K -----    [ anon ]
     total            14000K

Indeed, `ld.so` has brought in all the required dynamic libraries.

Note that there are two memory regions of 2044KB with null permissions.
As mentioned earlier, the ELF's 'execution view' is concerned with how
to load an executable binary into memory. When `ld.so` brings in the
dynamic libraries, it looks at the segments labelled as `LOAD` (look at
"Program Headers" and "Section to Segment mapping" from
`readelf -a xxx.so` command.) Usually there are two `LOAD` segments, and
there is a "hole" between the two segments (look at the VirtAddr and
MemSiz of these two segments), so `ld.so` will make this hole
inaccessible deliberately: Look for the `PROT_NONE` symbol in
`_dl_map_object_from_fd` in
[`elf/dl-load.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/dl-load.c)

Also note that each of `libc-2.5.so` and `libpthread-2.5.so` has a
read-only memory region (at 0x343454d000 and 0x3435015000,
respectively). This is a for
[`elf/dl-reloc.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/dl-reloc.c).
The `GNU_RELRO` segment is contained in the the second `LOAD` segment,
which contains the following sections (look at "Program Headers" and
"Section to Segment mapping" from `readelf -l xxx.so` command):
`.tdata`, `.fini_array`, `.ctors`, `.dtors`, `__libc_subfreeres`,
`__libc_atexit`, `__libc_thread_subfreeres`, `.data.rel.ro`, `.dynamic`,
`.got`, `.got.plt`, `.data`, and `.bss`. Except for `.got.plt`, `.data`,
and `.bss`, all sections in the the second `LOAD` segment are also in
the `GNU_RELRO` segment, and they are thus made read-only.

The two `[anon]` memory segments at 0x3434552000 and 0x3435017000 are
for sections which do not take space in the ELF binary files. For
example, `readelf -t xxx.so` will show that `.bss` section has `NOBITS`
flag, which means that section takes no disk space. When segments
containing `NOBITS` sections are mapped into memory, `ld.so` allocates
extra memory pages to accomodate these `NOBITS` sections. A `LOAD`
segment is usually structured as a series of **contiguous** sections,
and if a segment contains `NOBITS` sections, these `NOBITS` sections
will be grouped together and placed at the tail of the segment.

So what does `_dl_start` do ?

-   Allocate the initial TLS block and initialize the Thread Pointer if
    needed (these are for `ld.so`'s own, not for the user program)

-   Call `_dl_sysdep_start`, which will call `dl_main`

-   `dl_main` does the majority of the hard work, for example:

    It calls `process_envvars` to handle these `LD_` prefix
    environmental variables such as `LD_PRELOAD`, `LD_LIBRARY_PATH`.

    It examines the `NEEDED` field(s) in the user executable binary's
    `DYNAMIC` segment section (see below) to determine the dependencies.

    It calls `_dl_init_paths` (in
    [`elf/dl-load.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/dl-load.c))
    to initialize the dynamic libraries search paths. According to
    [`ld.so` man
    page](https://web.archive.org/web/20201202024834/http://manpages.courier-mta.org/htmlman8/ld.so.8.html)
    and [this
    page](https://web.archive.org/web/20201202024834/http://blog.lxgcc.net/?tag=dt_runpath),
    the dynamic libraries are searched in the following order:

    1.  The `RPATH` in the `DYNAMIC` segment if there is no `RUNPATH` in
        the `DYNAMIC` segment.

        `RPATH` can be specified when the code is compiled with
        `gcc -Wl,-rpath=...`

        Use of `RPATH` is deprecated because it has an obvious drawback:
        There is no way to override it except using `LD_PRELOAD`
        environmental variable or removing it from the `DYNAMIC`
        segment.

        Both `RPATH` and `RUNPATH` can contain `$ORIGIN` (or
        equivalently `${ORIGIN}`), which will be expanded to the value
        of environmental variable `LD_ORIGIN_PATH` or the full path of
        the loaded object (unless the programs use `setuid` or `setgid`)

    2.  The `LD_LIBRARY_PATH` environmental variable (unless the
        programs use `setuid` or `setgid`)

    3.  The `RUNPATH` in the `DYNAMIC` segment.  
        `RUNPATH` can be specified when the code is compiled with
        `gcc -Wl,-rpath=...,--enable-new-dtags`  
        One can use
        [chrpath](https://web.archive.org/web/20201202024834/http://linux.die.net/man/1/chrpath)
        tool to manipulate `RPATH` and `RUNPATH` settings.

    4.  [`/etc/ld.so.cache`](https://web.archive.org/web/20201202024834/http://manpages.courier-mta.org/htmlman8/ldconfig.8.html)

    5.  `/lib`

    6.  `/usr/lib`

    It calls `_dl_map_object_from_fd` (in
    [`elf/dl-load.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/dl-load.c))
    to load the dynamic libraries, sets up the right read/write/execute
    permissions for the memory segments, (within
    `_dl_map_object_from_fd`, look at calls to `mmap`, `mprotect` and
    symbols such as `PROT_READ`, `PROT_WRITE`, `PROT_EXEC`,
    `PROT_NONE`), **zeroes out BSS sections of dynamic libraries**
    (inside `_dl_map_object_from_fd` function, look at calls to
    `memset`), updates the link map, and performs relocations.

    It calls `_dl_relocate_object` (in
    [`elf/dl-reloc.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/dl-reloc.c))
    to perform **runtime relocations** (see details below).

-   When `_dl_start` returns, it continues to execute code in
    `_dl_start_user` (see
    [`sysdeps/x86_64/dl-machine.h`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=sysdeps/x86_64/dl-machine.h))

-   `_dl_start_user` will call `_dl_init_internal`, which will call
    `call_init` to invoke initialization function of each dynamic
    library loaded.

    Note that `_dl_init_internal` is defined in
    [`elf/dl-init.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/dl-init.c)
    as:

        void
        internal_function
        _dl_init (struct link_map *main_map, int argc, char **argv, char **env)

    `call_init` is also in
    [`elf/dl-init.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/dl-init.c)

-   The initialization function of a dynamic library, say `libfoo.so`,
    is located at the address marked with type "`INIT`" in the output of
    `readelf -d libfoo.so` For Glibc, its initialization function is
    named `_init` (not to be confused with the `_init` inside the user's
    executable binary) and its source code is in
    [`sysdeps/unix/sysv/linux/x86_64/init-first.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=sysdeps/unix/sysv/linux/x86_64/init-first.c).

    `_init` will do the following things:

    -   Save `argc`, `argv`, `envp` to hidden variables `__libc_argc`,
        `__libc_argv`, `__environ`

    -   Call `VDSO_SETUP` to set up Virtaul Dynamic Shared Objects (see
        [here](https://web.archive.org/web/20201202024834/http://www.acsu.buffalo.edu/%7Echarngda/x86assembly.html))
        `VDSO_SETUP` is a platform-dependent macro. For x86\_64, this
        macro is defined as `_libc_vdso_platform_setup` in
        [`sysdeps/unix/sysv/linux/x86_64/init-first.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=sysdeps/unix/sysv/linux/x86_64/init-first.c)

    -   Call `__init_misc` (in
        [`misc/init-misc.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=misc/init-misc.c))
        which saves `argv[0]` to two global variables:
        `program_invocation_name` and `program_invocation_short_name`

    -   Call `__libc_global_ctors` (in
        [`elf/soinit.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/soinit.c))
        to invoke each function listed in the `.ctors` section (see
        below).

        For x86\_64, `.ctors` section contains only one function:
        `init_cacheinfo`

-   At the end of `_dl_start_user`, the control transfers to user
    program's entry point address (use `readelf -h a.out|grep Entry` to
    see) which is usually the initial address of `.text` section and
    contains the entry of a function named `_start`, and in the control
    transfer, the finalizer function `_dl_fini` is passed as an
    argument, and the stack frames are completely clobbered, as if the
    user program is run without any `ld.so` intervention. The latter is
    done by manipulating the stack (see the [on-stack auxiliary
    vector](https://web.archive.org/web/20201202024834/http://articles.manugarg.com/aboutelfauxiliaryvectors.html)
    adjustment code and `HAVE_AUX_VECTOR` in `dl_main`)

Here is the [call graph](https://web.archive.org/web/20201202024834/http://www.acsu.buffalo.edu/%7Echarngda/code/gdb_callgraph/examples/callgraphLDSO.gif), which is worth a thousand words
===========================================================================================================================================================================================

and see
[here](https://web.archive.org/web/20201202024834/http://www.acsu.buffalo.edu/%7Echarngda/callgraph.html)
on how it is generated.

To see `ld.so` in action, set the environmental variable `LD_DEBUG` to
`all` and then run a user program.

The above debugging information does not show `mmap` and `mprotect`
calls. However, we can use `strace`. If we run the user program again
with

    strace -e trace=mmap,mprotect,munmap,open a.out

we should see something like the following:

    mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ae62c0d1000

      .... (a lot of failed attempts to open 'libpthread.so.0' using LD_LIBRARY_PATH)

    open("/etc/ld.so.cache", O_RDONLY) = 3
    mmap(NULL, 104801, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2ae62c0d2000
    open("/lib64/libpthread.so.0", O_RDONLY) = 3
    mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ae62c0ec000
    mmap(0x3434e00000, 2204528, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3434e00000     <-- Bring in the first "LOAD" segment
    mprotect(0x3434e16000, 2093056, PROT_NONE) = 0     <-- Make the "hole" inaccessible
    mmap(0x3435015000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x3435015000     <-- Bring in the second "LOAD" segment
    mmap(0x3435017000, 13168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x3435017000
           (note: 0x3435017000 is the [anon] part which follows immediately after libpthread-2.5.so)
      ...
      .... (a lot of failed attempts to open 'libc.so.6' using LD_LIBRARY_PATH)

    open("/lib64/libc.so.6", O_RDONLY) = 3
    mmap(0x3434200000, 3498328, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3434200000     <-- Bring in the first "LOAD" segment
    mprotect(0x343434e000, 2093056, PROT_NONE) = 0     <-- Make the "hole" inaccessible
    mmap(0x343454d000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14d000) = 0x343454d000     <-- Bring in the second "LOAD" segment
    mmap(0x3434552000, 16728, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x3434552000
           (note: 0x3434552000 is the [anon] part which follows immediately after libc-2.5.so)
    mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ae62c0ed000
    mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ae62c0ee000
    mprotect(0x343454d000, 16384, PROT_READ) = 0    <-- Make the GNU_RELRO segment read-only
    mprotect(0x3435015000, 4096, PROT_READ) = 0     <-- Make the GNU_RELRO segment read-only
    mprotect(0x343401b000, 4096, PROT_READ) = 0
    munmap(0x2ae62c0d2000, 104801)= 0
    mmap(NULL, 10489856, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_32BIT, -1, 0) = 0x40dc7000
    mprotect(0x40dc7000, 4096, PROT_NONE)   = 0
    mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaaaaab000

`.plt` section
--------------

This section contains trampolines for functions defined in dynamic
libraries. A sample disassembly (run the command
`objdump -M intel -dj .plt a.out`) will show the following:

    4003c0 <printf@plt-0x10>:
      4003c0: push   QWORD PTR [RIP+0x2004d2]        # 600898 <_GLOBAL_OFFSET_TABLE_+0x8>
      4003c6: jmp    QWORD PTR [RIP+0x2004d4]        # 6008a0 <_GLOBAL_OFFSET_TABLE_+0x10>
      4003cc: nop    DWORD PTR [RAX+0x0]

    4003d0 <printf@plt>:
      4003d0: jmp    QWORD PTR [RIP+0x2004d2]        # 6008a8 <_GLOBAL_OFFSET_TABLE_+0x18>
      4003d6: push   0
      4003db: jmp    4003c0 <printf@plt-0x10>

    4003e0 <__libc_start_main@plt>:
      4003e0: jmp    QWORD PTR [RIP+0x2004ca]        # 6008b0 <_GLOBAL_OFFSET_TABLE_+0x20>
      4003e6: push   1
      4003eb: jmp    4003c0 <printf@plt-0x10>

The `_GLOBAL_OFFSET_TABLE_` (labeled as `R_X86_64_JUMP_SLOT` and starts
at address 0x600890) is located in `.got.plt` section (to see this, run
the command `objdump -h a.out |grep -A 1 600890` or the command
`readelf -r a.out`) The data in `.got.plt` section look like the
following during runtime (use gdb to see them)

    (gdb) b *0x4003d0
    (gdb) run
    (gdb) x/6a 0x600890
    0x600890: 0x6006e8 <_DYNAMIC>     0x32696159a8
    0x6008a0: 0x326950aa20 <_dl_runtime_resolve>     0x4003d6 <printf@plt+6>
    0x6008b0: 0x326971c3f0 <__libc_start_main>       0x0

When `printf` is called the first time in the user program, the jump at
4003d0 will jump to 4003d6, which is just the next instruction
(`push 0`) The it jumps to 4003c0, which does not have a function name
(so it is shown as `<printf@plt-0x10>`). At 4003c6, it will jumps to
`_dl_runtime_resolve`. This function (in Glibc's source file
[`sysdeps/x86_64/dl-trampoline.S`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=sysdeps/x86_64/dl-trampoline.S))
is a trampoline to `_dl_fixup` (in Glibc's source file
[`elf/dl-runtime.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/dl-runtime.c)).
`_dl_fixup` again, is part of Glibc runtime linker `ld.so`. In
particular, it will change the address stored at 6008a8 to the actual
address of `printf` in `libc.so.6`. To see this, set up a hardware
watchpoint

    (gdb) watch *0x6008a8
    (gdb) cont
    Continuing.
    Hardware watchpoint 2: *0x6008a8

    Old value = 4195286
    New value = 1769244016
    0x000000326950abc2 in fixup () from /lib64/ld-linux-x86-64.so.2

If we continue execution, `printf` will be called, as expected. When
`printf` is called again in the user program, the jump at 4003d0 will
bounce directly to `printf`:

    (gdb) x/6a 0x600890
    0x600890: 0x6006e8 <_DYNAMIC>     0x32696159a8
    0x6008a0: 0x326950aa20 <_dl_runtime_resolve>     0x3269748570 <printf>
    0x6008b0: 0x326971c3f0 <__libc_start_main>       0x0

`.init`, `.fini`, `.preinit_array`, `.init_array` and `.fini_array` sections
----------------------------------------------------------------------------

`.init` and `.fini` sections contain code to do [initialization and
termination](https://web.archive.org/web/20201202024834/http://download.oracle.com/docs/cd/E19963-01/html/819-0690/chapter3-8.html),
as specified by the [System V Application Binary
Interface](https://web.archive.org/web/20201202024834/http://www.sco.com/developers/gabi/latest/ch4.sheader.html#special_sections).
If the code is compiled by GCC, then one will see the following code in
`.init` and `.fini` sections, respectively:

    4003a8 <_init>:
      4003a8: sub    RSP, 8
      4003ac: call   call_gmon_start
      4003b1: call   frame_dummy
      4003b6: call   __do_global_ctors_aux
      4003bb: add    RSP, 8
      4003bf: ret

    400618 <_fini>:
      400618: sub    RSP, 8
      40061c: call   __do_global_dtors_aux
      400621: add    RSP, 8
      400625: ret

There is only one function: `_init`, in `.init` section, and likewise,
only one function: `_fini` in `.fini` section. Both `_init` and `_fini`
are **synthesized** at compile time by the compiler/linker. Glibc
provides its own prolog and epilog for `_init` and `_fini`, but the
compiler is free to choose how to use them and add more code into
`_init` and `_fini`.

In Glibc, the source file
[`sysdeps/generic/initfini.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=sysdeps/generic/initfini.c)
(and some system dependent ones, such as
[`sysdeps/x86_64/elf/initfini.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=sysdeps/x86_64/elf/initfini.c))
is compiled into two files: `/usr/lib64/crti.o` for prolog and
`/usr/lib64/crtn.o` for epilog.

For the compiler part, GCC uses different prolog and epilog files,
depending on the compiler command-line options. To see them, execute
`gcc -dumpspec`, and one can see

    ...

    *endfile:
      %{ffast-math|funsafe-math-optimizations:crtfastmath.o%s}
      %{mpc32:crtprec32.o%s}
      %{mpc64:crtprec64.o%s}
      %{mpc80:crtprec80.o%s}
      %{shared|pie:crtendS.o%s;:crtend.o%s}
      crtn.o%s

    ...

    *startfile:
      %{!shared: %{pg|p|profile:gcrt1.o%s;pie:Scrt1.o%s;:crt1.o%s}}
      crti.o%s
      %{static:crtbeginT.o%s;shared|pie:crtbeginS.o%s;:crtbegin.o%s}

    ...

The detailed explanation of GCC spec file is
[here](https://web.archive.org/web/20201202024834/http://gcc.gnu.org/onlinedocs/gcc/Spec-Files.html).
For above snippet, it means, for example, if compiler command-line
option `-ffast-math` is used, include GCC's `crtfastmath.o` file (this
file can be found under `/usr/lib/gcc/<arch>/<version>/`) at the end of
the linking process. Glibc's `crtn.o` is always included at the end of
linking. The `%s` means this preceding file is a startup file. (GCC
allows to skip startup files during linking using `-nostartfiles`
compiler option)

Similarly, if `-shared` compiler command-line option is not used, then
always include Glibc's `crt1.o` at the start of the linking process.
`crt1.o` contains the function `_start` in `.text` section (not `.init`
section!) `_start` is the function that is executed before anything
else... see below. Next, include Glibc's `crti.o` in the linking.
Finally, include either `crtbeginT.o`, `crtbeginS.o`, or `crtbegin.o`
(both are part of GCC, of course), depending on whether `-static` or
`-shared` (or neither) is used.

So, for example, if a program is compiled using dynamic linking (which
is default), no profiling, no fast math optimizations, then the linking
will include the following files in the following order:

1.  `crt1.o` (part of Glibc)
2.  `crti.o` (part of Glibc and contributes the code at 4003a8, 4003ac,
    400618, and the body of `call_gmon_start`)
3.  `crtbegin.o` (part of GCC and contributes the code at 4003b1 and
    40061c, and the body of `frame_dummy` and `__do_global_dtors_aux`)
4.  user's code
5.  `crtend.o` (part of GCC and contributes the code at 4003b6 and the
    body of `__do_global_ctors_aux`)
6.  `crtn.o` (part of Glibc and contributes the code at 4003bb, 4003bf,
    400621, 400625)

Why `__do_global_ctors_aux` is in `crtend*.o` and
`__do_global_dtors_aux` is in `crtbegin*.o` ? Recall the order of
invocation of destructors should be the reverse order of invocation of
constructors. Therefore, GCC doing so will ensure
`__do_global_ctors_aux` is called as late as possible in `.init` section
and `__do_global_dtors_aux` is called as early as possible in `.fini`
section.

Now back to the `4003a8 <_init>`.

`call_gmon_start` is part of the Glibc prolog `/usr/lib64/crti.o`. It
initializes
[gprof](https://web.archive.org/web/20201202024834/http://sourceware.org/binutils/docs/gprof/)
related data structures.

`frame_dummy` is in GCC code
[`gcc/crtstuff.c`](https://web.archive.org/web/20201202024834/http://gcc.gnu.org/viewcvs/trunk/gcc/crtstuff.c?view=markup)
and it is used to set up excepion handling and Java class registration
(JCR) information.

The most interesting code is `__do_global_ctors_aux` (in GCC's
[`gcc/crtstuff.c`](https://web.archive.org/web/20201202024834/http://gcc.gnu.org/viewcvs/trunk/gcc/crtstuff.c?view=markup)
and
[`gcc/gbl-ctors.h`](https://web.archive.org/web/20201202024834/http://gcc.gnu.org/viewcvs/trunk/gcc/gbl-ctors.h?view=markup))
What it does is to call functions which are marked as
`__attribute__ ((constructor))` (and static C++ objects' constructors)
one by one:

      __SIZE_TYPE__ nptrs = (__SIZE_TYPE__) __CTOR_LIST__[0];
      unsigned i;

      if (nptrs == (__SIZE_TYPE__)-1)
        for (nptrs = 0; __CTOR_LIST__[nptrs + 1] != 0; nptrs++);

      for (i = nptrs; i >= 1; i--)
        __CTOR_LIST__[i] ();

The array `__CTOR_LIST__` is stored in a special section called
`.ctors`. Suppose a function called `foo` is marked as
`__attribute__ ((constructor))`, then the runtime call stack trace would
be

    (gdb) break foo
    (gdb) run
    (gdb) bt
    #0  0x00000000004004d8 in foo ()
    #1  0x0000000000400606 in __do_global_ctors_aux ()
    #2  0x00000000004003bb in _init ()
    #3  0x00000000004005a0 in ?? ()
    #4  0x0000000000400561 in __libc_csu_init ()
    #5  0x000000326971c46f in __libc_start_main ()
    #6  0x000000000040041a in _start ()

Similarly, the `__do_global_dtors_aux` in `_fini` function will invoke
all functions which are marked as `__attribute__ ((destructor))`.
`__do_global_dtors_aux` code is also in GCC's source tree at
[`gcc/crtstuff.c`](https://web.archive.org/web/20201202024834/http://gcc.gnu.org/viewcvs/trunk/gcc/crtstuff.c?view=markup).
If a function called `foo` is marked as `__attribute__ ((destructor))`
(and static C++ objects' destructors), then the runtime call stack trace
would be

    (gdb) bt
    #0  0x0000000000400518 in foo ()
    #1  0x00000000004004ca in __do_global_dtors_aux ()
    #2  0x0000000000400641 in _fini ()
    #3  0x00000032699367e8 in ?? () from /lib64/tls/libc.so.6
    #4  0x0000003269730c95 in exit () from /lib64/tls/libc.so.6
    #5  0x000000326971c4d2 in __libc_start_main () from /lib64/tls/libc.so.6
    #6  0x000000000040045a in _start ()

The array `__DTOR_LIST__` contains the addresses of these destructors
and it is stored in a special section called `.dtors`.

What user functions will be executed before `main` and at program exit?
-----------------------------------------------------------------------

As above call strack trace shows, `_init` is NOT the only function to be
called before `main`. It is `__libc_csu_init` function (in Glibc's
source file
[`csu/elf-init.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=csu/elf-init.c))
that determines what functions to be run before `main` and the order of
running them. Its code is like this

       void __libc_csu_init (int argc, char **argv, char **envp)
       {
       #ifndef LIBC_NONSHARED
         {
           const size_t size = __preinit_array_end - __preinit_array_start;
           size_t i;
           for (i = 0; i < size; i++)
             (*__preinit_array_start [i]) (argc, argv, envp);
         }
       #endif

         _init ();

         const size_t size = __init_array_end - __init_array_start;
         for (size_t i = 0; i < size; i++)
             (*__init_array_start [i]) (argc, argv, envp);
       }

(Symbols such as `__preinit_array_start`, `__preinit_array_end`,
`__init_array_start`, `__init_array_end` are defined by the default `ld`
script; look for [`PROVIDE` and `PROVIDE_HIDDEN`
keywords](https://web.archive.org/web/20201202024834/http://sourceware.org/binutils/docs/ld/PROVIDE.html)
in the output of `ld -verbose` command.)

The `__libc_csu_fini` function has similar code, but what functions to
be executed at program exit are actually determined by `exit`:

        void __libc_csu_fini (void)
        {
        #ifndef LIBC_NONSHARED
          size_t i = __fini_array_end - __fini_array_start;
          while (i-- > 0)
            (*__fini_array_start [i]) ();

          _fini ();
        #endif
        }

To see what's going on, consider the following C code example:

       #include <stdio.h>
       #include <stdlib.h>

       void preinit(int argc, char **argv, char **envp) {
         printf("%s\n", __FUNCTION__);
       }

       void init(int argc, char **argv, char **envp) {
         printf("%s\n", __FUNCTION__);
       }

       void fini() {
         printf("%s\n", __FUNCTION__);
       }

       __attribute__((section(".init_array"))) typeof(init) *__init = init;
       __attribute__((section(".preinit_array"))) typeof(preinit) *__preinit = preinit;
       __attribute__((section(".fini_array"))) typeof(fini) *__fini = fini;

       void  __attribute__ ((constructor)) constructor() {
         printf("%s\n", __FUNCTION__);
       }

       void __attribute__ ((destructor)) destructor() {
         printf("%s\n", __FUNCTION__);
       }

       void my_atexit() {
         printf("%s\n", __FUNCTION__);
       }

       void my_atexit2() {
         printf("%s\n", __FUNCTION__);
       }

       int main() {
         atexit(my_atexit);
         atexit(my_atexit2);
       }

The output will be

       preinit
       constructor
       init
       my_atexit2
       my_atexit
       fini
       destructor

The `.preinit_array` and `.init_array` sections must contain **function
pointers** (NOT code!) The prototype of these functions must be

    void func(int argc,char** argv,char** envp)

`__libc_csu_init` execute them in the following order:

1.  Function pointers in `.preinit_array` section
2.  Functions marked as `__attribute__ ((constructor))`, via `_init`
3.  Function pointers in `.init_array` section

The `.fini_array` section must also contain **function pointers** and
the prototype is like the destructor, i.e. taking no arguments and
returning void. If the program exits **normally**, then the `exit`
function (Glibc source file
[`stdlib/exit.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=stdlib/exit.c))
is called and it will do the following:

1.  In reverse order, functions registered via `atexit` or `on_exit`
2.  Function pointers in `.fini_array` section, via `__libc_csu_fini`
3.  Functions marked as `__attribute__ ((destructor))`, via
    `__libc_csu_fini` (which calls `_fini` after Step 2)
4.  stdio cleanup functions

It is not advisable to put a code in `.init` section, e.g.

    void __attribute__((section(".init"))) foo() {
      ...
    }

because doing so will cause `__do_global_ctors_aux` NOT to be called.
The `.init` section will now look like this:

    4003a0 <_init>:
      4003a0: sub    RSP, 8
      4003a4: call   call_gmon_start
      4003a9: call   frame_dummy

    4003ae <foo>:
      4003ae: push   RBP
      4003af: mov    RBP, RSP

         ....  (foo's body)

      4003b2: leave
      4003b3: ret
      4003b4: call   __do_global_ctors_aux
      4003b9: add    RSP, 8
      4003bd: ret

Now `.init` section contains more than one function, but the epilog of
`_init` is distorted by the insertion of `foo`

Similarly, it is not advisable to put a code in `.fini` section, because
otherwise the code will look like this:

    4006d8 <_fini>:
      4006d8: sub    RSP, 8
      4006dc: call   __do_global_dtors_aux

    4006e1 <foo>:
      4006e1: push   RBP
      4006e2: mov    RBP, RSP

         ....  (foo's body)

      4006ef: leave
      4006f0: ret
      4006f1: add    RSP, 8
      4006f5: ret

Now the epilog of `_fini` is distorted by the insertion of `foo`, so the
stack frame pointer will not be adjusted (`add RSP, 8` is not executed),
causing segmentation fault.

What do `_start` and `__libc_start_main` do?
--------------------------------------------

The above call stack traces show that `_start` calls
`__libc_start_main`, which runs all of the code before `main`.

`_start` is part of Glibc code, as in
[`sysdeps/x86_64/elf/start.S`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=sysdeps/x86_64/elf/start.S).
As mentioned earlier, it is compiled as `/usr/lib64/crt1.o` and is
statically linked to user's executable binary during compilation. To see
this, run gcc with `-v` command, and the last line would be something
like:

    .../collect2 ... /usr/lib64/crt1.o /usr/lib64/crti.o ...  /usr/lib64/crtn.o

`_start` is always placed at the beginning of `.text` section, and the
default `ld` script specifies "Entry point address" (in ELF header, use
`readelf -h ld.so|grep Entry` command to see) to be the address of
`_start` (use `ld -verbose | grep ENTRY` command to see), so `_start` is
guaranteed to be run before anything else. (This is changeable, however,
at compile time one can specify a different initial address by [`-e`
option](https://web.archive.org/web/20201202024834/http://sourceware.org/binutils/docs/ld/Entry-Point.html))

`_start` does only one thing: It sets up the arguments needed by
`__libc_start_main` and then call it. `__libc_start_main`'s source code
is
[`csu/libc-start.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=csu/libc-start.c)
and its function prototype is:

    __libc_start_main (int (*main) (int, char **, char **),
                       int argc,
                       char *argv,
                       int  (*init) (int, char **, char **),
                       void (*fini) (void),
                       void (*rtld_fini) (void),
                       void *stack_end)
                      )

`__libc_start_main` does quite a lot of work in addition to kicking off
`__libc_csu_init`:

1.  Set up `argv` and `envp`

2.  Initialize the thread local storage by calling
    `__pthread_initialize_minimal` (which only calls
    `__libc_setup_tls`).

    `__libc_setup_tls` will initialize Thread Control Block and Dynamic
    Thread Vector.

3.  Set up the thread stack guard

4.  Register the destructor (i.e. the `rtld_fini` argument passed to
    `__libc_start_main`) of the dynamic linker (by calling
    `__cxa_atexit`) if there is any

5.  Initialize Glibc inself by calling `__libc_init_first`

6.  Register `__libc_csu_fini` (i.e. the `fini` argument passed to
    `__libc_start_main`) using `__cxa_atexit`

7.  Call `__libc_csu_init` (i.e. the `init` argument passed to
    `__libc_start_main`)
    1.  Call function pointers in `.preinit_array` section
    2.  Execute the code in `.init` section, which is usually `_init`
        function. What `_init` function does is compiler-specific. For
        GCC, `_init` executes user functions marked as
        `__attribute__ ((constructor))` (in `__do_global_dtors_aux`)
    3.  Call function pointers in `.init_array` section

8.  Set up data structures needed for thread unwinding/cancellation

9.  Call `main` of user's program.

10. Call `exit`

So if the last line of user program's `main` is `return XX`, then the
`XX` will be passed to `exit` at Step \#11 above. If the last line is
not `return XX` or is simply `return`, then the value passed to `exit`
would be undefined.

Of course, if the user program calls `exit` or `abort`, then `exit` will
gets called.

Here is the [call graph](https://web.archive.org/web/20201202024834/http://www.acsu.buffalo.edu/%7Echarngda/code/gdb_callgraph/examples/callgraphEmpty.gif), which is worth a thousand words
============================================================================================================================================================================================

and see
[here](https://web.archive.org/web/20201202024834/http://www.acsu.buffalo.edu/%7Echarngda/callgraph.html)
on how it is generated.

If one tries to build a program which does not contain `main`, then one
should see the following error:

    /usr/lib/crt1.o: In function `_start': (.text+0x20): undefined reference to `main'
    collect2: ld returned 1 exit status

As mentioned earlier, `crt1.o` (part of Glibc) contains the function
`_start`, which will call `__libc_start_main` and pass `main` (a
function pointer) as one of the arguments. If one uses

    nm -u /usr/lib/crt1.o

then it will show `main` is a undefined symbol in `crt1.o`. Now let's
disassemble `crt1.o`:

    $ objdump -M intel -dj .text /usr/lib/crt1.o

    crt1.o:     file format elf64-x86-64

    Disassembly of section .text:

    0000000000000000 <_start>:
       0:   31 ed                   xor    ebp,ebp
       2:   49 89 d1                mov    r9,rdx
       5:   5e                      pop    rsi
       6:   48 89 e2                mov    rdx,rsp
       9:   48 83 e4 f0             and    rsp,0xfffffffffffffff0
       d:   50                      push   rax
       e:   54                      push   rsp
       f:   49 c7 c0 00 00 00 00    mov    r8,0x0
      16:   48 c7 c1 00 00 00 00    mov    rcx,0x0
      1d:   48 c7 c7 00 00 00 00    mov    rdi,0x0
      24:   e8 00 00 00 00          call   29 <_start+0x29>
      29:   f4                      hlt
      ...

Above shows .text+0x20 refers to the 4 bytes of an `mov` instruction.
This means during the linking, the address of `main` should be resolved
and then inserted at the right memory location: .text+0x20. Now let's
cross reference the relocation table:

    $ readelf -p /usr/lib/crt1.o

    Relocation section '.rela.text' at offset 0x410 contains 4 entries:
      Offset          Info           Type           Sym. Value    Sym. Name + Addend
    000000000012  00090000000b R_X86_64_32S      0000000000000000 __libc_csu_fini + 0
    000000000019  000b0000000b R_X86_64_32S      0000000000000000 __libc_csu_init + 0
    000000000020  000c0000000b R_X86_64_32S      0000000000000000 main + 0
    000000000025  000f00000002 R_X86_64_PC32     0000000000000000 __libc_start_main - 4

Above shows where 0x20 comes from.

How to find the address of `main` of an executable binary ?
-----------------------------------------------------------

When an ELF executable binary is stripped off symbolic information, it
is not clear where the `main` is located.

From above analysis, it's possible to find out the address of `main`
(which is NOT the "Entry point address" seen from the output of
`readelf -h a.out | grep Entry` command. "Entry point address" is the
address of `_start`)

Since the address of `main` is the first argument to the call to
`__libc_start_main`, we can extract the value of the first argument as
follows.

On 64-bit x86, the [calling
convention](https://web.archive.org/web/20201202024834/http://www.acsu.buffalo.edu/%7Echarngda/x86assembly.html)
requires that the first argument goes to `RDI` register, so the address
can be extracted by

    objdump -j .text -d a.out | grep -B5 'call.*__libc_start_main' | awk '/mov.*%rdi/ { print $NF }'

On 32-bit x86, the C calling convention
("[cdecl](https://web.archive.org/web/20201202024834/http://www.wikipedia.org/wiki/X86_calling_conventions#cdecl)")
is that the first argument is the last item to be pushed onto the stack
before the call, so the address can be extracted by

    objdump -j .text -d a.out | grep -B2 'call.*__libc_start_main' | awk '/push.*0x/ { print $NF }'

PIC, TLS, and AMD64 code models
-------------------------------

Relocation is the process of connecting symbolic references with
symbolic definitions. The runtime relocation is done by `ld.so`, as in
`elf_machine_rela` function in Glibc's source file
[`sysdeps/x86_64/dl-machine.h`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=sysdeps/x86_64/dl-machine.h).
The link-time relocation is done by the link-editor `ld`, which uses the
relocation table in the object file (`.rela.text` section). Each
symbolic reference has an entry in the relocation table, and each entry
contains three fields: offset, info (relocation type, symbol table
index), and addend. The relocation types are:

| Relocation type            | Meaning                                                                     | Used when              |
|----------------------------|-----------------------------------------------------------------------------|------------------------|
| `R_X86_64_16`              | Direct 16 bit zero extended                                                 |                        |
| `R_X86_64_32`              | Direct 32 bit zero extended                                                 |                        |
| `R_X86_64_32S`             | Direct 32 bit sign extended                                                 |                        |
| `R_X86_64_64`              | Direct 64 bit                                                               | Large code model       |
| `R_X86_64_8`               | Direct 8 bit sign extended                                                  |                        |
| `R_X86_64_COPY`            | Copy symbol at runtime                                                      |                        |
| `R_X86_64_DTPMOD64`        | ID of module containing symbol                                              | TLS                    |
| `R_X86_64_DTPOFF32`        | Offset in TLS block                                                         | TLS                    |
| `R_X86_64_DTPOFF64`        | Offset in module's TLS block                                                | TLS                    |
| `R_X86_64_GLOB_DAT`        | `.got` section, which contains addresses to the actual functions in DLL     |                        |
| `R_X86_64_GOT32`           | 32 bit GOT entry                                                            |                        |
| `R_X86_64_GOT64`           | 64-bit GOT entry offset                                                     | PIC & Large code model |
| `R_X86_64_GOTOFF64`        | 64-bit GOT offset                                                           | PIC & Large code model |
| `R_X86_64_GOTPC32`         | 32-bit PC relative offset to GOT                                            |                        |
| `R_X86_64_GOTPC32_TLSDESC` | 32-bit PC relative to TLS descriptor in GOT                                 | TLS                    |
| `R_X86_64_GOTPC64`         | 64-bit PC relative offset to GOT                                            | PIC & Large code model |
| `R_X86_64_GOTPCREL`        | 32 bit signed PC relative offset to GOT                                     | PIC                    |
| `R_X86_64_GOTPCREL64`      | 64-bit PC relative offset to GOT entry                                      | PIC & Large code model |
| `R_X86_64_GOTPLT64`        | Like GOT64, indicates that PLT entry needed                                 | PIC & Large code model |
| `R_X86_64_GOTTPOFF`        | 32 bit signed PC relative offset to GOT entry for IE symbol                 | TLS                    |
| `R_X86_64_JUMP_SLOT`       | `.got.plt` section, which contains addresses to the actual functions in DLL | DLL                    |
| `R_X86_64_PC16`            | 16 bit sign extended PC relative                                            |                        |
| `R_X86_64_PC32`            | PC relative 32 bit signed                                                   |                        |
| `R_X86_64_PC64`            | 64-bit PC relative                                                          | Large code model       |
| `R_X86_64_PC8`             | 8 bit sign extended PC relative                                             |                        |
| `R_X86_64_PLT32`           | 32 bit PLT address                                                          |                        |
| `R_X86_64_PLTOFF64`        | 64-bit GOT relative offset to PLT entry                                     | PIC & Large code model |
| `R_X86_64_RELATIVE`        | Adjust by program base                                                      |                        |
| `R_X86_64_SIZE32`          |                                                                             |                        |
| `R_X86_64_SIZE64`          |                                                                             |                        |
| `R_X86_64_TLSDESC`         | 2 by 64-bit TLS descriptor                                                  | TLS                    |
| `R_X86_64_TLSDESC_CALL`    | Relaxable call through TLS descriptor                                       | TLS                    |
| `R_X86_64_TLSGD`           | 32 bit signed PC relative offset to two GOT entries for GD symbol           | TLS & PIC              |
| `R_X86_64_TLSLD`           | 32 bit signed PC relative offset to two GOT entries for LD symbol           | TLS                    |
| `R_X86_64_TPOFF32`         | Offset in initial TLS block                                                 | TLS                    |
| `R_X86_64_TPOFF64`         | Offset in initial TLS block                                                 | TLS & Large code model |

According to Chapter 3.5 of [AMD64 System V Application Binary
Interface](https://web.archive.org/web/20201202024834/http://www.x86-64.org/documentation/abi.pdf),
there are four code models and they differ in addressing modes (absolute
versus relative):

-   **Small**: All compile- and link-time addresses and symbols are
    assumed to fit in 32-bit immediate operands. This model restricts
    code and global data to the low 2 GB of the address space (to be
    exact, between 0x0 and 0x7eff ffff, which is 2031 MB)

    The compiler can encode symbolic references

    -   In sign-extended immediate operands for offsets in the range of
        0x8000 0000 (-2<sup>31</sup>) to 0x100 0000 (2<sup>24</sup>)
    -   In zero-extended immediate operands for offsets in the range of
        0x0 to 0x7f00 0000 (2<sup>31</sup> - 2<sup>24</sup>)
    -   In [RIP relative addressing
        mode](https://web.archive.org/web/20201202024834/http://www.tortall.net/projects/yasm/manual/html/nasm-effaddr.html)
        for offsets in the range 0xff00 0000 (-2<sup>24</sup> = -16 MB)
        to 0x100 0000 (2<sup>24</sup> = 16 MB)

    This mode is the default mode for most compilers.

-   **Large**: No restrictions are placed on the size or placement of
    code and data. The max virtual memory space is 48 bits (256 TB).

-   **Medium**: Like the Small code model, except the data sections are
    split into two parts, e.g. instead of having just `.data`,
    `.rodata`, `.bss` sections, there would also be `.ldata`,
    `.lrodata`, `.lbss` sections. The smaller `.data`, etc are still the
    same as in the Small code model, and the larger `.ldata`, etc are as
    in the Large code model.

-   **Kernel**: Like the Small code model, but the 2 GB address space
    spans from 0xffff ffff 8000 0000 (2<sup>64</sup>-2<sup>31</sup>) to
    0xffff ffff ff00 0000 (2<sup>64</sup>-2<sup>24</sup>) Because of
    this, the offsets which can be encoded using sign-extended and
    zero-extended immediate operands change.

Now consider the following C code

    extern int esrc[100];
           int gsrc[100];
    static int ssrc[100];

    void foo() {
      int k;
      k = esrc[5];
      k = gsrc[5];
      k = ssrc[5];
    }

-   **Small** code model, no PIC (i.e. compiled with just `gcc -c`):

        k = esrc[5];     mov EAX, DWORD PTR[RIP+0x0]
                         mov DWORD PTR[RBP-0x4], EAX
        k = gsrc[5];     mov EAX, DWORD PTR[RIP+0x0]
                         mov DWORD PTR[RBP-0x4], EAX
        k = ssrc[5];     mov EAX, DWORD PTR[RIP+0x0]
                         mov DWORD PTR[RBP-0x4], EAX

    and the relocation table is (use `readelf -r foo.o` command)

        type           Sym. Name + Addend
        R_X86_64_PC32  esrc + 10
        R_X86_64_PC32  gsrc + 10
        R_X86_64_PC32  .bss + 10

    All of the 0x0's in the generated assembly will be filled at
    link-time with their relative offsets in respective sections, as
    indicated by the relocation table.

-   **Large** code model, no PIC (i.e. compiled with
    `gcc -c -mcmodel=large`)

        k = esrc[5];     mov RAX, 0x0
                         mov EAX, DWORD PTR[RAX+0x10]
                         mov DWORD PTR[RBP-0x4], EAX
        k = gsrc[5];     mov RAX, 0x0
                         mov EAX, DWORD PTR[RAX+0x10]
                         mov DWORD PTR[RBP-0x4], EAX
        k = ssrc[5];     mov RAX, 0x0
                         mov EAX, DWORD PTR[RAX+0x10]
                         mov DWORD PTR[RBP-0x4], EAX

    and the relocation table is:

        type         Sym. Name + Addend
        R_X86_64_64  esrc + 0
        R_X86_64_64  gsrc + 0
        R_X86_64_64  .bss + 0

    All of the 0x0's in the generated assembly will be filled at
    link-time with their (64-bit) absolute addresses.

-   **Small** code model, PIC (i.e. compiled with `gcc -c -fPIC`. Note
    that adding `-shared` or not will not affect the generated code)

        k = esrc[5];     mov RAX, QWORD PTR[RIP+0x0]
                         mov EAX, DWORD PTR[RAX+0x10]
                         mov DWORD PTR[RBP-0x4], EAX
        k = gsrc[5];     mov RAX, QWORD PTR[RIP+0x0]
                         mov EAX, DWORD PTR[RAX+0x10]
                         mov DWORD PTR[RBP-0x4], EAX
        k = ssrc[5];     mov EAX, DWORD PTR[RIP+0x0]
                         mov DWORD PTR[RBP-0x4], EAX

    and the relocation table is:

        type              Sym. Name + Addend
        R_X86_64_GOTPCREL esrc - 4
        R_X86_64_GOTPCREL gsrc - 4
        R_X86_64_PC32     .bss + 10

    The first two 0x0's in the generated assembly will be filled with
    the relative offset of `_GLOBAL_OFFSET_TABLE_` (i.e. the `.got.plt`
    section)

-   **Large** code model, PIC (i.e. compiled with
    `gcc -c -fPIC -mcmodel=large`)

                         lea RBX, [RIP-0x7]
                         mov R11, 0x0
                         add RBX, R11
        k = esrc[5];     mov RAX, 0x0
                         mov RAX, QWORD PTR[RBX+RAX*1]
                         mov EAX, DWORD PTR[RAX+0x10]
                         mov DWORD PTR[RBP-0x4], EAX
        k = gsrc[5];     mov RAX, 0x0
                         mov RAX, QWORD PTR[RBX+RAX*1]
                         mov EAX, DWORD PTR[RAX+0x10]
                         mov DWORD PTR[RBP-0x4], EAX
        k = ssrc[5];     mov RAX, 0x0
                         mov RAX, QWORD PTR[RBX+RAX*1]
                         mov EAX, DWORD PTR[RAX+0x10]
                         mov DWORD PTR[RBP-0x4], EAX

    The first 0x0 is in the generated assembly will be filled with the
    absolute address of `_GLOBAL_OFFSET_TABLE_`

`_GLOBAL_OFFSET_TABLE_`, `.got.plt` section, and `DYNAMIC` segment
------------------------------------------------------------------

Earlier we see that the `_GLOBAL_OFFSET_TABLE_` is located in `.got.plt`
section:

    (gdb) b *0x4003d0
    (gdb) run
    (gdb) x/6a 0x600890
    0x600890: 0x6006e8 <_DYNAMIC>     0x32696159a8
    0x6008a0: 0x326950aa20 <_dl_runtime_resolve>     0x4003d6 <printf@plt+6>
    0x6008b0: 0x326971c3f0 <__libc_start_main>       0x0

According to Chapter 5.2 of [AMD64 System V Application Binary
Interface](https://web.archive.org/web/20201202024834/http://www.x86-64.org/documentation/abi.pdf),
the first 3 entries of this table are reserved for special purposes. The
first entry is set up during compilation by the link editor `ld`. The
second and third entries are set up during runtime by the runtime linker
`ld.so` (see function `_dl_relocate_object` in Glibc source file
[`elf/dl-reloc.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/dl-reloc.c)
and in particular, notice the `ELF_DYNAMIC_RELOCATE` macro, which calls
function `elf_machine_runtime_setup` in
[`sysdeps/x86_64/dl-machine.h`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=sysdeps/x86_64/dl-machine.h))

The first entry `_DYNAMIC` has value 6006e8, and this is exactly the
starting address of `.dynamic` section (or `DYNAMIC` segment, in ELF's
"execution view".) The runtime linker `ld.so` uses this section to find
the all necessary information needed for runtime relocation and dynamic
linking.

To see `DYNAMIC` segment's content, use `readelf -d a.out` command, or
`objdump -x a.out`, or just use `x/50a 0x6006e8` in gdb. The
`readelf -d a.out` command will show something like this:

    Dynamic section at offset 0x6e8 contains 21 entries:
      Tag        Type                 Name/Value
     0x0000000000000001 (NEEDED)     Shared library: [libc.so.6]  <-- dependent dynamic library name
     0x000000000000000c (INIT)       0x4003a8     <-- address of .init section
     0x000000000000000d (FINI)       0x400618     <-- address of .fini section
     0x0000000000000004 (HASH)       0x400240     <-- address of .hash section
     0x000000006ffffef5 (GNU_HASH)   0x400268     <-- address of .gnu.hash section
     0x0000000000000005 (STRTAB)     0x4002e8     <-- address of .strtab section
     0x0000000000000006 (SYMTAB)     0x400288     <-- address of .symtab section
     0x000000000000000a (STRSZ)      63 (bytes)   <-- size of .strtab section
     0x000000000000000b (SYMENT)     24 (bytes)   <-- size of an entry in .symtab section
     0x0000000000000015 (DEBUG)      0x0          <-- see below
     0x0000000000000003 (PLTGOT)     0x600860     <-- address of .got.plt section
     0x0000000000000002 (PLTRELSZ)   48 (bytes)   <-- total size of .rela.plt section
     0x0000000000000014 (PLTREL)     RELA         <-- RELA or REL ?
     0x0000000000000017 (JMPREL)     0x400368     <-- address of .rela.plt section
     0x0000000000000007 (RELA)       0x400350     <-- address of .rela.dyn section
     0x0000000000000008 (RELASZ)     24 (bytes)   <-- total size of .rela.dyn section
     0x0000000000000009 (RELAENT)    24 (bytes)   <-- size of an entry in .rela.dyn section
     0x000000006ffffffe (VERNEED)    0x400330     <-- address of .gnu.version_r section
     0x000000006fffffff (VERNEEDNUM) 1            <-- number of needed versions
     0x000000006ffffff0 (VERSYM)     0x400328     <-- address of .gnu.version section
     0x0000000000000000 (NULL)       0x0          <-- marks the end of .dynamic section

Each entry in `DYNAMIC` segment is a struct of only two members: "tag"
and "value". The `NEEDED`, `INIT` ... above are "tags" (see
`/usr/include/elf.h`)

Other tags of interest are:

    BIND_NOW           The same as BIND_NOW in FLAGS. This has been superseded by
                       BIND_NOW in FLAGS

    CHECKSUM           The checksum value used by prelink.

    DEBUG              At runtime ld.so will fill its value with the runtime
                       address of r_debug structure (see elf/rtld.c)
                       and this info is used by GDB (see elf_locate_base function
                       in GDB's source tree).

    FINI               Address of .fini section
    FINI_ARRAY         Address of .fini_array section
    FINI_ARRAYSZ       Size of .fini_array section

    FLAGS              Additional flags, such as BIND_NOW, STATIC_TLS, TEXTREL..

    FLAGS_1            Additional flags used by Solaris, such as NOW (the same as BIND_NOW), INTERPOSE..

    GNU_PRELINKED      The timestamp string when the binary object is last prelinked.

    INIT               Address of .init section
    INIT_ARRAY         Address of .init_array section
    INIT_ARRAYSZ       Size of .init_array section

    INTERP             Address of .interp section

    PREINIT_ARRAY      Address of .preinit_array section
    PREINIT_ARRAYSZ    Size of .preinit_array section

    RELACOUNT          Number of R_X86_64_RELATIVE entries in RELA segment (.rela.dyn
                       section)

    RPATH              Dynamic library search path, which has higher precendence than
                       LD_LIBRARY_PATH. RPATH is ignored if RUNPATH is present.

                       Use of RPATH is deprecated.

                       When one uses "gcc -Wl,-rpath=... " to build binaries, the info
                       is stored here.

    RUNPATH            Dynamic library search path, which has lower precendence than
                       LD_LIBRARY_PATH.

                       When one uses "gcc -Wl,-rpath=...,--enable-new-dtags"
                       to build binaries, the info is stored here.
                       (See here for details.)

                       One can use chrpath
                       tool to manipulate RPATH and RUNPATH settings.


    SONAME             Shared object (i.e. dynamic library) name. When one uses
                       "gcc -Wl,-soname=... " to build binaries, the info is
                       stored here.

    TEXTREL            Relocation might modify .text section.

    VERDEF             Address of .gnu.version_d section
    VERDEFNUM          Number of version definitions.

Runtime Relocation
------------------

After exploring `DYNAMIC` segment, we can explain how `ld.so` performs
runtime relocation.

First, before `ld.so` loads all dependent libraries of a dynamic
executable, it needs to run its own relocation! Even if `ld.so` is a
statically-linked binary, it also has a `DYNAMIC` segment and thus
`PLTREL` (`.rela.dyn` section) and `JMPREL` (`.rela.plt` section) tags:

    $ readelf -a `readelf -p .interp /bin/sh | awk '/ld/ {print $3}'`

     ....

    Dynamic section at offset 0x14e18 contains 22 entries:
      Tag        Type                         Name/Value
     0x000000000000000e (SONAME)             Library soname: [ld-linux-x86-64.so.2]
     0x0000000000000004 (HASH)               0x3269500190
     0x0000000000000005 (STRTAB)             0x3269500578
     0x0000000000000006 (SYMTAB)             0x3269500260
     0x000000000000000a (STRSZ)              388 (bytes)
     0x000000000000000b (SYMENT)             24 (bytes)
     0x0000000000000003 (PLTGOT)             0x3269614f98
     0x0000000000000002 (PLTRELSZ)           120 (bytes)
     0x0000000000000014 (PLTREL)             RELA
     0x0000000000000017 (JMPREL)             0x32695009a0
     0x0000000000000007 (RELA)               0x32695007c0
     0x0000000000000008 (RELASZ)             480 (bytes)
     0x0000000000000009 (RELAENT)            24 (bytes)
     0x000000006ffffffc (VERDEF)             0x3269500740
     0x000000006ffffffd (VERDEFNUM)          4
     0x0000000000000018 (BIND_NOW)
     0x000000006ffffffb (FLAGS_1)            Flags: NOW
     0x000000006ffffff0 (VERSYM)             0x32695006fc
     0x000000006ffffff9 (RELACOUNT)          19
     0x000000006ffffdf8 (CHECKSUM)           0x4c4e099e
     0x000000006ffffdf5 (GNU_PRELINKED)      2010-08-26T08:13:28
     0x0000000000000000 (NULL)               0x0

    Relocation section '.rela.dyn' at offset 0x7c0 contains 20 entries:
      Offset          Info           Type           Sym. Value    Sym. Name + Addend
    003269614cf0  000000000008 R_X86_64_RELATIVE                    000000326950dd80
      ....
    003269615820  000000000008 R_X86_64_RELATIVE                    0000003269501140
    003269614fe0  001e00000006 R_X86_64_GLOB_DAT 0000003269615980 _r_debug + 0

    Relocation section '.rela.plt' at offset 0x9a0 contains 5 entries:
      Offset          Info           Type           Sym. Value    Sym. Name + Addend
    003269614fb0  000b00000007 R_X86_64_JUMP_SLO 000000326950f1b0 __libc_memalign + 0
    003269614fb8  000c00000007 R_X86_64_JUMP_SLO 000000326950f2b0 malloc + 0
    003269614fc0  001200000007 R_X86_64_JUMP_SLO 000000326950f2c0 calloc + 0
    003269614fc8  001800000007 R_X86_64_JUMP_SLO 000000326950f340 realloc + 0
    003269614fd0  002000000007 R_X86_64_JUMP_SLO 000000326950f300 free + 0

Note that the `ld.so` is prelinked. On Fedora and Red Hat Enterprise
Linux (RHEL) systems, [prelink is run every two
weeks](https://web.archive.org/web/20201202024834/http://lwn.net/Articles/341244/).
To see if your Linux has similar setup, check `/etc/sysconfig/prelink`
and `/etc/prelink.conf`

**What does this prelink do**? It changes the base address of a dynamic
library to the actual address in the user program's address space when
it is loaded into memory. Of course, `ld.so` recognizes `GNU_PRELINKED`
tag and will load a dynamic library to its this base address (recall the
first argument of `mmap` is the preferred address; of course, this is
subject to the operating system.)

Normally, a dynamic library is built as [position independent
code](https://web.archive.org/web/20201202024834/http://www.wikipedia.org/wiki/Position_independent_code),
i.e. the `-fPIC` compiler command-line option, and thus the base address
is 0. For example, a normal libc.so has ELF program header as follows
(`readelf -l` command):

    Program Headers:
      Type  Offset             VirtAddr           PhysAddr
            FileSiz            MemSiz              Flags  Align
      LOAD  0x0000000000000000 0x0000000000000000 0x0000000000000000
            0x0000000000179058 0x0000000000179058  R E    200000
      LOAD  0x0000000000179730 0x0000000000379730 0x0000000000379730
            0x0000000000004668 0x00000000000090f8  RW     200000
       ....

And when calling `mmap` with address 0 (i.e. NULL) the operating system
can choose any address it feels appropriate.

A prelinked one, on the other hand, has its ELF program header as
follows:

    Program Headers:
      Type Offset             VirtAddr           PhysAddr
           FileSiz            MemSiz              Flags  Align
      LOAD 0x0000000000000000 0x0000003433e00000 0x0000003433e00000
           0x000000000001bb80 0x000000000001bb80  R E    200000
      LOAD 0x000000000001bb90 0x000000343401bb90 0x000000343401bb90
           0x0000000000000f58 0x00000000000010f8  RW     200000

**What is the advantage of prelinking**? `ld.so` will not process
`R_X86_64_RELATIVE` relocation types since they are already in the
"right" place in user program's address space. The extra benefit of this
is the memory regions which `ld.so` would have written to (if
`R_X86_64_RELATIVE` needs processing) will not incur any Copy-On-Writes
and thus can be made Read-Only.

According to [this
post](https://web.archive.org/web/20201202024834/http://lwn.net/Articles/341309/),
for GUI programs, which tend to link against dozens of dynamic libraries
and use lengthy C++ demangled names, the speed up can be an order of
magnitude.

**How to disable prelinking at runtime**? Run the user program with
`LD_USE_LOAD_BIAS` environmental variable set to 0.

**How does `ld.so` process its own relocation**?

The relocation is done by `_dl_relocate_object` function in Glibc's
[`elf/dl-reloc.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/dl-reloc.c),
which will call `elf_machine_rela` function in
[`sysdeps/x86_64/dl-machine.h`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=sysdeps/x86_64/dl-machine.h)
to do the majority of work.

First to be processed is the `.rela.dyn` relocation table, which
contains a bunch of `R_X86_64_RELATIVE` types and one
`R_X86_64_GLOB_DAT` type (the variable `_r_debug`)

If prelink is used, i.e. `ld.so` is indeed loaded to the desired
address, then `R_X86_64_RELATIVE` relocation types will be ignored. If
not, then the address calculation for `R_X86_64_RELATIVE` types is

    Base Address + Value Stored at [Base Address + Offset]

For example, in `ld.so`'s case, its base address is 2a95556000 (can be
obtained from `pmap` command; inside `ld.so`, it calls
`elf_machine_load_address` function to get this value)

    0000400000    4K r-x--  /tmp/a.out
    0000500000    4K rw---  /tmp/a.out
    2a95556000   92K r-x--  /lib64/ld.so
    2a9556d000    8K rw---  [ anon ]
    2a95599000    4K rw---  [ anon ]
    2a9566c000    4K r----  /lib64/ld.so
    2a9566d000    4K rw---  /lib64/ld.so
    3269700000 1216K r-x--  /lib64/libc-2.3.4.so
    ...

And `ld.so`'s `.rela.dyn` relocation table is (no prelinked! If `ld.so`
is prelinked, the offset will be in a much higher address)

    Relocation section '.rela.dyn' at offset 0x7c0 contains 20 entries:
      Offset          Info           Type           Sym. Value    Sym. Name + Addend
    000000116d50  000000000008 R_X86_64_RELATIVE                    000000000000e250
    ...

so the relocation for 000000116d50 is processed as

    0x2a95556000 + *(0x116d50+0x2a95556000)

and this new value is stored at 0x2a9566cd50 (=0x116d50+0x2a95556000)

As `R_X86_64_RELATIVE` types do not require symbol lookups, they are
handled in a tight loop in `elf_machine_rela_relative` function in
[`sysdeps/x86_64/dl-machine.h`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=sysdeps/x86_64/dl-machine.h)

**Any relocation types other than `R_X86_64_RELATIVE` need to go through
symbol resolution first.**

So what about `R_X86_64_GLOB_DAT` relocation type in `ld.so` ? First,
`RESOLVE_MAP` (a macro defined within
[`elf/dl-reloc.c`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/dl-reloc.c))
is called (with r\_type = `R_X86_64_GLOB_DAT`) to find out which ELF
binary (could be the user's program or its dependent dynamic libraries)
contains this symbol. Then `R_X86_64_GLOB_DAT` relocation type is
calculated as

    Base Address + Symbol Value + Addend

where `Base Address` is the base address of ELF binary which contains
the symbol, and `Symbol Value` is the symbol value from the symbol table
of ELF binary which contains the symbol.

So for `ld.so`,

    Relocation section '.rela.dyn' at offset 0x7c0 contains 20 entries:
      Offset          Info           Type           Sym. Value    Sym. Name + Addend
      ....
    000000116fe0  001e00000006 R_X86_64_GLOB_DAT 00000000001179c0 _r_debug + 0

The relocation for 000000116fe0 is processed as

    0x2a95556000 + 0x1179c0 + 0

because `ld.so` determines `_r_debug` can be found from itself. The
calculated value is stored at 0x2a9566cfe0 (=0x116fe0+0x2a95556000).

The next to be processed by `ld.so` is its own `.rela.plt` relocation
table, which contains a bunch of `R_X86_64_JUMP_SLOT` types. This
reloction type is handled exactly the same way as `R_X86_64_GLOB_DAT`.

After `ld.so` finishes its own relocation, it loads user program's
dependent libraries and process their relocations one by one. First,
`ld.so` handles `libc.so`'s relocation. `libc.so` has two relocation
types we have not covered so far: `R_X86_64_64` and `R_X86_64_TPOFF64`.

`R_X86_64_64` relocation type is processed by first looking up the
symbol's runtime **absolute** address, and then calculating

    Absolute Address + Addend

And the `R_X86_64_TPOFF64` relocation type is calculated as

    Symbol Value + Addend - TLS Offset

which usually results in a negative value.

`R_X86_64_COPY` relocation type
-------------------------------

`R_X86_64_COPY` relocation type is used when a dynamic binary refers to
an **initialized** global variable (not a function!) defined in a
dynamic link library. Unlike functions, for variables, there is no lazy
binding, and the trampoline trick used in `.plt` section does not work.
Instead, the global variable will actually be allocated in dynamic
binary's `.bss` section.

To see how `R_X86_64_COPY` relocation type works, consider the following
two code:

    foo.c

        int foo=4;

        void foo_access() {
          foo=5;
        }

    bar.c

        #include <stdio.h>
        extern int foo;

        int main() {
           printf("foo=%d\n",foo);
        }

Now compile them as follows:

    $ gcc -shared -fPIC -Wl,-soname=libfoo.so foo.c -o /tmp/libfoo.so
    $ gcc bar.c -o bar -L/tmp -lfoo

And run them as

    $ LD_PRELOAD=/tmp/libfoo.so ./bar

Before explaining what happened during runtime, we need to examine the
binaries first.

The `foo_access` in `libfoo.so` is like this:

    69c <foo_access>:
     69c:  push   rbp
     69d:  mov    rbp,rsp
     6a0:  mov    rax,QWORD PTR [rip+0x100269]  # 100910 <_DYNAMIC+0x198>
     6a7:  mov    DWORD PTR [rax],0x5
     6ad:  leave
     6ae:  ret

So for `libfoo.so`, the **address** of variable `foo` is in its `.got`
section, not `.data` section:

    $ readelf -a /tmp/libfoo.so

    Section Headers:
      [Nr] Name              Type             Address           Offset
           Size              EntSize          Flags  Link  Info  Align
    ...
      [18] .got              PROGBITS         0000000000100908  00000908
           0000000000000020  0000000000000008  WA       0     0     8
      [19] .got.plt          PROGBITS         0000000000100928  00000928
           0000000000000020  0000000000000008  WA       0     0     8
    ...
      [20] .data             PROGBITS         0000000000100948  00000948
           0000000000000014  0000000000000000  WA       0     0     8
    ...

    Relocation section '.rela.dyn' at offset 0x520 contains 6 entries:
      Offset          Info           Type           Sym. Value    Sym. Name + Addend
    000000100948  000000000008 R_X86_64_RELATIVE                    0000000000100948
    000000100950  000000000008 R_X86_64_RELATIVE                    0000000000100768
    000000100908  000f00000006 R_X86_64_GLOB_DAT 0000000000000000 __cxa_finalize + 0
    000000100910  001100000006 R_X86_64_GLOB_DAT 0000000000100958 foo + 0
    ....

But what about the address 0x100958 ? This address is in `libfoo.so`'s
`.data` section! Well, 0x100958 has the initial value of `foo` (in our
example, 4) At runtime, `ld.so` will copy this value to `bar`'s `.bss`
section:

    $ objdump -sj .data libfoo.so

    libfoo.so:     file format elf64-x86-64

    Contents of section .data:
     100948 48091000 00000000 68071000 00000000  H.......h.......
     100958 04000000                             ....

Next, disassemble the `main` function of `bar`:

    4005f8 <main>:
      4005f8:  push   rbp
      4005f9:  mov    rbp,rsp
      4005fc:  mov    esi,DWORD PTR [rip+0x1003de] # 5009e0 <__bss_start>
      400602:  mov    edi,0x40070c
      400607:  mov    eax,0x0
      40060c:  call   400528 <printf@plt>
      400611:  leave
      400612:  ret

So the variable `foo` is indeed located in `bar`'s `.bss` section. Let's
double check with `nm`:

    $ nm -n bar | grep 5009e0

    00000000005009e0 A __bss_start
    00000000005009e0 A _edata
    00000000005009e0 B foo

(Symbols such as `__bss_start` and `_edata` are defined by the default
`ld` script; one can search them in the output of `ld -verbose`
command.)

The dynamic relocation table of `bar` is:

    Relocation section '.rela.dyn' at offset 0x490 contains 2 entries:
      Offset          Info           Type           Sym. Value    Sym. Name + Addend
    000000500998  000c00000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0
    0000005009e0  000700000005 R_X86_64_COPY     00000000005009e0 foo + 0

**Now what happens during runtime is this**: After `ld.so` loads all
dependent dynamic libraries, it starts processing their relocations.
When it sees `foo` of `libfoo.so`, it calls `RESOLVE_MAP` with r\_type =
`R_X86_64_GLOB_DAT` to get the Base Address, which is 0, and Symbol
Value, which is 5009e0. Next it sees `foo` of `libfoo.so` has
`R_X86_64_GLOB_DAT` relocation type, so it calculates the new address as
5009e0 = 0 + 5009e0 + 0 (addend) and stores the result somewhere inside
`.got` section.

After `ld.so` has processed relocations of all dynamic libraries, it
starts processing the relocation table of `bar`. When it sees `foo` of
`bar`, it calls `RESOLVE_MAP` again, but with r\_type = `R_X86_64_COPY`.
This time, the address returned is the runtime address of `foo` in
`libfoo.so`'s `.data` section. As mentioned earlier, this address holds
the initial value of `foo`. Next it sees `foo` of `bar` has
`R_X86_64_COPY` relocation type, so it uses `memcpy` to copy data to
5009e0 (see the `Sym. Value` of `.rela.dyn` section of `bar` above) from
the runtime address of `foo` in `libfoo.so`'s `.data` section (see Glibc
source file
[`sysdeps/x86_64/dl-machine.h`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=sysdeps/x86_64/dl-machine.h))

The above example also illustrates the difference between `.got` section
and `.got.plt` section. For the runtime linker `ld.so`, all it knows is
entries in `PLTREL` segment, i.e. `.rela.dyn` section, (which
corresponds to `.got` section) must be resolved/relocated immediately,
while entries in `JMPREL` segment, i.e. `.rela.plt` section, (which
corresponds to `.got.plt` section) can use lazy binding. For x86\_64
architecture, the relocation is actually not needed for
`R_X86_64_JUMP_SLOT` relocation types (albeit the symbol resolution is
still needed)

PIC or no PIC
-------------

When building a dynamic library, we are told to **always** compile the
code with `-fPIC` option.

**What's the difference then** ?

Consider the following simple code:

    #include <stdio.h>
    int bar;

    void foo() {
      printf("%d\n",bar);
    }

Compile the above code in 32-bit mode with and without `-fPIC`:

    $ gcc -shared -m32 foo.c -o nopic.so
    $ gcc -shared -m32 -fPIC foo.c -o pic.so

(If you try to compile the above in 64-bit mode, GCC will stop and
insist you should compile with `-fPIC` option, i.e. you are going to see
error message such as
`` relocation R_X86_64_PC32 against symbol `XXXYYY' can not be used when making a shared object; recompile with -fPIC ``)
The sections and relocation tables of `nopic.so` and `pic.so` are shown
at left and right hand side, respectively:

    Section Headers:                                                 Section Headers:
    [Nr] Name           Type     Addr                                [Nr] Name              Type     Addr
    [ 0]                NULL     0000                                [ 0]                   NULL     0000
      ...                                                              ...
    [ 8] .init          PROGBITS 02f8                                [ 8] .init             PROGBITS 02f0
    [ 9] .plt           PROGBITS 0310                                [ 9] .plt              PROGBITS 0308
    [10] .text          PROGBITS 0340                                [10] .text             PROGBITS 0350
    [11] .fini          PROGBITS 0488                                [11] .fini             PROGBITS 04a8
    [12] .rodata        PROGBITS 04a4                                [12] .rodata           PROGBITS 04c4
      ...                                                              ...
    [17] .dynamic       DYNAMIC  14c0                                [17] .dynamic          DYNAMIC  14e0
    [18] .got           PROGBITS 1590                                [18] .got              PROGBITS 15a8
    [19] .got.plt       PROGBITS 159c                                [19] .got.plt          PROGBITS 15b8
    [20] .data          PROGBITS 15b0                                [20] .data             PROGBITS 15d0

      ...                                                              ...

    Relocation section '.rel.dyn' at offset 0x2b0                    Relocation section '.rel.dyn' at offset 0x2b0
    contains 7 entries:                                              contains 5 entries:
     Offset     Info    Type            Sym.Value  Sym. Name         Offset     Info    Type            Sym.Value  Sym. Name
    00000439  00000008 R_386_RELATIVE                                000015d0  00000008 R_386_RELATIVE
    000015b0  00000008 R_386_RELATIVE                                000015a8  00000106 R_386_GLOB_DAT    000015dc   bar
    00000434  00000101 R_386_32          000015bc   bar                     ...
    00000445  00000602 R_386_PC32        00000000   printf
     ...

    Relocation section '.rel.plt' at offset 0x2e8:                   Relocation section '.rel.plt' at offset 0x2d8
    contains 2 entries:                                              contains 3 entries:
     Offset     Info    Type            Sym.Value  Sym. Name         Offset     Info    Type            Sym.Value  Sym. Name
    000015a8  00000207 R_386_JUMP_SLOT   00000000   __gmon_start__   000015c4  00000207 R_386_JUMP_SLOT   00000000   __gmon_start__
    000015ac  00000a07 R_386_JUMP_SLOT   00000000   __cxa_finalize   000015c8  00000607 R_386_JUMP_SLOT   00000000   printf
                                                                          ...

When we compile with `-fPIC` we can see the variable `bar` has the right
relocation type (`R_386_GLOB_DAT`) and the relocation takes place in the
right section (`.got`) The same for `printf`.

Without `-fPIC`, the relocations of the format string "\\n", `bar` and
`printf` all take place inside the `.text` section! But we know `.text`
section is in a Read-Only `LOAD` segment, so what `ld.so` would do ?

As expected, `ld.so` will make `.text` section writeable, patch the
bytes, and make it Read-Only again. Since the relocation of both `bar`
and `printf` are in `.rel.dyn`, their relocations are performed
immediately (no lazy binding), so this approach is feasible.

So how does `ld.so` handle `R_386_RELATIVE`, `R_386_32` and `R_386_PC32`
relocation types ?

Let's look at the disassembly:

    0000042c <foo>:
     42c: 55                push   ebp
     42d: 89 e5             mov    ebp,esp
     42f: 83 ec 18          sub    esp,0x18
     432: 8b 15 00 00 00 00 mov    edx,DWORD PTR ds:0x0   <-- reference to bar
     438: b8 a4 04 00 00    mov    eax,0x4a4              <-- reference to "%d\n" format string in .rodata
     43d: 89 54 24 04       mov    DWORD PTR [esp+0x4],edx
     441: 89 04 24          mov    DWORD PTR [esp],eax
     444: e8 fc ff ff ff    call   445 <foo+0x19>  <-- reference to printf
     449: c9                leave
     44a: c3                ret

How would the 4 bytes starting at 445 (`R_386_PC32` type) be patched ?
Suppose at runtime, our `nopic.so` is loaded into memory with base
address 8000, and the 4 bytes to be patched are now at 8000 + 445 =
8445. Furthermore, suppose `ld.so` has determined the entry address of
`printf` to be 10000, then `ld.so` calculates the **relative** offset as
follows:

    10000 - 8445 + fffffffc = 7bb7

(fffffffc is -4) so `ld.so` replaces `fc ff ff ff` with `b7 7b 00 00`

To patch the 4 bytes starting at 434 (`R_386_32` type) is simpler.
`ld.so` will simply overwrite the 4 bytes with the runtime **absolute**
address of `bar`.

To patch the 4 bytes starting at 439 (`R_386_RELATIVE` type) `ld.so`
calculates the address as

    10000 + 4a4 = 104a4

so `ld.so` replaces `a4 04 00 00` with `a4 04 01 00`

Finally, what about the `R_386_RELATIVE` relocation at 15b0 ? 15b0 is
the starting address of `.data` section, and the first 4 bytes of
`.data` section stores its own address, 15b0. So it has to be relocated
and patched as `115b0`.

In conclusion, `R_386_RELATIVE` means "32-bit relative to base address",
`R_386_PC32` means the "32-bit IP-relative offset" and `R_386_32` means
the "32-bit absolute."

Troubleshooting `ld.so`
-----------------------

### What is "error while loading shared libraries: requires glibc 2.5 or later dynamic linker" ?

The cause of this error is the dynamic binary (or one of its dependent
shared libraries) you want to run only has `.gnu.hash` section, but the
`ld.so` on the target machine is too old to recognize `.gnu.hash`; it
only recognizes the old-school `.hash` section.

This usually happens when the dynamic binary in question is built using
newer version of GCC. The solution is to recompile the code with either
`-static` compiler command-line option (to create a static binary), or
the following option:

    -Wl,--hash-style=both

This tells the link editor `ld` to create both `.gnu.hash` and `.hash`
sections.

According to `ld` documentation
[here](https://web.archive.org/web/20201202024834/http://sourceware.org/binutils/docs/ld/Options.html),
the old-school `.hash` section is the default, but the compiler can
override it. For example, the GCC (which is version 4.1.2) on RHEL (Red
Hat Enterprise Linux) Server release 5.5 has this line:

    $ gcc -dumpspecs
    ....
    *link:
    %{!static:--eh-frame-hdr} %{!m32:-m elf_x86_64} %{m32:-m elf_i386} --hash-style=gnu   %{shared:-shared}   ....
    ...

For more information, see
[here](https://web.archive.org/web/20201202024834/http://crtags.blogspot.com/2010/11/elf-elf-elf-dont-do-it.html).

### What is "Floating point exception" ?

The cause of this error is the same as the previous question. On certain
systems, e.g. RHEL, the old version `ld.so` is
[backported](https://web.archive.org/web/20201202024834/http://www.wikipedia.org/wiki/Backporting)
to emit "error while loading shared libraries: requires glibc 2.5 or
later dynamic linker", but this is not always the case, and you will see
this error instead.

### What is ".../libc.so.6: version \`GLIBC\_2.4' not found " ?

As the error message says, some of the symbols need Glibc version 2.4 or
higher. This can also be seen by

    $ objdump -x foo | grep 'Version References' -A10

    Version References:
      required from libc.so.6:
        0x0d696914 0x00 03 GLIBC_2.4
        0x09691a75 0x00 02 GLIBC_2.2.5

    ...

The fix is to recompile the code with `-static` compiler command-line
option.

### What is "FATAL: kernel too old" ?

Even if you recompile the code with `-static` compiler command-line
option to avoid any dependency on the dynamic Glibc library, you could
still encounter the error in question, and your code will exit with
Segmentation Fault error.

This kernel version check is done by `DL_SYSDEP_OSCHECK` macro in
Glibc's
[`sysdeps/unix/sysv/linux/dl-osinfo.h`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=sysdeps/unix/sysv/linux/dl-osinfo.h)
It calls `_dl_discover_osversion` to get current kernel's version.

To wit, run your code (suppose it is not stripped) inside gdb,

    (gdb) run
    Starting program: foo
    FATAL: kernel too old

    Program received signal SIGSEGV, Segmentation fault.
    0x00000000004324a9 in ptmalloc_init ()
    (gdb) call _dl_discover_osversion()
    $1 = 132617
    (gdb) p/x $1
    $2 = 0x20609
    (gdb)

Here `0x20609` means the current kernel version is 2.6.9.

The fix (or hack) is to add the following function in your code:

    int _dl_discover_osversion() { return 0xffffff; }

and compile your code with `-static` compiler command-line option.

Exploring Glibc's `pthread_t`
-----------------------------

When one creates a thread using the Pthread API, one will get a
`pthread_t` object as a handle. In Glibc, `pthread_t` is actually a
pointer pointing to a `pthread` struct, which is opaque. Its definition
can be found in Glibc's source tree at
[`nptl/descr.h`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=nptl/descr.h).
The first member of `pthread` struct is yet another struct called
`tcbhead_t` defined in system-dependent header files such as
[`nptl/sysdeps/x86_64/tls.h`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=nptl/sysdeps/x86_64/tls.h).
It holds TLS related information. It contains at least an integer member
called `multiple_threads` which indicates if the process is running in
multi-thread mode.

The second member of `pthread` struct is also a struct called `list_t`
defined in
[`nptl/sysdeps/pthread/list.h`](https://web.archive.org/web/20201202024834/http://sourceware.org/git/?p=glibc.git;a=blob_plain;f=nptl/sysdeps/pthread/list.h).

The third and fourth members of `pthread` struct are thread ID and
thread group ID (both are of `pid_t` type).

Other members of `pthread` struct which are of interest:
`int cancelhandling` for cancellation information, `int flags` for
thread attributes, `start_routine` for start position of the code to be
executed for the thread, `void *arg` for the argument to `start_routine`
`void *stackblock` and `size_t stackblock_size` for thread-specific
stack information.

Since `pthread` struct is opaque, how can one obtain the above
information, or more precisely, how can one obtain the offsets of these
members within the `pthread` struct ? We can use the known information
and search for the memory region pointed by `pthread_t`, as in this
[code
snippet](https://web.archive.org/web/20201202024834/http://www.acsu.buffalo.edu/%7Echarngda/code/tcb.c).
