X86-64 Application Binary Interface

Note: any information here does not necessarily apply to any other computing architecture - but do note that the same x86_64 architecture may be called x86-64 or x64, and that many architectures are based off x86_64 - such as IA64 or AMD64. x86_64 is simply an extension of the x86 architecture (the general format shared by 32-bit architectures) - extending the old 32-bit to 64-bit.

Lesson objectives:

Understand what an ABI is.
Understand what segments are.
Understand what memory frames are.
Understand how a compiler/linker works in structuring a program to conform to the ABI.

Lesson

What is an ABI?:

An Application Binary Interface (ABI) is like an API applied on an executable based level - it defines the layout of binary code to conform to the requirements of a processor & binary format. There are many different executable formats, each of which conform to a CPU's requirements, while utilizing different frames in different ways.

What is a memory segment?:

A memory segment is a partition of RAM allocated for some particular usage. A list of all memory segments is kept in the Global Descriptor Table (GDT), which may differ between architectures (or not be present in others). Each entry in the GDT may specify a few things, but for a memory segment, a base address in memory (note: the author believes this to be physical memory, but cannot be sure), some offset for how long the segment is, and some flags and control bits to alter the functionality of the entry. While in x86-based architectures the flags are completely necessary, in some x86_64 architectures, most of the flags etc. are ignored - the only ones to worry about are the privilege level, executable and R/W bits. These are the only flags worth discussing, however: the R/W bits can stop a process from overwriting a read-only segment, or executing data, or altering executable code; the executable bit stops people from executing code not intended to be run; and the privilege level stops code from running privileged code or reading privileged data - even if it finds a way to access said segments. They are basically a level of protection to control what programs can do - you don't want any program being able to read your entire hard disk! When code is executed, a set of "Segment Registers" are loaded with information on where each segment is in memory. Then, different offsets can be stored. This means that programs get restricted to using whatever chunk of memory they are allocated (unless they perform a far jump: this is still controlled by the privilege level of the segment). These segment registers are, however, ignored in some 64-bit architectures, due to use of a different system called paging.

Notes about the ABI:

The ABI, as it defines how the code is laid out, of course also defines how the code works. One main thing to note is that the ABI defines that addresses are used at an offset related to the correct segment to refer to within the current operation, or within paging if paging is enabled. The ABI layout is constructed of "frames", which each contain some sort of data (or code). These "frames" have their data loaded into the corresponding segments if they are ones which do so.

Frames:

A "frame" of memory is a section (as it may also in fact be known as) within an executable, possibly required by the ABI, which contains some particularly formatted code. The point of frames is to be able to group information as is required by necessity - there are loaded frames which contain information required to run a program, and are loaded into memory, and ones that are not loaded. The ones that are not loaded contain extra information, such as debugging information. When an executable is loaded, the OS writes from the hard disk the correct program data into memory, by copying the required data from each frame into its correct place. Then, to execute a program, the GDT is altered as necessary, and a TSS is added to the GDT (not explained here). Finally, code can now be run! Some of the frames include: the .text frame for code; the .data frame for global variables and data (ie. initialized data); the .rodata frame for read-only memory (ie. constant global variables, which by nature MUST be initialized); the .bss frame for program data and necessary extra symbols; and the .eh_frame, which contains information for a stack unwind (this doesn't apply to C, it's a C++ feature with exception handling) and possibly some extra information which can be used for debugging. Note that in ELF x86_64, the most common format for Linux binaries, .rodata tends to be put in the .text section, since they are both non-readable, non-writable data. Also, eh_frame, though unnecessary in C, MUST be included in the ELF binary. This can be annoying, since it may take about a sixth to a third of the size of executables - therefore it is recommended to pass the -fno-asynchronous-unwind-tables and -g options to GCC, to place extra debug information within a different section .debug_frame, which can then be stripped using the GNU strip command (a part of binutils), which removes the extra segments (of which the .debug_frame is one). Note that when writing code, people may prefer to simply have a makefile which allows for making a debug version and a release version.

Compilers and linkers:

These are your best friend for understanding the numerous frames, and not having to read the x86_64 ABI standards yourself. How? Because you don't actually need to know any of the frames. Even when writing your own linker file, extra segments are automatically put wherever they should be. GCC is already programmed to actually call the linker when passed multiple files, and with the right options too! Actually using the linker by itself can be problematic without GCC's help. Basically, with C the ones mentioned before are the ones to worry about (the author believes that strip actually ends up removing all the other segments from compiling C normally anyways), and only ever when you are concerned about kernel-level code, or possibly some hardware-related code. Still it can be useful to know what exactly is going wrong when you try to execute a file and errors relating to the ABI. The way a compiler/linker works, is that when code is transformed into machine code by a compiler, all the data from each section is organised. Each frame/section is placed, with its metadata, into the correct layout specified by the format (the author believes that ELF uses a set of headers at the start of the file, and then places the frames after this), and then the file is written to the disk.

How are the frames useful?:

Each frame helps with debugging, and also allows the OS to correctly load a program into memory. Without frames, it is not possible to properly format code and data together to allow proper formatting with a GDT entry. Finally, with frames, it is possible to classify information in such a way that different OSes and architectures have ABI compatability, so binaries (that avoid architecture specific instructions) can run on a variety of different systems,without the need to recompile.

References