toggle

LLD and GNU linker incompatibilities

Subtitle: Is LLD a drop-in replacement for GNU ld?

The motivation for this article was someone challenging the “drop-in replacement” claim on LLD’s website (the discussion was about Linux-like ELF toolchain):

LLD is a linker from the LLVM project that is a drop-in replacement for system linkers and runs much faster than them. It also provides features that are useful for toolchain developers.

99.9% pieces of software work with LLD without a change. Some linker script applications may need an adaption (such adaption is oftentimes due to brittle assumptions: asking too much from GNU ld’s behavior which should be fixed anyway). So I defended for this claim.

Piotr Kubaj said that this is a probably more of a marketing term than a technical term, the term tries to lure existing users into thinking “it’s the same you know, but better!”. I think that this is fair in some senses: for many applications LLD has achieved much faster speed and much lower memory usage than GNU ld. A more important thing is that LLD adds a third choice to the spectrum. It brings competitive pressure to both sides, gives incentive for improvement, and makes for more standardized future features/extensions. One reason that I am subscribed to the binutils mailing list is I want to participate in its design processes (I am proud to say that I have managed to find some early issues of various new things).

Anyway, I thought documenting the compatibility problems between the ELF ports of LLD and GNU ld is useful, not only to others but also to my future self, hence this article. I will try to describe GNU gold behaviors as well.

So here is the long list. Please keep in mind that many compatibility issues do not really matter and a user may never run into such an issue. Many of them just serve as educational purposes and my personal reference. There some some user perceivable differences but quite a lot are WONTFIX on both GNU ld and LLD. LLD, as a newer linker, has less legacy compatibility burden and can make good default choices in some cases and say no to some unneeded features/behaviors. A large number of features are duplicated in GNU ld’s various ports. It is also common that one thing behaves this way in port A and another way in port B.

Semantics of --wrap

GNU ld and LLD have slightly different --wrap semantics. I use “slightly” because in most use cases users will not observe a difference.

In GNU ld, --wrap only applies to undefined symbols. In LLD, --wrap happens after all other symbol resolution steps. The implementation is to mangle the symbol table of each object file (foo -> __wrap_foo; __real_foo -> foo) so that all relocations to foo or __real_foo will be redirected.

The LLD semantics have the advantage that non-LTO, LTO and relocatable link behaviors are consistent. I filed https://sourceware.org/bugzilla/show_bug.cgi?id=26358 for GNU ld.

# GNU ld: call bar
# LLD: call __wrap_bar
  call bar
.globl bar
bar:

Relocation referencing a local relative to a discarded input section

A symbol table entry with STB_LOCAL binding that is defined relative to one of a group’s sections, and that is contained in a symbol table section that is not part of the group, must be discarded if the group members are discarded. References to this symbol table entry from outside the group are not allowed.

ld.bfd/gold/lld error if the section containing the relocation is SHF_ALLOC. .debug* do not have the SHF_ALLOC flag and those relocations are allowed.

lld resolves such relocations to 0. ld.bfd and gold, however, have some CB_PRETEND/PRETEND logic to resolve relocations to the definitions in the prevailing comdat groups. The code is hacky and may not suit lld.

https://bugs.llvm.org/show_bug.cgi?id=42030

Canonical PLT entry for ifunc

How to handle a direct access relocation referencing a STT_GNU_IFUNC?

c.f. GNU indirect function.

__rela_iplt_start

GNU ld and gold define __rela_iplt_start in -no-pie mode, but not in -pie mode. LLD defines __rela_iplt_start regardless of -no-pie, -pie or -shared.

Static pie and static no-pie relocation processing is very different in glibc.

nsz has a glibc patch that moves the self-relocation later so everything is set up for ifunc resolvers.

Linker scripts

I’ll also mention some LLD release notes which can demonstrate some GNU incompatibility in previous versions. (For example, if one thing is supported in version N, then the implication is that it is unsupported in previous versions. Well, it could be that it worked in older versions but regressed at some version. However, I don’t know the existence of such things.)

LLD 12.0.0

LLD 11.0.0

LLD 10.0.0

LLD 9.0.0

LLD 8.0.0

In the LLD 7.0.0 era, https://reviews.llvm.org/D44264 was my first meaningful (albeit trivial) patch to LLD. Next I made contribution to --warn-backrefs. Then I started to fix tricky issues like copy relocations of a versioned symbol, duplicate --wrap, and section ranks. I have learned a lot from these code reviews. In the 8.0.0, 9.0.0 and 10.0.0 era, I have fixed a number of tricky issues and improved a dozen of other things and am confident to say that other than MIPS ;-) and certain other ISA specific things I am familiar with every corner of the code base. These are still challenges such as integration of RISC-V style linker relaxation and post-link optimization, improvement to some aspects of the linker script, but otherwise LLD is a stable and finished part of the toolchain.

A few random notes: