Fails to link against libgc.a in Crystal 0.31.0

After the (default) crystal version is updated to 0.31.0, linking against libgc.a will fail, in this and this builds.

$ crystal spec
/usr/bin/ld: /usr/bin/../lib/crystal/lib/libgc.a(alloc.o): unrecognized relocation (0x2a) in section `.text'
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
Error: execution of command failed with code: 1: `cc "${@}" -o '/home/travis/.cache/crystal/crystal-run-spec.tmp' -rdynamic -lpcre -lm /usr/bin/../lib/crystal/lib/libgc.a -lpthread /usr/share/crystal/src/ext/libcrystal.a -levent -lrt -ldl -L/usr/bin/../lib/crystal/lib -L/usr/lib -L/usr/local/lib`
The command "crystal spec" exited with 1.

One of builds (tempdir) uses default Travis CI environment, Ubuntu Trusty. Another build (crystal-rpm) uses CentOS 6 (rpm-4.8.1) and Fedora 22 (rpm-4.12.0.1) via Docker is also failing with same error.

Although, I’ve not tested outside of Travis CI.

/cc @jhass, @bcardiff, (limited for new user’s restriction)

This problem is very similar to this article which is for Go:

For 0.31.0 release there was an update in which environment the linux binaries are built and linked.

These changes are https://github.com/crystal-lang/distribution-scripts/pull/44/files

  • debian:8 to debian:10
  • alpine:3.8 to alpine:3.10
  • llvm 4 to llvm 8

Maybe some of the toolchain got updated there and from https://stackoverflow.com/questions/52737698/unable-to-compile-unrecognized-relocation-0x2a-in-section-text I guess that is the reason for the errors?

I don’t think is specific to the gc the error.

It seems to be.

In that case, any object files generated by crystal may also cause same problem? (In other words, just using vendor’s gc (via apt-get install libgc-dev and so on) or building my own gc to avoid this problem may not work?)

If so,

I would like to change dist: to xenial, which is the suggested solution in the thread about Go, for the repositories that Linux distribution does not matter.

And I’ll try to build crystal for old Linux distributions as old as CentOS 6, I’m still using them for some reasons.

I have further investigated about this problem.

In short, the answer to

will be No.

And,

will also be No, but in different reason.

First, the answer in StackOverflow on the link provided by @bcardiff (thanks!) says that the problem is that the object files use R_X86_64_REX_GOTPCRELX relocations. This relocation will be used when your code has external linkage, and you invoke GCC with argument -fno-plt. (If GC has been built without this option, Debian seems to have patched GCC to -fno-plt as default.)

For example, consider following C code, which has external linkages to foo and printf:

#include <stdio.h>

extern int foo(void);

int main(int argc, char **argv)
{
	printf("%p\n", foo);
	return 0;
}

Compiling this with -fno-plt generates an object file using R_X86_64_REX_GOTPCRELX relocation.

$ gcc -fno-plt -c c.c
$ readelf -r c.o

Relocation section '.rela.text' at offset 0x248 contains 3 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000000012  000b0000002a R_X86_64_REX_GOTP 0000000000000000 foo - 4
00000000001a  00050000000a R_X86_64_32       0000000000000000 .rodata + 0
000000000025  000c00000029 R_X86_64_GOTPCREL 0000000000000000 printf - 4

Relocation section '.rela.eh_frame' at offset 0x290 contains 1 entry:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000000020  000200000002 R_X86_64_PC32     0000000000000000 .text + 0

Otherwise, these relocations won’t be used and PLTs are used instead.

$ gcc -c c.c
$ readelf -r c.o

Relocation section '.rela.text' at offset 0x218 contains 3 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000000010  000a0000000a R_X86_64_32       0000000000000000 foo + 0
000000000015  00050000000a R_X86_64_32       0000000000000000 .rodata + 0
00000000001f  000b00000004 R_X86_64_PLT32    0000000000000000 printf - 4

Relocation section '.rela.eh_frame' at offset 0x260 contains 1 entry:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000000020  000200000002 R_X86_64_PC32     0000000000000000 .text + 0

For Clang, no modifications about code generation has been made when they introduced -fno-plt option (⚙ D39079 New clang option -fno-plt to avoid PLT for external calls), and just added small number of method calls. So, when not specified, it seems that this is dependent to LLVM whether use this relocation on code generation.

If Crystal does not do so, these relocations are not used until LLVM change the default. Therefore, any object files generated by crystal won’t cause this problem, currently.

On the other hand, Crystal distribution uses the patched version of GC (https://github.com/crystal-lang/distribution-scripts/blob/master/linux/files/feature-thread-stackbottom-upstream.patch), so using vendor GC may not work correctly.

I still want an official suggestion about this problem.

PS: The version of GCC used here is:

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-momonga-linux/9.2.1/lto-wrapper
Target: x86_64-momonga-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-pkgversion='Momonga Linux 9.2.1-0.5m.mo8' --program-suffix= --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --enable-multilib --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --host=x86_64-momonga-linux --build=x86_64-momonga-linux --target=x86_64-momonga-linux --with-default-libstdcxx-abi=gcc4-compatible --with-build-config=bootstrap-lto --with-isl --enable-libmpx --enable-gnu-indirect-function --with-tune=generic --without-system-libunwind --enable-languages=c,c++,objc,obj-c++,fortran,lto,go --with-ppl --enable-cloog-backend=isl --enable-gnu-indirect-function --with-tune=generic --with-arch_32=i686 --build=x86_64-momonga-linux
Thread model: posix
gcc version 9.2.1 20190831 (Momonga Linux 9.2.1-0.5m.mo8)

I have built Crystal compiler using binaries of LLVM8 provided by the LLVM team, and it works fine without upgrading ld for CentOS 6 and Fedora 22, currently.

To use officially distributed Crystal compiler under CentOS 6, Fedora 22, or Ubuntu trusty, you may use conda to upgrade ld. See BFD version 2.20.51.0.2-5.47.el6_9.1 20100205 internal error, aborting at reloc.c line 443 in bfd_get_reloc_size · Issue #8323 · crystal-lang/crystal · GitHub. I did not test this, so I don’t know how long take to upgrade ld.

You can use Ubuntu xenial (or even more recent), which includes newer ld.

Oh, I tested under the CentOS 6 using official Crystal compiler and bdwgc provided by CentOS team, it causes same problem. Sorry.

My “fix” for this error on an 15.04 box was to download and install binutils “the latest version” over the top of the existing package…works now at least…

As far as Travis is concerned, they provide a standard Ubuntu installation and install the crystal binary package from the official Apt repo, in accordance with the official installation instructions.

If that package uses the distro’s linker in the course of its work but has and/or produces binaries incompatible with it, this means that this package is incompatible with the distribution – while the vendor advertises it as compatible by providing an official binary package for it.

So this is a problem with the package. Raise this as an issue with its maintainers.

As a follow up there is a recent discussion in https://github.com/crystal-lang/crystal/issues/8653

But also, in order to simplify crystal support in travis through snap I opened https://github.com/travis-ci/travis-build/pull/1834 . The downside of that PR is that Crystal will only be installed through snap and AFAIK that is not supported by trusty nor precise.

With the release of Crystal 0.33.0 this should be working again on Trusty.

I confirmed that Crystal 0.33.0 works on CentOS 6 and Fedora 22 again.

Thanks for releasing fixed version!