Cross-compiling CGO to ARM

This is a story about a a piece of Go code that wanted to be cross-compiled for the Raspberri Pi. From a private repo. On Wercker. With C/CGO files. And link it against libbluetooth.

Requirements

  • Cross-compile to ARM. Target the Raspberry Pi.
  • Include C source files. Needs cgo.
  • Link against libbluetooth. Needs custom ldflags.
  • Do all this on Wercker.

Steps I go through

  • Read Dave Cheney’s blog post on cross compilation.
  • Follow the link to Dave’s new blog post about the same topic.
  • Try goxc. It is supposed to be the “easy” route.
  • Issue #54 is merged, goxc now supports Go 1.4. Sweet!
  • It requires building the Go toolchain for every target.
  • Doesn’t work with the system Go on Gentoo — missing .hg, bad file permissions, etc.
  • Try fixing all these issues. Fail, then look for a better solution.
  • Find out about Péter Szilágyi’s xgo. Seems to fit the bill.
  • No need to build the toolchain. It’s all up in the Docker registry.
  • Find out about Issue #1 (can’t build non-public repos).
  • Péter isn’t interested in solving the issue. He uses Mitchell Hashimoto’s gox now.
  • Find out about gox. Seems to fit the bill even better.
  • Fail to build the toolchain locally, for the same reason as gox.
  • Look for a Wercker box that uses gox.
  • Find out about Taichi Nakashima’s wercker-box-gox and wercker-step-gox.
  • Send a few pull requests. Taichi accepts them shortly. The Wercker box now runs Go 1.4!
  • Tell Wercker to use tcnksm/gox.
  • Set up a build step to use tcnxsm/gox.
  • Oops! Go 1.4 won’t build C source files by default any more.
  • Add a simple build step to export CGO_ENABLED="1".
  • Hooray! Now it starts to build at least!
  • However, bluetooth.h was not found. OK, this one was expected.
  • Create a new Wercker box, gox-bluetooth, that apt-get installs libbluetooth-dev.
  • Change my app to use box: attilaolah/gox-bluetooth. I’me one more step closer to my goal.
  • cc1: error: unrecognized command line option '-marm' — well, that doesn’t say too much…
  • Google brings me to Go Issue #1880. It is already fixed.
  • Scrolling down, I notice this comment.

I meet this problem too, when compile go for arm, there are some problems, eg:

# runtime/cgo cc1: error: unrecognized command line option '-marm'

when compiling finished, cgo.a for arm not creat.

And the answer:

[…] I believe you are cross compiling from non arm to arm, and your gcc is complaining because it only known hows to compile for x86/x64. The short answer to this is, when cross compiling for arm, pass CGO_ENABLED=0 to disable cgo. If you need cgo on arm, you will have to compile on arm directly.

Damn. No CGO when cross compiling to ARM. This can’t be right. This is all because GCC doesn’t speak ARM. We can fix that. We have Gentoo. We have crossdev. We can compile a toolchain for ARM. Let’s roll.

Installing the ARM toolchain under Gentoo

This is relatively simple, but rather time consuming. I Install crossdev (in fact, I already have it installed, since I had already built the AVR toolchain for Arduino). Then I set up the toolchain. This will emerge binutils, gcc and friends.

sudo emerge crossdev
sudo crossdev -S -v -t armv6j-hardfloat-linux-gnueabi

It takes a few hours on my sistem, so I go grab a(nother) coffee.

A note on i386

At some point, I tried to cross-compile for i386 as well, but since it wasn’t that important, I stopped after I hit the first few failures.

  • The tcnxsm/gox build step allows me to specify multiple build targets, let’s try i386.
  • Apparently CGO for i386 needs bits/predefs.h. I need libc6-dev:i386. Welcome to Ubuntu multilib.
  • At this point I just try apt-get install -y build-essential:{i386,amd64} and hope for the best.
  • Bam! build-essential:i386 conflicts with build-essential:amd64. Apparently multilib isn’t fully supported in 12.04.
  • Let’s try to apt-get install build-essential libc6-dev:i386?
  • Nope! Installing libc6-dev:i386 removes build-essential:amd64, g++:amd64 and gcc:amd64 due to the conflicts.
  • Screw that. I don’t need i386. Let’s just stick with ARM. One less problem.

Apparently, compiling cgo code to multiple targets on the same host requires a true multilib system. One that has a multilib GCC. My Gentoo installation is true multilib, but Ubuntu 12.04 doesn’t seem to be. Without proper multilib support, I need a separate container for each architecture — not so fun.