Integrating Golang


Leo Schwab <lschwab@...>
 

We have gotten it into our minds to write some of the apps on our
device in Go, a shiny new compiled language from Google (
http://golang.org/ ). However, Go has very -- shall we say --
independent ideas about how to organize, fetch, build, and deploy code
which would, at first glance, appear to conflict with Yocto's
mechanisms for same. Go's support for cross-compilation is also
fairly new and undergoing refinement. We've kluged together a couple
of recipes which are target-specific, but a general solution for
Go-built targets in Yocto has not revealed itself.

Has anyone else done any work here? Is there anything that I can
steal^H^H^H^H^Htake inspiration from? Or am I in completely
unexplored territory?

Thanks in advance,
Schwab


Hermanus Botha <wkj.id10t@...>
 

So I don't really know much about Yocto. But I would also like to see some
more Go-lang action on Yocto.

This section of their manual seems promising.
http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#cross-
development-toolchain-generation


Hermanus Botha <wkj.id10t@...>
 

Hermanus Botha <wkj.id10t@...> writes:


So I don't really know much about Yocto. But I would also like to see some
more Go-lang action on Yocto.

This section of their manual seems promising.
http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#cross-
development-toolchain-generation

I guess this link is broken. In that manual, it's section:
"4.2. Cross-Development Toolchain Generation"

Manual:
http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html


Khem Raj
 

On Wed, Apr 16, 2014 at 12:10 PM, Hermanus Botha <wkj.id10t@gmail.com> wrote:
So I don't really know much about Yocto. But I would also like to see some
more Go-lang action on Yocto.
Start by generating the gcc runtime and language support in gcc
recipes. add it to RUNTIMETARGET and adjust PACKAGES
you also need to add it to LANGUAGES that should get it to some point.


This section of their manual seems promising.
http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#cross-
development-toolchain-generation

--
_______________________________________________
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto


Martin Donnelly <martin.donnelly@...>
 

On 05/03/2014 21:55, Leo Schwab wrote:
Has anyone else done any work here? Is there anything that I can
steal^H^H^H^H^Htake inspiration from? Or am I in completely
unexplored territory?
I posted an RFC patch set last year but it didn't get any traction and
we subsequently ruled out use of go so I never followed up on it but it
might be a useful starting point.

http://lists.openembedded.org/pipermail/openembedded-core/2013-May/078606.html

Cheers

- Martin


Khem Raj
 

On Thu, Apr 17, 2014 at 1:49 AM, Martin Donnelly <martin.donnelly@ge.com> wrote:
On 05/03/2014 21:55, Leo Schwab wrote:
Has anyone else done any work here? Is there anything that I can
steal^H^H^H^H^Htake inspiration from? Or am I in completely
unexplored territory?
I posted an RFC patch set last year but it didn't get any traction and
we subsequently ruled out use of go so I never followed up on it but it
might be a useful starting point.

http://lists.openembedded.org/pipermail/openembedded-core/2013-May/078606.html
Thats looks reasonable patch to me. Provided it can be forward ported
to gcc 4.8+
you should resubmit it for master

Cheers

- Martin
--
_______________________________________________
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto


Leo Schwab <lschwab@...>
 

On Thu, Apr 17, 2014 at 1:49 AM, Martin Donnelly <martin.donnelly@ge.com> wrote:
On 05/03/2014 21:55, Leo Schwab wrote:
Has anyone else done any work here? Is there anything that I can
steal^H^H^H^H^Htake inspiration from? Or am I in completely
unexplored territory?
I posted an RFC patch set last year but it didn't get any traction and
we subsequently ruled out use of go so I never followed up on it but it
might be a useful starting point.

http://lists.openembedded.org/pipermail/openembedded-core/2013-May/078606.html
We cobbled together our own recipes to build the Go compiler
(available upon request), which seem to be working well enough. We
didn't need to make any changes to the recipes building the GCC
toolchain.

What I was more interested in learning was how people were integrating
the building of Go applications into Yocto. Specifically: 'go get
foo' will build 'foo' and, in the process, fetch and build all its
dependencies. This conflicts directly with Yocto's duties -- fetching
and building dependencies.

My current not-solution is rather ugly: Create a repository that is a
standard Go build environment pre-populated with the app source code
and all of its dependencies. The idea is that 'go get foo' will find
everything it needs already there and won't try and fetch anything
else. Then create a simple Yocto recipe that fetches this repository
and builds it using 'go get ...'. The repository itself is maintained
via rather delicate use of git-subtree.

This gets the job done, and prevents Go and Yocto having a fight about
who fetches what and how, but it's damned ugly...

Schwab


Ilya Dmitrichenko
 

Hi Leo and Martin,

On 17 April 2014 20:54, Leo Schwab <lschwab@sensity.com> wrote:
On Thu, Apr 17, 2014 at 1:49 AM, Martin Donnelly <martin.donnelly@ge.com> wrote:
On 05/03/2014 21:55, Leo Schwab wrote:
Has anyone else done any work here? Is there anything that I can
steal^H^H^H^H^Htake inspiration from? Or am I in completely
unexplored territory?
I posted an RFC patch set last year but it didn't get any traction and
we subsequently ruled out use of go so I never followed up on it but it
might be a useful starting point.

http://lists.openembedded.org/pipermail/openembedded-core/2013-May/078606.html
We cobbled together our own recipes to build the Go compiler
(available upon request), which seem to be working well enough. We
didn't need to make any changes to the recipes building the GCC
toolchain.
I would be very much interested to take a look at what you did. My
use-case right now is to build Docker with Yocto.

Are using gogcc? From what understood it doen't build static binary
like gc does... which is, to be honest, my favourite feature of Go.

What I was more interested in learning was how people were integrating
the building of Go applications into Yocto. Specifically: 'go get
foo' will build 'foo' and, in the process, fetch and build all its
dependencies. This conflicts directly with Yocto's duties -- fetching
and building dependencies.
I understand your concern with `go get` fetching dependencies that
bitbake should rather fetch. However, I would rather avoid using
global $GOPATH and set it to be relative to the build directory, i.e.
have per-project $GOPATH. That way `go get` is just a step in the
build process and doesn't intervine with bitbake in any way.

I have previously written a few recipes for some Python packages, and
do know that Yocto doesn't use any of the convinient tools (i.e.
easy_install or pip), but I would debate that it should and, perhaps,
in combination with virtualenv as well. There is currently no support
for Python 3 in Yocto, which is something that needs looking into and
I would also look into using pip then also.

My current not-solution is rather ugly: Create a repository that is a
standard Go build environment pre-populated with the app source code
and all of its dependencies. The idea is that 'go get foo' will find
everything it needs already there and won't try and fetch anything
else. Then create a simple Yocto recipe that fetches this repository
and builds it using 'go get ...'. The repository itself is maintained
via rather delicate use of git-subtree.
This gets the job done, and prevents Go and Yocto having a fight about
who fetches what and how, but it's damned ugly...
This is kind of an okay solution, as Go community is not interested to
have `go get` to handle versions of dependcy, which is somewhat fair.
So if you needs to pin specific revisions, you will have to do either
subtree or submodule or script that just clones those deps or or
anything else of this kind.

I don't think it's any useful if you have to write bitbake recipe for
each little Go library you might be using, this is not C and `go get`
just works for most part. And when it doesn't, I would either fork and
import from my own fork or checkout a working revision of give
dependency and do git add on it, checking it into my project's main
tree.

I think at this stage it's improtant to provide a working Go compiler
for the Yocto community to use on their own projects. There are
currently only very few things that one might treat as system-level
tools, well, I can only recall of one actually, and it's the one I'm
attempting to integrate - Docker.

Cheers,
--
Ilya


Ilya Dmitrichenko
 

Just a quick update, thought I'd let you know that I have a basic
working layer here:

https://github.com/errordeveloper/oe-meta-go


Ross Burton
 

On 10 September 2014 13:53, Ilya Dmitrichenko <errordeveloper@gmail.com> wrote:
Just a quick update, thought I'd let you know that I have a basic
working layer here:

https://github.com/errordeveloper/oe-meta-go
Could you consider submitting this to the layer index, layers.openembedded.org?

Cheers,
Ross


Paul Eggleton
 

On Wednesday 10 September 2014 14:36:37 Burton, Ross wrote:
On 10 September 2014 13:53, Ilya Dmitrichenko <errordeveloper@gmail.com>
wrote:
Just a quick update, thought I'd let you know that I have a basic
working layer here:

https://github.com/errordeveloper/oe-meta-go
Could you consider submitting this to the layer index,
layers.openembedded.org?
He has and it's been published. Thanks for the reminder though ;)

Cheers,
Paul

--

Paul Eggleton
Intel Open Source Technology Centre


Maciej Borzecki <maciej.borzecki@...>
 

On Wednesday 10 of September 2014 13:53:14 Ilya Dmitrichenko wrote:
Just a quick update, thought I'd let you know that I have a basic
working layer here:

https://github.com/errordeveloper/oe-meta-go
There's also https://github.com/digitallumens/meta-golang

Based on that I've prepped some patches not long ago that integrate go into
OE-core base on what Chris did in meta-golang. The patches are for both meta--
golang and OE-core. OE-core piece enables Go GCC frontend and libgo same way
as Fortran or Java is handled. Also, Chris has already defined a usable
golang.bbclass that handles gccgo build for you. So far it seems to work with
some simple use cases on BeagleBone Black and Raspberry.

If there's interest I can update the patches to current master and post them
to the list.

--

Maciej Borzęcki
Senior Software Engineer Open-RnD Sp. z o.o.
www.open-rnd.pl, Facebook, Twitter
mobile: +48 telefon, fax: +48 42 657 9079

Niniejsza wiadomość wraz z załącznikami może zawierać chronione prawem lub
poufne informacje i została wysłana wyłącznie do wiadomości i użytku osób, do
których została zaadresowana. Jeśli wiadomość została otrzymana przypadkowo
zabrania się jej kopiowania lub rozsyłania do osób trzecich. W takim przypadku
uprasza się o natychmiastowe zniszczenie wiadomości oraz poinformowanie
nadawcy o zaistniałej sytuacji za pomocą wiadomości zwrotnej. Dziękujemy.

This message, including any attachments hereto, may contain privileged or
confidential information and is sent solely for the attention and use of the
intended addressee(s). If you are not an intended addressee, you may neither
use this message nor copy or deliver it to anyone. In such case, you should
immediately destroy this message and kindly notify the sender by reply email.
Thank you.


Chris Elledge
 

I'm actively using meta-golang for development and continuing to add features as I find them necessary. If anyone has suggestions on what would be useful as a developer of embedded go applications, let me know.

Specifically, I am most interested on input for what would be useful configuration variables to have available for golang based recipes:
* Ways to request how to breakdown a golang project into combinations of shared libraries, static libraries, and executables.
* Being able to define which golang packages are to be placed into libraries vs statically linked into the executables.

If someone manages to get this properly integrated into yocto, I would be happy to move my development directly to the project.

-Chris

On Oct 30, 2014, at 5:47 AM, Maciej Borzecki <maciej.borzecki@open-rnd.pl> wrote:
There's also https://github.com/digitallumens/meta-golang

Based on that I've prepped some patches not long ago that integrate go into
OE-core base on what Chris did in meta-golang. The patches are for both meta--
golang and OE-core. OE-core piece enables Go GCC frontend and libgo same way
as Fortran or Java is handled. Also, Chris has already defined a usable
golang.bbclass that handles gccgo build for you. So far it seems to work with
some simple use cases on BeagleBone Black and Raspberry.

If there's interest I can update the patches to current master and post them
to the list.