Re: Creating a QEmu (x86-64) Image, which can execute binary applications from other x86-64 linux OSes


Christian Lohr <christian.lohr.ext@...>
 

Hello Khem,

Thanks for that advice, it seems to be the best way to handle this.

Best regards,

Christian Lohr

-----Ursprüngliche Nachricht-----
Von: yocto@... <yocto@...> Im Auftrag von Khem Raj
Gesendet: Mittwoch, 12. Februar 2020 19:12
An: Lohr, Christian [ext] <christian.lohr.ext@...>; Alexander Kanavin <alex.kanavin@...>
Cc: yocto@...
Betreff: Re: [yocto] Creating a QEmu (x86-64) Image, which can execute binary applications from other x86-64 linux OSes

On 2/12/20 3:19 AM, Christian Lohr wrote:
Ok, I created a symbolic link with "ln -s /lib /lib64" and it seemed
to work. Thanks a lot.
right, if you built multilib image then it will be using /lib64 automatically.

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.yoctoproject.org%2Fdocs%2Fcurrent%2Fdev-manual%2Fdev-manual.html%23combining-multiple-versions-library-files-into-one-image&;data=02%7C01%7C%7Cafab24d3bd894869269008d7afe71947%7C28042244bb514cd680347776fa3703e8%7C1%7C0%7C637171279406982418&amp;sdata=k7AMcA2eXvLgqBbtaUQFNSFIFBVitJ8v9oQK5%2FYacRU%3D&amp;reserved=0


*Von:* yocto@... <yocto@...> *Im
Auftrag von *Alexander Kanavin
*Gesendet:* Mittwoch, 12. Februar 2020 12:00
*An:* Lohr, Christian [ext] <christian.lohr.ext@...>
*Cc:* yocto@...
*Betreff:* Re: [yocto] Creating a QEmu (x86-64) Image, which can
execute binary applications from other x86-64 linux OSes

But this is exactly what happens: the kernel reads the dynamic
loader/interpreter path from the binary (which is different than the
list of dynamically linked libraries printed by ldd), isn't able to
find it and stops right there.

Try like this:

/lib/ld-linux-x86-64.so.2 /usr/share/dotnet/dotnet

Alex

On Wed, 12 Feb 2020 at 11:45, Lohr, Christian [ext]
<christian.lohr.ext@... <mailto:christian.lohr.ext@...>> wrote:

I used the x86_64 variant from the layer (it only downloads the
binaries and copies them).

And I checked that with ldd, it seemed ok so far:

---------------------------------------------------------

root@qemux86-64:/usr/share/dotnet# ldd dotnet

            linux-vdso.so.1 (0x00007fffea543000)

            libpthread.so.0 => /lib/libpthread.so.0
(0x00007f9fde06c000)

            libdl.so.2 => /lib/libdl.so.2 (0x00007f9fde067000)

libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f9fddee5000)

libm.so.6 => /lib/libm.so.6 (0x00007f9fddda4000)

            libgcc_s.so.1 => /lib/libgcc_s.so.1
(0x00007f9fddd8a000)

            libc.so.6 => /lib/libc.so.6 (0x00007f9fddbd0000)

            /lib64/ld-linux-x86-64.so.2 =>
/lib/ld-linux-x86-64.so.2 (0x00007f9fde097000)

Strace didn't help either:

-------------------------------

root@qemux86-64:/# strace /usr/share/dotnet/dotnet

execve("/usr/share/dotnet/dotnet", ["/usr/share/dotnet/dotnet"],
0x7ffe22f0ab70 /* 22 vars */) = -1 ENOENT (No such file or
directory)

strace: exec: No such file or directory

+++ exited with 1 +++

It's strange that it denies that the binaries are there. Normally I
would have expected something like "wrong elf architecture" or
something about missing libraries. The only thing I think I could do
now, is to turn this "-enable-default-pie" off, but I'm not sure if
this helps, and I don't know where to turn it off. And what I'm also
trying is to go back to Yocto Rocko Release (for this experiment I
used Warrior Release)

*Von:* yocto@...
<mailto:yocto@...> <yocto@...
<mailto:yocto@...>> *Im Auftrag von *Alexander
Kanavin
*Gesendet:* Mittwoch, 12. Februar 2020 10:26
*An:* Lohr, Christian [ext] <christian.lohr.ext@...
<mailto:christian.lohr.ext@...>>
*Cc:* yocto@... <mailto:yocto@...>
*Betreff:* Re: [yocto] Creating a QEmu (x86-64) Image, which can
execute binary applications from other x86-64 linux OSes

That layer does have the x86_64 variant as well, no? Is it not working?

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FRDunkley%2Fmeta-dotnet-core%2Fblob%2Fmaster%2Frecipes-runtime%2Fdotnet-core%2Fdotnet-core_3.1.1.inc&;data=02%7C01%7C%7Cafab24d3bd894869269008d7afe71947%7C28042244bb514cd680347776fa3703e8%7C1%7C0%7C637171279406982418&amp;sdata=S0WxuzuKiJcCdXLuVKJjgCxcl0kup6TUNT1fDPyQNUc%3D&amp;reserved=0

<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit
hub.com%2FRDunkley%2Fmeta-dotnet-core%2Fblob%2Fmaster%2Frecipes-runtim
e%2Fdotnet-core%2Fdotnet-core_3.1.1.inc&amp;data=02%7C01%7C%7Cafab24d3
bd894869269008d7afe71947%7C28042244bb514cd680347776fa3703e8%7C1%7C0%7C
637171279406982418&amp;sdata=S0WxuzuKiJcCdXLuVKJjgCxcl0kup6TUNT1fDPyQN
Uc%3D&amp;reserved=0>

The error you're seeing is almost certainly due to Yocto using
/lib/ld-so... for dynamic loader, and the binary hardcoding /lib64/....

Alex

On Wed, 12 Feb 2020 at 10:15, Lohr, Christian [ext]
<christian.lohr.ext@... <mailto:christian.lohr.ext@...>>
wrote:

Hello Alex,

Thanks for replying. Yes, I know that this isn't the way it
works on Yocto (and I told the managers it is a crappy idea to
do that more than once). But they need .NET Core in that company
I work for, and Mono doesn't work (that's what they told me).
Compiling .NET Core through the Yocto process is ugly, because
Microsoft used a mixture of shell scripts to compile it for some
platforms, it won't work this way on Yocto. Actually one already
tried it, but only until .NET Core 2.2:
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FTragetaschen%2Fmeta-aspnet&;data=02%7C01%7C%7Cafab24d3bd894869269008d7afe71947%7C28042244bb514cd680347776fa3703e8%7C1%7C0%7C637171279406982418&amp;sdata=NxojWXOJbRlLijXNQ74h4%2FbPWgvmkvX47w%2FhyqjJUL0%3D&amp;reserved=0

<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit
hub.com%2FTragetaschen%2Fmeta-aspnet&amp;data=02%7C01%7C%7Cafab24d3bd8
94869269008d7afe71947%7C28042244bb514cd680347776fa3703e8%7C1%7C0%7C637
171279406982418&amp;sdata=NxojWXOJbRlLijXNQ74h4%2FbPWgvmkvX47w%2FhyqjJ
UL0%3D&amp;reserved=0>

And despite this, I already managed to get the dotnet binaries
for ARM32 and ARM64 already working on a i.MX6 and i.MX8

There's a layer which just deploys the binaries:
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FRDunkley%2Fmeta-dotnet-core&;data=02%7C01%7C%7Cafab24d3bd894869269008d7afe71947%7C28042244bb514cd680347776fa3703e8%7C1%7C0%7C637171279406982418&amp;sdata=T00TXDmEFPBo4T66Quvzy4P%2Bkqki5BWaeWqehh%2B2gXY%3D&amp;reserved=0

<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit
hub.com%2FRDunkley%2Fmeta-dotnet-core&amp;data=02%7C01%7C%7Cafab24d3bd
894869269008d7afe71947%7C28042244bb514cd680347776fa3703e8%7C1%7C0%7C63
7171279406982418&amp;sdata=T00TXDmEFPBo4T66Quvzy4P%2Bkqki5BWaeWqehh%2B
2gXY%3D&amp;reserved=0>

This is currently the last step. I thought if it worked on i.MX6
and i.MX8 it shouldn't be a problem to get it running on
Virtualbox with x86-64. It should only make the things easy for
the developers. It isn't even our target platform.

Best regards,

Christian Lohr

*Von:* yocto@...
<mailto:yocto@...>
<yocto@...
<mailto:yocto@...>> *Im Auftrag von
*Alexander Kanavin
*Gesendet:* Mittwoch, 12. Februar 2020 09:51
*An:* Lohr, Christian [ext] <christian.lohr.ext@...
<mailto:christian.lohr.ext@...>>
*Cc:* yocto@...
<mailto:yocto@...>
*Betreff:* Re: [yocto] Creating a QEmu (x86-64) Image, which can
execute binary applications from other x86-64 linux OSes

Yocto generally does not support this use case. The binary was
compiled in a different environment and expects things in
different places, and probably being different versions too. I
could point out the specific problem why the executable doesn't
even start, but it's really the wrong way to approach it. Is the
source code for it available?

Microsoft specifically lists which distributions are supported,
and there is nothing Yocto-based in it:

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fdotnet%2Fcore%2Finstall%2Fdependencies%3Ftabs%3Dnetcore31%26pivots%3Dos-linux&;data=02%7C01%7C%7Cafab24d3bd894869269008d7afe71947%7C28042244bb514cd680347776fa3703e8%7C1%7C0%7C637171279406982418&amp;sdata=RAHH6UvIQpXPUyxI5OhlYv1UZyuInRwBQgptqc8g%2Bss%3D&amp;reserved=0

<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc
s.microsoft.com%2Fen-us%2Fdotnet%2Fcore%2Finstall%2Fdependencies%3Ftab
s%3Dnetcore31%26pivots%3Dos-linux&amp;data=02%7C01%7C%7Cafab24d3bd8948
69269008d7afe71947%7C28042244bb514cd680347776fa3703e8%7C1%7C0%7C637171
279406982418&amp;sdata=RAHH6UvIQpXPUyxI5OhlYv1UZyuInRwBQgptqc8g%2Bss%3
D&amp;reserved=0>

For mono things you can use meta-mono layer, but I am not sure
if it provides exactly the item you're after.

Alex

On Wed, 12 Feb 2020 at 09:36, Christian Lohr
<christian.lohr.ext@...
<mailto:christian.lohr.ext@...>> wrote:

Hello,

I'm trying to create a normal QEmu (x86-64) Image, which I
can let run in Virtualbox. As a addition I deployed .NET
Core, which I got from this side:

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdotnet.microsoft.com%2Fdownload%2Fdotnet-core%2Fthank-you%2Fruntime-aspnetcore-3.1.1-linux-x64-binaries&;data=02%7C01%7C%7Cafab24d3bd894869269008d7afe71947%7C28042244bb514cd680347776fa3703e8%7C1%7C0%7C637171279406992365&amp;sdata=hDN%2BmtyxOpjAzwD9oNPzqVcSJrYgnoZYV6e31yLtlsI%3D&amp;reserved=0

<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdot
net.microsoft.com%2Fdownload%2Fdotnet-core%2Fthank-you%2Fruntime-aspne
tcore-3.1.1-linux-x64-binaries&amp;data=02%7C01%7C%7Cafab24d3bd8948692
69008d7afe71947%7C28042244bb514cd680347776fa3703e8%7C1%7C0%7C637171279
406992365&amp;sdata=hDN%2BmtyxOpjAzwD9oNPzqVcSJrYgnoZYV6e31yLtlsI%3D&a
mp;reserved=0>

But I can't execute it:

----------------------------

root@qemux86-64:/usr/share/dotnet# ./dotnet

-sh: ./dotnet: No such file or directory

But it is there:

------------------

root@qemux86-64:/usr/share/dotnet# ls -lh

total 116K

-rw-r--r-- 1 root root 1.1K Feb 10 02:33 LICENSE.txt

-rw-r--r-- 1 root root  31K Feb 10 02:33
ThirdPartyNotices.txt

-rwxr-xr-x 1 root root  72K Feb 10 02:33 dotnet

drwxr-xr-x 3 root root 4.0K Feb 10 02:36 host

drwxr-xr-x 4 root root 4.0K Feb 10 02:36 shared

It tried to get more information about the
dotnet-executable


----------------------------------------------------------------------
--------

root@qemux86-64:/usr/share/dotnet# readelf -h dotnet

ELF Header:

  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00
00

  Class:                             ELF64

  Data:                              2's complement, little
endian

  Version:                           1 (current)

  OS/ABI:                            UNIX - System V

  ABI Version:                       0

  Type:                              EXEC (Executable
file)

  Machine:                           Advanced Micro Devices
X86-64

  Version:                           0x1

  Entry point address:               0x402f17

  Start of program headers:          64 (bytes into file)

  Start of section headers:          71032 (bytes into
file)

  Flags:                             0x0

  Size of this header:               64 (bytes)

  Size of program headers:           56 (bytes)

  Number of program headers:         10

  Size of section headers:           64 (bytes)

  Number of section headers:         31

  Section header string table index: 30

root@qemux86-64:/usr/share/dotnet#  file dotnet

dotnet: ELF 64-bit LSB executable, x86-64, version 1 (SYSV),
dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2,
for GNU/Linux 2.6.32,
BuildID[sha1]=28c244c1953bcbee994709a4b849086ee7cf0c99,
stripped

I compared those values with that from Python, which does
run on this system


----------------------------------------------------------------------
---------------------------------

root@qemux86-64:/opt/jre-8/bin# readelf -h
/usr/bin/python3.7

ELF Header:

  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00
00

  Class:                             ELF64

  Data:                              2's complement, little
endian

  Version:                           1 (current)

  OS/ABI:                            UNIX - System V

  ABI Version:                       0

  Type:                              DYN (Shared object
file)

  Machine:                           Advanced Micro Devices
X86-64

  Version:                           0x1

  Entry point address:               0x1060

  Start of program headers:          64 (bytes into file)

  Start of section headers:          12568 (bytes into
file)

  Flags:                             0x0

  Size of this header:               64 (bytes)

  Size of program headers:           56 (bytes)

  Number of program headers:         11

  Size of section headers:           64 (bytes)

  Number of section headers:         27

  Section header string table index: 26

root@qemux86-64:/usr/share/dotnet# file /usr/bin/python3.7

/usr/bin/python3.7: ELF 64-bit LSB pie executable, x86-64,
version 1 (SYSV), dynamically linked, interpreter
/lib/ld-linux-x86-64.so.2,
BuildID[sha1]=a455873f278466378405802b0e0171337e52a81c, for
GNU/Linux 3.2.0, stripped


======================================================================
==========

The only difference I found, is that Python is a "ELF 64-bit
LSB pie executable" whereas dotnet is a "ELF 64-bit LSB
executable". I tried to turn that PIE (seemed to be a gcc
option: --enable-default-pie) feature of, but that didn't
work well, and I couldn't find a way to remove it.

----

Best regards,

Christian Lohr

Im Auftrag von:

Carl Zeiss Meditec AG
Göschwitzer Strasse 51-52

07745 Jena, Deutschland

christian.lohr.ext@...
<mailto:christian.lohr.ext@...>

Tel: +493641220206



Join yocto@lists.yoctoproject.org to automatically receive all group messages.