Topics

Can't find pyconfig.h with python3


Emily
 

Hi all - 

I'm trying to build an OS with a custom recipe (https://github.com/kratsg/meta-l1calo/blob/add/opcServer/recipes-core/opc-ua/opc-ua-server-gfex_git.bb). I can build it as-is in the link above, but when actually trying to use the output of the recipe (Poverty.so) I get the below error in python3: 
>>> import Poverty Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: /usr/bin/Poverty.so: undefined symbol: _ZN5boost6python6detail11init_moduleEPKcPFvvE

I think that's because I built against python2 in some places. So I replaced essentially all of the python dependencies with their corresponding python3 dependencies, but that gives a build error in bitbake like:  
| /local/d6/easmith5/rocko_bitbake/poky/build/tmp/work/aarch64-poky-linux/opc-ua-server-gfex/1.0+gitAUTOINC+921c563309-r0/recipe-sysroot/usr/include/boost/python/detail/wrap_python.hpp:50:11: fatal error: pyconfig.h: No such file or directory
|  # include <pyconfig.h>
|            ^~~~~~~~~~~~
| compilation terminated.

However, I checked and the offending file pyconfig.h is actually present at | /local/d6/easmith5/rocko_bitbake/poky/build/tmp/work/aarch64-poky-linux/opc-ua-server-gfex/1.0+gitAUTOINC+921c563309-r0/recipe-sysroot/usr/include/python3.5m/. When I build with python2 dependencies the file is in the same location, just /python2.7/.

I can't quite tell if this is a yocto problem, or a problem with my recipe and/or build files for that recipe. If anyone has suggestions they would be much appreciated! 

Thanks!
Emily Smith


Khem Raj
 

On 4/24/20 4:17 PM, Emily wrote:
Hi all - 

I'm trying to build an OS with a custom recipe
(https://github.com/kratsg/meta-l1calo/blob/add/opcServer/recipes-core/opc-ua/opc-ua-server-gfex_git.bb).
I can build it as-is in the link above, but when actually trying to use
the output of the recipe (Poverty.so) I get the below error in python3: 

import Poverty Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: /usr/bin/Poverty.so: undefined symbol: _ZN5boost6python6detail11init_moduleEPKcPFvvE

I think that's because I built against python2 in some places. So I
replaced essentially all of the python dependencies with their
corresponding python3 dependencies, but that gives a build error in
bitbake like:  

| /local/d6/easmith5/rocko_bitbake/poky/build/tmp/work/aarch64-poky-linux/opc-ua-server-gfex/1.0+gitAUTOINC+921c563309-r0/recipe-sysroot/usr/include/boost/python/detail/wrap_python.hpp:50:11: fatal error: pyconfig.h: No such file or directory
|  # include <pyconfig.h>
|            ^~~~~~~~~~~~
| compilation terminated.


However, I checked and the offending file pyconfig.h is actually present
at |
/local/d6/easmith5/rocko_bitbake/poky/build/tmp/work/aarch64-poky-linux/opc-ua-server-gfex/1.0+gitAUTOINC+921c563309-r0/recipe-sysroot/usr/include/python3.5m/.
When I build with python2 dependencies the file is in the same location,
just /python2.7/.

I can't quite tell if this is a yocto problem, or a problem with my
recipe and/or build files for that recipe. If anyone has suggestions
they would be much appreciated!
this seems to be that it is needing python2, so yes it can perhaps be
ported to py3 but you might have to check if it needs some code changes

Looking at the recipe it is ineheriting py2 class here [1]

perhaps you should change that to python3native

[1]
https://github.com/kratsg/meta-l1calo/blob/add/opcServer/recipes-core/opc-ua/opc-ua-server-gfex_git.bb#L15




Thanks!
Emily Smith




Tim Orling
 


I see you are building for "rocko" and that your layer supports "sumo" and "rocko". Do you have a strong compelling reason to stay on these EOL releases? meta-xilinx has a zeus branch which was released in October 2019 and has newer components and will be easier for you and us to support. (We've had "thud" and "warrior" releases in-between, so that's at least a year and a half ahead of your code base).

I started trying to build your layer (on "zeus" release), but unfortunately you have recipes fetching from internal CERN git repos that are not accessible to the rest of us.

Here's what I have so far:

Again, I can't go much further because I can't fetch from your internal repos.

On Fri, Apr 24, 2020 at 6:38 PM Khem Raj <raj.khem@...> wrote:


On 4/24/20 4:17 PM, Emily wrote:
> Hi all - 
>
> I'm trying to build an OS with a custom recipe
> (https://github.com/kratsg/meta-l1calo/blob/add/opcServer/recipes-core/opc-ua/opc-ua-server-gfex_git.bb).
> I can build it as-is in the link above, but when actually trying to use
> the output of the recipe (Poverty.so) I get the below error in python3: 
>
>>>> import Poverty Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: /usr/bin/Poverty.so: undefined symbol: _ZN5boost6python6detail11init_moduleEPKcPFvvE
>
>
> I think that's because I built against python2 in some places. So I
> replaced essentially all of the python dependencies with their
> corresponding python3 dependencies, but that gives a build error in
> bitbake like:  
>
> | /local/d6/easmith5/rocko_bitbake/poky/build/tmp/work/aarch64-poky-linux/opc-ua-server-gfex/1.0+gitAUTOINC+921c563309-r0/recipe-sysroot/usr/include/boost/python/detail/wrap_python.hpp:50:11: fatal error: pyconfig.h: No such file or directory
> |  # include <pyconfig.h>
> |            ^~~~~~~~~~~~
> | compilation terminated.
>
>
> However, I checked and the offending file pyconfig.h is actually present
> at |
> /local/d6/easmith5/rocko_bitbake/poky/build/tmp/work/aarch64-poky-linux/opc-ua-server-gfex/1.0+gitAUTOINC+921c563309-r0/recipe-sysroot/usr/include/python3.5m/.
> When I build with python2 dependencies the file is in the same location,
> just /python2.7/.
>
> I can't quite tell if this is a yocto problem, or a problem with my
> recipe and/or build files for that recipe. If anyone has suggestions
> they would be much appreciated!

this seems to be that it is needing python2, so yes it can perhaps be
ported to py3 but you might have to check if it needs some code changes

Looking at the recipe it is ineheriting py2 class here [1]

perhaps you should change that to python3native

[1]
https://github.com/kratsg/meta-l1calo/blob/add/opcServer/recipes-core/opc-ua/opc-ua-server-gfex_git.bb#L15



>
> Thanks!
> Emily Smith
>
>
>
>



Tim Orling
 

I meant to say "zeus" was released in October 2019. I'm sure xilinx support was after that.


On Fri, Apr 24, 2020 at 8:00 PM Tim Orling <ticotimo@...> wrote:

I see you are building for "rocko" and that your layer supports "sumo" and "rocko". Do you have a strong compelling reason to stay on these EOL releases? meta-xilinx has a zeus branch which was released in October 2019 and has newer components and will be easier for you and us to support. (We've had "thud" and "warrior" releases in-between, so that's at least a year and a half ahead of your code base).

I started trying to build your layer (on "zeus" release), but unfortunately you have recipes fetching from internal CERN git repos that are not accessible to the rest of us.

Here's what I have so far:

Again, I can't go much further because I can't fetch from your internal repos.

On Fri, Apr 24, 2020 at 6:38 PM Khem Raj <raj.khem@...> wrote:


On 4/24/20 4:17 PM, Emily wrote:
> Hi all - 
>
> I'm trying to build an OS with a custom recipe
> (https://github.com/kratsg/meta-l1calo/blob/add/opcServer/recipes-core/opc-ua/opc-ua-server-gfex_git.bb).
> I can build it as-is in the link above, but when actually trying to use
> the output of the recipe (Poverty.so) I get the below error in python3: 
>
>>>> import Poverty Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: /usr/bin/Poverty.so: undefined symbol: _ZN5boost6python6detail11init_moduleEPKcPFvvE
>
>
> I think that's because I built against python2 in some places. So I
> replaced essentially all of the python dependencies with their
> corresponding python3 dependencies, but that gives a build error in
> bitbake like:  
>
> | /local/d6/easmith5/rocko_bitbake/poky/build/tmp/work/aarch64-poky-linux/opc-ua-server-gfex/1.0+gitAUTOINC+921c563309-r0/recipe-sysroot/usr/include/boost/python/detail/wrap_python.hpp:50:11: fatal error: pyconfig.h: No such file or directory
> |  # include <pyconfig.h>
> |            ^~~~~~~~~~~~
> | compilation terminated.
>
>
> However, I checked and the offending file pyconfig.h is actually present
> at |
> /local/d6/easmith5/rocko_bitbake/poky/build/tmp/work/aarch64-poky-linux/opc-ua-server-gfex/1.0+gitAUTOINC+921c563309-r0/recipe-sysroot/usr/include/python3.5m/.
> When I build with python2 dependencies the file is in the same location,
> just /python2.7/.
>
> I can't quite tell if this is a yocto problem, or a problem with my
> recipe and/or build files for that recipe. If anyone has suggestions
> they would be much appreciated!

this seems to be that it is needing python2, so yes it can perhaps be
ported to py3 but you might have to check if it needs some code changes

Looking at the recipe it is ineheriting py2 class here [1]

perhaps you should change that to python3native

[1]
https://github.com/kratsg/meta-l1calo/blob/add/opcServer/recipes-core/opc-ua/opc-ua-server-gfex_git.bb#L15



>
> Thanks!
> Emily Smith
>
>
>
>



Emily
 

Hi -

Not a super compelling reason other than the momentum required to update! Does Zeus support having both python2 and python3? We have a couple things that haven’t upgraded to python3 yet as well. It’s almost certainly a good idea to move to Zeus, but everything is further complicated because we don’t have physical access to the board right now to update the OS and make sure things still work. 

Sorry about the internal repos, but thanks for the start! 

Best,
Emily 


On Apr 24, 2020, at 10:01 PM, Tim Orling <ticotimo@...> wrote:


I meant to say "zeus" was released in October 2019. I'm sure xilinx support was after that.

On Fri, Apr 24, 2020 at 8:00 PM Tim Orling <ticotimo@...> wrote:

I see you are building for "rocko" and that your layer supports "sumo" and "rocko". Do you have a strong compelling reason to stay on these EOL releases? meta-xilinx has a zeus branch which was released in October 2019 and has newer components and will be easier for you and us to support. (We've had "thud" and "warrior" releases in-between, so that's at least a year and a half ahead of your code base).

I started trying to build your layer (on "zeus" release), but unfortunately you have recipes fetching from internal CERN git repos that are not accessible to the rest of us.

Here's what I have so far:

Again, I can't go much further because I can't fetch from your internal repos.

On Fri, Apr 24, 2020 at 6:38 PM Khem Raj <raj.khem@...> wrote:


On 4/24/20 4:17 PM, Emily wrote:
> Hi all - 
>
> I'm trying to build an OS with a custom recipe
> (https://github.com/kratsg/meta-l1calo/blob/add/opcServer/recipes-core/opc-ua/opc-ua-server-gfex_git.bb).
> I can build it as-is in the link above, but when actually trying to use
> the output of the recipe (Poverty.so) I get the below error in python3: 
>
>>>> import Poverty Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: /usr/bin/Poverty.so: undefined symbol: _ZN5boost6python6detail11init_moduleEPKcPFvvE
>
>
> I think that's because I built against python2 in some places. So I
> replaced essentially all of the python dependencies with their
> corresponding python3 dependencies, but that gives a build error in
> bitbake like:  
>
> | /local/d6/easmith5/rocko_bitbake/poky/build/tmp/work/aarch64-poky-linux/opc-ua-server-gfex/1.0+gitAUTOINC+921c563309-r0/recipe-sysroot/usr/include/boost/python/detail/wrap_python.hpp:50:11: fatal error: pyconfig.h: No such file or directory
> |  # include <pyconfig.h>
> |            ^~~~~~~~~~~~
> | compilation terminated.
>
>
> However, I checked and the offending file pyconfig.h is actually present
> at |
> /local/d6/easmith5/rocko_bitbake/poky/build/tmp/work/aarch64-poky-linux/opc-ua-server-gfex/1.0+gitAUTOINC+921c563309-r0/recipe-sysroot/usr/include/python3.5m/.
> When I build with python2 dependencies the file is in the same location,
> just /python2.7/.
>
> I can't quite tell if this is a yocto problem, or a problem with my
> recipe and/or build files for that recipe. If anyone has suggestions
> they would be much appreciated!

this seems to be that it is needing python2, so yes it can perhaps be
ported to py3 but you might have to check if it needs some code changes

Looking at the recipe it is ineheriting py2 class here [1]

perhaps you should change that to python3native

[1]
https://github.com/kratsg/meta-l1calo/blob/add/opcServer/recipes-core/opc-ua/opc-ua-server-gfex_git.bb#L15



>
> Thanks!
> Emily Smith
>
>
>
>



Tim Orling
 



On Fri, Apr 24, 2020 at 8:28 PM Emily Smith <easmith5555@...> wrote:
Hi -

Not a super compelling reason other than the momentum required to update! Does Zeus support having both python2 and python3? We have a couple things that haven’t upgraded to python3 yet as well. It’s almost certainly a good idea to move to Zeus, but everything is further complicated because we don’t have physical access to the board right now to update the OS and make sure things still work. 

Right. I remember a prior thread about you needing to update remotely. I assume this board is stuffed deep in ATLAS somewhere. (In a prior life I was a materials scientist)

Are you stuck with "rocko" on the deeply buried remote-only device or can you use "sumo"?

Zeus still had python2 [1] and python3 [2]. But you can also get python2 support with meta-python2 [3], since the latest release "dunfell" [4].  For random development reasons, the zeus branch should also work [5]. Note that meta-python2 will have limited support going forward (as in I might not support it after this month, but others might). It was created so we could drop python2 from the other layers, without breaking everything everywhere for everyone.


Sorry about the internal repos, but thanks for the start! 

Best,
Emily 


On Apr 24, 2020, at 10:01 PM, Tim Orling <ticotimo@...> wrote:


I meant to say "zeus" was released in October 2019. I'm sure xilinx support was after that.

On Fri, Apr 24, 2020 at 8:00 PM Tim Orling <ticotimo@...> wrote:

I see you are building for "rocko" and that your layer supports "sumo" and "rocko". Do you have a strong compelling reason to stay on these EOL releases? meta-xilinx has a zeus branch which was released in October 2019 and has newer components and will be easier for you and us to support. (We've had "thud" and "warrior" releases in-between, so that's at least a year and a half ahead of your code base).

I started trying to build your layer (on "zeus" release), but unfortunately you have recipes fetching from internal CERN git repos that are not accessible to the rest of us.

Here's what I have so far:

Again, I can't go much further because I can't fetch from your internal repos.

On Fri, Apr 24, 2020 at 6:38 PM Khem Raj <raj.khem@...> wrote:


On 4/24/20 4:17 PM, Emily wrote:
> Hi all - 
>
> I'm trying to build an OS with a custom recipe
> (https://github.com/kratsg/meta-l1calo/blob/add/opcServer/recipes-core/opc-ua/opc-ua-server-gfex_git.bb).
> I can build it as-is in the link above, but when actually trying to use
> the output of the recipe (Poverty.so) I get the below error in python3: 
>
>>>> import Poverty Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: /usr/bin/Poverty.so: undefined symbol: _ZN5boost6python6detail11init_moduleEPKcPFvvE
>
>
> I think that's because I built against python2 in some places. So I
> replaced essentially all of the python dependencies with their
> corresponding python3 dependencies, but that gives a build error in
> bitbake like:  
>
> | /local/d6/easmith5/rocko_bitbake/poky/build/tmp/work/aarch64-poky-linux/opc-ua-server-gfex/1.0+gitAUTOINC+921c563309-r0/recipe-sysroot/usr/include/boost/python/detail/wrap_python.hpp:50:11: fatal error: pyconfig.h: No such file or directory
> |  # include <pyconfig.h>
> |            ^~~~~~~~~~~~
> | compilation terminated.
>
>
> However, I checked and the offending file pyconfig.h is actually present
> at |
> /local/d6/easmith5/rocko_bitbake/poky/build/tmp/work/aarch64-poky-linux/opc-ua-server-gfex/1.0+gitAUTOINC+921c563309-r0/recipe-sysroot/usr/include/python3.5m/.
> When I build with python2 dependencies the file is in the same location,
> just /python2.7/.
>
> I can't quite tell if this is a yocto problem, or a problem with my
> recipe and/or build files for that recipe. If anyone has suggestions
> they would be much appreciated!

this seems to be that it is needing python2, so yes it can perhaps be
ported to py3 but you might have to check if it needs some code changes

Looking at the recipe it is ineheriting py2 class here [1]

perhaps you should change that to python3native

[1]
https://github.com/kratsg/meta-l1calo/blob/add/opcServer/recipes-core/opc-ua/opc-ua-server-gfex_git.bb#L15



>
> Thanks!
> Emily Smith
>
>
>
>



Emily
 

Hi -

The board isn’t buried *too* deeply yet, but lab restrictions are the main barrier at the moment. Hopefully I can figure out how to remotely update at some point, but I should be able to test with Zeus before we actually bury it underground. Right now it has rocko, and I’ve been updating things piecemeal as I go. 

Ah I see, so perhaps Zeus is ideal for me until we can drop python2 entirely. Thanks for the info and links! 

Best,
Emily

On Apr 24, 2020, at 10:59 PM, Tim Orling <ticotimo@...> wrote:




On Fri, Apr 24, 2020 at 8:28 PM Emily Smith <easmith5555@...> wrote:
Hi -

Not a super compelling reason other than the momentum required to update! Does Zeus support having both python2 and python3? We have a couple things that haven’t upgraded to python3 yet as well. It’s almost certainly a good idea to move to Zeus, but everything is further complicated because we don’t have physical access to the board right now to update the OS and make sure things still work. 

Right. I remember a prior thread about you needing to update remotely. I assume this board is stuffed deep in ATLAS somewhere. (In a prior life I was a materials scientist)

Are you stuck with "rocko" on the deeply buried remote-only device or can you use "sumo"?

Zeus still had python2 [1] and python3 [2]. But you can also get python2 support with meta-python2 [3], since the latest release "dunfell" [4].  For random development reasons, the zeus branch should also work [5]. Note that meta-python2 will have limited support going forward (as in I might not support it after this month, but others might). It was created so we could drop python2 from the other layers, without breaking everything everywhere for everyone.


Sorry about the internal repos, but thanks for the start! 

Best,
Emily 


On Apr 24, 2020, at 10:01 PM, Tim Orling <ticotimo@...> wrote:


I meant to say "zeus" was released in October 2019. I'm sure xilinx support was after that.

On Fri, Apr 24, 2020 at 8:00 PM Tim Orling <ticotimo@...> wrote:

I see you are building for "rocko" and that your layer supports "sumo" and "rocko". Do you have a strong compelling reason to stay on these EOL releases? meta-xilinx has a zeus branch which was released in October 2019 and has newer components and will be easier for you and us to support. (We've had "thud" and "warrior" releases in-between, so that's at least a year and a half ahead of your code base).

I started trying to build your layer (on "zeus" release), but unfortunately you have recipes fetching from internal CERN git repos that are not accessible to the rest of us.

Here's what I have so far:

Again, I can't go much further because I can't fetch from your internal repos.

On Fri, Apr 24, 2020 at 6:38 PM Khem Raj <raj.khem@...> wrote:


On 4/24/20 4:17 PM, Emily wrote:
> Hi all - 
>
> I'm trying to build an OS with a custom recipe
> (https://github.com/kratsg/meta-l1calo/blob/add/opcServer/recipes-core/opc-ua/opc-ua-server-gfex_git.bb).
> I can build it as-is in the link above, but when actually trying to use
> the output of the recipe (Poverty.so) I get the below error in python3: 
>
>>>> import Poverty Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: /usr/bin/Poverty.so: undefined symbol: _ZN5boost6python6detail11init_moduleEPKcPFvvE
>
>
> I think that's because I built against python2 in some places. So I
> replaced essentially all of the python dependencies with their
> corresponding python3 dependencies, but that gives a build error in
> bitbake like:  
>
> | /local/d6/easmith5/rocko_bitbake/poky/build/tmp/work/aarch64-poky-linux/opc-ua-server-gfex/1.0+gitAUTOINC+921c563309-r0/recipe-sysroot/usr/include/boost/python/detail/wrap_python.hpp:50:11: fatal error: pyconfig.h: No such file or directory
> |  # include <pyconfig.h>
> |            ^~~~~~~~~~~~
> | compilation terminated.
>
>
> However, I checked and the offending file pyconfig.h is actually present
> at |
> /local/d6/easmith5/rocko_bitbake/poky/build/tmp/work/aarch64-poky-linux/opc-ua-server-gfex/1.0+gitAUTOINC+921c563309-r0/recipe-sysroot/usr/include/python3.5m/.
> When I build with python2 dependencies the file is in the same location,
> just /python2.7/.
>
> I can't quite tell if this is a yocto problem, or a problem with my
> recipe and/or build files for that recipe. If anyone has suggestions
> they would be much appreciated!

this seems to be that it is needing python2, so yes it can perhaps be
ported to py3 but you might have to check if it needs some code changes

Looking at the recipe it is ineheriting py2 class here [1]

perhaps you should change that to python3native

[1]
https://github.com/kratsg/meta-l1calo/blob/add/opcServer/recipes-core/opc-ua/opc-ua-server-gfex_git.bb#L15



>
> Thanks!
> Emily Smith
>
>
>
>