Python function caching question


Michael Callahan <coder.callahan@...>
 

I am having trouble with sstate caching of my os-release.bbappend and
am stuck. The simple example file looks like something below, where I
am setting a variable from a computed python function. What's the magic
to make the find_version(d) always run but do_compile to only run if
VERSION changes? I basically want something like
`find_version[nostamp]="1"` but that seems to only work for tasks.

def find_version(d):
import subprocess
cmd = "git describe --long"
return subprocess.check_output(cmd).rstrip().decode('utf-8')

VERSION = "${@find_version(d)}"
# do_compile uses $VERSION


Khem Raj
 

On Sun, Nov 22, 2020 at 6:17 PM Michael Callahan
<coder.callahan@...> wrote:

I am having trouble with sstate caching of my os-release.bbappend and
am stuck. The simple example file looks like something below, where I
am setting a variable from a computed python function. What's the magic
to make the find_version(d) always run but do_compile to only run if
VERSION changes? I basically want something like
`find_version[nostamp]="1"` but that seems to only work for tasks.

def find_version(d):
import subprocess
cmd = "git describe --long"
return subprocess.check_output(cmd).rstrip().decode('utf-8')

VERSION = "${@find_version(d)}"
# do_compile uses $VERSION
perhaps mark the task which is using this variable as nostamp.



Michael Callahan <coder.callahan@...>
 

I do not want do_compile to run again, it rebuilds the whole image.
What is the best way to always run find_version?

On Sun, Nov 22, 2020 at 7:53 PM Khem Raj <raj.khem@...> wrote:

On Sun, Nov 22, 2020 at 6:17 PM Michael Callahan
<coder.callahan@...> wrote:

I am having trouble with sstate caching of my os-release.bbappend and
am stuck. The simple example file looks like something below, where I
am setting a variable from a computed python function. What's the magic
to make the find_version(d) always run but do_compile to only run if
VERSION changes? I basically want something like
`find_version[nostamp]="1"` but that seems to only work for tasks.

def find_version(d):
import subprocess
cmd = "git describe --long"
return subprocess.check_output(cmd).rstrip().decode('utf-8')

VERSION = "${@find_version(d)}"
# do_compile uses $VERSION
perhaps mark the task which is using this variable as nostamp.



Richard Purdie
 

On Sun, 2020-11-22 at 19:16 -0700, Michael Callahan wrote:
I am having trouble with sstate caching of my os-release.bbappend and
am stuck. The simple example file looks like something below, where
I
am setting a variable from a computed python function. What's the
magic
to make the find_version(d) always run but do_compile to only run if
VERSION changes? I basically want something like
`find_version[nostamp]="1"` but that seems to only work for tasks.

def find_version(d):
import subprocess
cmd = "git describe --long"
return subprocess.check_output(cmd).rstrip().decode('utf-8')

VERSION = "${@find_version(d)}"
# do_compile uses $VERSION
If you set BB_DONT_CACHE = "1" in the recipe, it will force the recipe
to reparse each time. That should cause it to rerun find_version and
then change hash when the value changes. Its how SRCREV = "${AUTOREV}"
works behind the scenes.

Cheers,

Richard


Quentin Schulz
 

Hi Michael,

On Mon, Nov 23, 2020 at 04:49:30PM -0700, Michael Callahan wrote:
I do not want do_compile to run again, it rebuilds the whole image.
What is the best way to always run find_version?
Use an anonymous python function that sets VERSION, e.g.:

python __anonymous() {
d.setVar('VERSION', find_version(d))
}

as anonymous python functions are always executed during parsing.

Then add the following to your recipe:
do_compile[vardepsexclude] += "VERSION"

VERSION will always be set appropriately every time the recipe is parsed
but do_compile will only be rerun when a change in the task has occurred
to the exception of the VERSION variable.

No idea if that works as I don't use those, but that's an hint. Anyway, you
probably want a behavior similar to DATETIME's.

Cheers,
Quentin

On Sun, Nov 22, 2020 at 7:53 PM Khem Raj <raj.khem@...> wrote:

On Sun, Nov 22, 2020 at 6:17 PM Michael Callahan
<coder.callahan@...> wrote:

I am having trouble with sstate caching of my os-release.bbappend and
am stuck. The simple example file looks like something below, where I
am setting a variable from a computed python function. What's the magic
to make the find_version(d) always run but do_compile to only run if
VERSION changes? I basically want something like
`find_version[nostamp]="1"` but that seems to only work for tasks.

def find_version(d):
import subprocess
cmd = "git describe --long"
return subprocess.check_output(cmd).rstrip().decode('utf-8')

VERSION = "${@find_version(d)}"
# do_compile uses $VERSION
perhaps mark the task which is using this variable as nostamp.





--
StreamUnlimited Engineering GmbH
High Tech Campus Vienna, Gutheil-Schoder-Gasse 10, 1100 Vienna, Austria
Fax: +43 1 667 20 02 4401
quentin.schulz@..., www.streamunlimited.com