Date   

[PATCH] matchbox-wm: Fix build with automake-1.13

Marko Lindqvist <cazfi74@...>
 

Automake-1.13 fix to matchbox-wm git attached.


- ML


Enabling shared state mirrors

Thornburg, Christopher A <christopher.a.thornburg@...>
 

Struggling to get shared state mirrors to work. I can see files with the same file name (and therefore the same hash) in my mirror and, after a build, in my local sstate cache direcotry. Ala:

/storage/yocto-cache/sstate-toolchain/Ubuntu-10.04/95/sstate-bzip2-native-i686-linux-1.0.6-r5-i686-2-95b6058a3f3f35386c8593e8447c59da_populate-lic.tgz.siginfo

and

<build dir>/sstate-cache/Ubuntu-10.04/95/sstate-bzip2-native-i686-linux-1.0.6-r5-i686-2-95b6058a3f3f35386c8593e8447c59da_populate-lic.tgz.siginfo

 

bitbake-diffsigs confirms that the signatures match. I have the following in my local.conf:

SSTATE_MIRRORS ?= "\

file://.* file:///storage/yocto-cache/sstate-toolchain/PATH \

file://.* file:///storage/yocto-cache/sstate-kernel/PATH"

 

…and yet bitbake insists on rebuilding everything every time. I had this working yesterday, and today it fails again.

 

Any suggestions?

 


[PATCH 7/7] scripts/lib/bsp/engine.py: add handling for JSON strings

tom.zanussi@...
 

From: Tom Zanussi <tom.zanussi@...>

Normally pre-canned properties are supplied as JSON from a file, which
the user can specify using e.g. the -i option.

We can reuse that basic functionality for dedicated command-line
parameters by sythesizing a JSON string containing those param values
on the fly and passing that in instead.

This adds the ability for the common creation code to accept JSON
strings as well as JSON files.

Signed-off-by: Tom Zanussi <tom.zanussi@...>
---
scripts/lib/bsp/engine.py | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/scripts/lib/bsp/engine.py b/scripts/lib/bsp/engine.py
index 7431860..aa26280 100644
--- a/scripts/lib/bsp/engine.py
+++ b/scripts/lib/bsp/engine.py
@@ -1387,6 +1387,9 @@ def yocto_common_create(machine, target, scripts_path, layer_output_dir, codedum

properties = json.load(infile)

+ if properties_str and not properties:
+ properties = json.loads(properties_str)
+
os.mkdir(layer_output_dir)

context = create_context(machine, target, scripts_path)
@@ -1413,7 +1416,7 @@ def yocto_common_create(machine, target, scripts_path, layer_output_dir, codedum
run_program_lines(program_lines, codedump)


-def yocto_layer_create(layer_name, scripts_path, layer_output_dir, codedump, properties_file):
+def yocto_layer_create(layer_name, scripts_path, layer_output_dir, codedump, properties_file, properties=""):
"""
Create yocto layer

@@ -1424,13 +1427,13 @@ def yocto_layer_create(layer_name, scripts_path, layer_output_dir, codedump, pro
properties_file - use values from this file if nonempty i.e no prompting
properties - use values from this string if nonempty i.e no prompting
"""
- yocto_common_create(layer_name, "layer", scripts_path, layer_output_dir, codedump, properties_file, False)
+ yocto_common_create(layer_name, "layer", scripts_path, layer_output_dir, codedump, properties_file, properties, False)

print "\nNew layer created in %s.\n" % (layer_output_dir)
print "Don't forget to add it to your BBLAYERS (for details see %s\README)." % (layer_output_dir)


-def yocto_bsp_create(machine, arch, scripts_path, bsp_output_dir, codedump, properties_file):
+def yocto_bsp_create(machine, arch, scripts_path, bsp_output_dir, codedump, properties_file, properties=None):
"""
Create bsp

@@ -1443,7 +1446,7 @@ def yocto_bsp_create(machine, arch, scripts_path, bsp_output_dir, codedump, prop
properties_file - use values from this file if nonempty i.e no prompting
properties - use values from this string if nonempty i.e no prompting
"""
- yocto_common_create(machine, arch, scripts_path, bsp_output_dir, codedump, properties_file)
+ yocto_common_create(machine, arch, scripts_path, bsp_output_dir, codedump, properties_file, properties)

print "\nNew %s BSP created in %s" % (arch, bsp_output_dir)

--
1.7.11.4


[PATCH 6/7] yocto-layer: add optional layer priority param

tom.zanussi@...
 

From: Tom Zanussi <tom.zanussi@...>

If the user specifies a layer priority following the layer name, layer
creation will proceed without further queries using the specified
layer priority and the remaining values defaulted.

Signed-off-by: Tom Zanussi <tom.zanussi@...>
---
scripts/lib/bsp/help.py | 14 ++++++++++++--
scripts/yocto-layer | 9 +++++++--
2 files changed, 19 insertions(+), 4 deletions(-)

diff --git a/scripts/lib/bsp/help.py b/scripts/lib/bsp/help.py
index eac172a..346bf0f 100644
--- a/scripts/lib/bsp/help.py
+++ b/scripts/lib/bsp/help.py
@@ -622,7 +622,8 @@ yocto_layer_create_usage = """

Create a new generic Yocto layer

- usage: yocto-layer create <layer-name> [-o <DIRNAME> | --outdir <DIRNAME>]
+ usage: yocto-layer create <layer-name> [layer_priority]
+ [-o <DIRNAME> | --outdir <DIRNAME>]
[-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]

This command creates a generic Yocto layer based on the specified
@@ -631,6 +632,10 @@ yocto_layer_create_usage = """
'meta-layer-name'. The -o option can be used to place the layer in a
directory with a different name and location.

+ If layer_priority is specified, a simple layer will be created using
+ the given layer priority, and the user will not be prompted for
+ further input.
+
NOTE: Once created, you should add your new layer to your
bblayers.conf file in order for it to be subsequently seen and
modified by the yocto-kernel tool. Instructions for doing this can
@@ -646,7 +651,8 @@ NAME
yocto-layer create - Create a new generic Yocto layer

SYNOPSIS
- yocto-layer create <layer-name> [-o <DIRNAME> | --outdir <DIRNAME>]
+ yocto-layer create <layer-name> [layer_priority]
+ [-o <DIRNAME> | --outdir <DIRNAME>]
[-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]

DESCRIPTION
@@ -656,6 +662,10 @@ DESCRIPTION
'meta-layer-name'. The -o option can be used to place the layer
in a directory with a different name and location.

+ If layer_priority is specified, a simple layer will be created
+ using the given layer priority, and the user will not be prompted
+ for further input.
+
The layer-specific properties that define the values that will be
used to generate the layer can be specified on the command-line
using the -i option and supplying a JSON object consisting of the
diff --git a/scripts/yocto-layer b/scripts/yocto-layer
index f759275..53d2aab 100755
--- a/scripts/yocto-layer
+++ b/scripts/yocto-layer
@@ -60,19 +60,24 @@ def yocto_layer_create_subcommand(args, usage_str):
default = False, help = "dump the generated code to layergen.out")
(options, args) = parser.parse_args(args)

- if len(args) != 1:
+ if len(args) < 1 or len(args) > 2:
logging.error("Wrong number of arguments, exiting\n")
parser.print_help()
sys.exit(1)

layer_name = args[0]
+ properties = ""
+
+ if len(args) == 2:
+ layer_priority = args[1]
+ properties = '{"layer_priority":"' + layer_priority + '"}'

if options.outdir:
layer_output_dir = options.outdir
else:
layer_output_dir = "meta-" + layer_name

- yocto_layer_create(layer_name, scripts_path, layer_output_dir, options.codedump, options.properties_file)
+ yocto_layer_create(layer_name, scripts_path, layer_output_dir, options.codedump, options.properties_file, properties)


def yocto_layer_list_subcommand(args, usage_str):
--
1.7.11.4


[PATCH 5/7] scripts/lib/bsp/engine.py: refactor bsp-creation code

tom.zanussi@...
 

From: Tom Zanussi <tom.zanussi@...>

This does a bit of refactoring of the bsp-generation code to make it
generically reusable for generating non-bsp layers.

The first user remains the existing yocto-bsp tool; these changes
allow a second user, the new yocto-layer tool, to use the same code.

Signed-off-by: Tom Zanussi <tom.zanussi@...>
---
scripts/lib/bsp/engine.py | 141 +++++++++++++++++++++++++---------------------
1 file changed, 76 insertions(+), 65 deletions(-)

diff --git a/scripts/lib/bsp/engine.py b/scripts/lib/bsp/engine.py
index d406e79..7431860 100644
--- a/scripts/lib/bsp/engine.py
+++ b/scripts/lib/bsp/engine.py
@@ -1039,7 +1039,9 @@ def gen_program_machine_lines(machine, program_lines):
Use the input values we got from the command line.
"""
line = "machine = \"" + machine + "\""
+ program_lines.append(line)

+ line = "layer_name = \"" + machine + "\""
program_lines.append(line)


@@ -1321,10 +1323,13 @@ def capture_context(context):
return captured_context


-def expand_targets(context, bsp_output_dir):
+def expand_targets(context, bsp_output_dir, expand_common=True):
"""
Expand all the tags in both the common and machine-specific
'targets'.
+
+ If expand_common is False, don't expand the common target (this
+ option is used to create special-purpose layers).
"""
target_files = []

@@ -1336,8 +1341,9 @@ def expand_targets(context, bsp_output_dir):
bsp_path = lib_path + '/bsp'
arch_path = bsp_path + '/substrate/target/arch'

- common = os.path.join(arch_path, "common")
- expand_target(common, target_files, bsp_output_dir)
+ if expand_common:
+ common = os.path.join(arch_path, "common")
+ expand_target(common, target_files, bsp_output_dir)

arches = os.listdir(arch_path)
if arch not in arches or arch == "common":
@@ -1352,21 +1358,22 @@ def expand_targets(context, bsp_output_dir):
return target_files


-def yocto_layer_create(layer_name, scripts_path, layer_output_dir, codedump, properties_file):
+def yocto_common_create(machine, target, scripts_path, layer_output_dir, codedump, properties_file, properties_str="", expand_common=True):
"""
- Create yocto layer
+ Common layer-creation code

- layer_name - user-defined layer name
+ machine - user-defined machine name (if needed, will generate 'machine' var)
+ target - the 'target' the layer will be based on, must be one in
+ scripts/lib/bsp/substrate/target/arch
scripts_path - absolute path to yocto /scripts dir
- bsp_output_dir - dirname to create for BSP
+ layer_output_dir - dirname to create for layer
codedump - dump generated code to bspgen.out
- properties_file - use values from here if nonempty i.e no prompting
-
- arch - the arch for a generic layer is 'generic-layer', defined in
- scripts/lib/bsp/substrate/target/generic-layer
+ properties_file - use values from this file if nonempty i.e no prompting
+ properties_str - use values from this string if nonempty i.e no prompting
+ expand_common - boolean, use the contents of (for bsp layers) arch/common
"""
- if os.path.exists(bsp_output_dir):
- print "\nBSP output dir already exists, exiting. (%s)" % bsp_output_dir
+ if os.path.exists(layer_output_dir):
+ print "\nlayer output dir already exists, exiting. (%s)" % layer_output_dir
sys.exit(1)

properties = None
@@ -1380,10 +1387,10 @@ def yocto_layer_create(layer_name, scripts_path, layer_output_dir, codedump, pro

properties = json.load(infile)

- os.mkdir(bsp_output_dir)
+ os.mkdir(layer_output_dir)

- context = create_context(machine, arch, scripts_path)
- target_files = expand_targets(context, bsp_output_dir)
+ context = create_context(machine, target, scripts_path)
+ target_files = expand_targets(context, layer_output_dir, expand_common)

input_lines = gather_inputlines(target_files)

@@ -1405,7 +1412,22 @@ def yocto_layer_create(layer_name, scripts_path, layer_output_dir, codedump, pro

run_program_lines(program_lines, codedump)

- print "New %s BSP created in %s" % (arch, bsp_output_dir)
+
+def yocto_layer_create(layer_name, scripts_path, layer_output_dir, codedump, properties_file):
+ """
+ Create yocto layer
+
+ layer_name - user-defined layer name
+ scripts_path - absolute path to yocto /scripts dir
+ layer_output_dir - dirname to create for layer
+ codedump - dump generated code to bspgen.out
+ properties_file - use values from this file if nonempty i.e no prompting
+ properties - use values from this string if nonempty i.e no prompting
+ """
+ yocto_common_create(layer_name, "layer", scripts_path, layer_output_dir, codedump, properties_file, False)
+
+ print "\nNew layer created in %s.\n" % (layer_output_dir)
+ print "Don't forget to add it to your BBLAYERS (for details see %s\README)." % (layer_output_dir)


def yocto_bsp_create(machine, arch, scripts_path, bsp_output_dir, codedump, properties_file):
@@ -1418,49 +1440,12 @@ def yocto_bsp_create(machine, arch, scripts_path, bsp_output_dir, codedump, prop
scripts_path - absolute path to yocto /scripts dir
bsp_output_dir - dirname to create for BSP
codedump - dump generated code to bspgen.out
- properties_file - use values from here if nonempty i.e no prompting
+ properties_file - use values from this file if nonempty i.e no prompting
+ properties - use values from this string if nonempty i.e no prompting
"""
- if os.path.exists(bsp_output_dir):
- print "\nBSP output dir already exists, exiting. (%s)" % bsp_output_dir
- sys.exit(1)
+ yocto_common_create(machine, arch, scripts_path, bsp_output_dir, codedump, properties_file)

- properties = None
-
- if properties_file:
- try:
- infile = open(properties_file, "r")
- except IOError:
- print "Couldn't open properties file %s for reading, exiting" % properties_file
- sys.exit(1)
-
- properties = json.load(infile)
-
- os.mkdir(bsp_output_dir)
-
- context = create_context(machine, arch, scripts_path)
- target_files = expand_targets(context, bsp_output_dir)
-
- input_lines = gather_inputlines(target_files)
-
- program_lines = []
-
- gen_program_header_lines(program_lines)
-
- gen_initial_property_vals(input_lines, program_lines)
-
- if properties:
- gen_supplied_property_vals(properties, program_lines)
-
- gen_program_machine_lines(machine, program_lines)
-
- if not properties:
- gen_program_input_lines(input_lines, program_lines, context)
-
- gen_program_lines(target_files, program_lines)
-
- run_program_lines(program_lines, codedump)
-
- print "New %s BSP created in %s" % (arch, bsp_output_dir)
+ print "\nNew %s BSP created in %s" % (arch, bsp_output_dir)


def print_dict(items, indent = 0):
@@ -1508,15 +1493,15 @@ def get_properties(input_lines):
return properties


-def yocto_bsp_list_properties(arch, scripts_path, properties_file):
+def yocto_layer_list_properties(arch, scripts_path, properties_file, expand_common=True):
"""
List the complete set of properties for all the input items in the
- BSP. If properties_file is non-null, write the complete set of
+ layer. If properties_file is non-null, write the complete set of
properties as a nested JSON object corresponding to a possibly
nested dictionary.
"""
context = create_context("unused", arch, scripts_path)
- target_files = expand_targets(context, "unused")
+ target_files = expand_targets(context, "unused", expand_common)

input_lines = gather_inputlines(target_files)

@@ -1605,7 +1590,7 @@ def print_values(type, values_list):
print "[\"%s\", \"%s\"]" % (value[0], value[1])


-def yocto_bsp_list_property_values(arch, property, scripts_path, properties_file):
+def yocto_layer_list_property_values(arch, property, scripts_path, properties_file, expand_common=True):
"""
List the possible values for a given input property. If
properties_file is non-null, write the complete set of properties
@@ -1614,7 +1599,7 @@ def yocto_bsp_list_property_values(arch, property, scripts_path, properties_file
context = create_context("unused", arch, scripts_path)
context["name"] = property

- target_files = expand_targets(context, "unused")
+ target_files = expand_targets(context, "unused", expand_common)

input_lines = gather_inputlines(target_files)

@@ -1697,13 +1682,39 @@ def yocto_bsp_list(args, scripts_path, properties_file):

if len(args) == 2:
if args[1] == "properties":
- yocto_bsp_list_properties(arch, scripts_path, properties_file)
+ yocto_layer_list_properties(arch, scripts_path, properties_file)
else:
return False

if len(args) == 3:
if args[1] == "property":
- yocto_bsp_list_property_values(arch, args[2], scripts_path, properties_file)
+ yocto_layer_list_property_values(arch, args[2], scripts_path, properties_file)
+ else:
+ return False
+
+ return True
+
+
+def yocto_layer_list(args, scripts_path, properties_file):
+ """
+ Print the complete list of input properties defined by the layer,
+ or the possible values for a particular layer property.
+ """
+ if len(args) < 1:
+ return False
+
+ if len(args) < 1 or len(args) > 2:
+ return False
+
+ if len(args) == 1:
+ if args[0] == "properties":
+ yocto_layer_list_properties("layer", scripts_path, properties_file, False)
+ else:
+ return False
+
+ if len(args) == 2:
+ if args[0] == "property":
+ yocto_layer_list_property_values("layer", args[1], scripts_path, properties_file, False)
else:
return False

--
1.7.11.4


[PATCH 0/7] new 'yocto-layer' tool for creating generic Yocto layers, v1

tom.zanussi@...
 

From: Tom Zanussi <tom.zanussi@...>

Since I've been doing kind of similar work lately for the 'custom kernel'
support for yocto-bsp and have gotten several requests lately (mainly
from Darren in support of the new kernel documentation) for something
like this, I decided to just go ahead and try to quickly implement a
general-purpose layer-generation tool based on the BSP-generation code
used in the yocto-bsp tool.

There's actually an enhancement request bug for this already in the
Yocto bugzilla, but it doesn't contain many details:

Bug 3094 - Add a layer generation tool:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=3094

v1 changes (the previous version was an RFC):

- if layer_priority is specified on the command line, the user is not
queried and the layer is generated immediately
- explicitly return 0 in the hello.c example code

Below are a couple examples of how it's used - you really have to try it
yourself to see what's generated - I don't want to post tarballs or such on
the list and it's simple to generate and look at the layers.

The first case is just a very simple layer with a layer.conf and README -
basically the simplest layer you can create, and which exists mainly because
even that is easy to get wrong. As with the yocto-bsp tool, the script
queries the user for a couple items, here we just take the defaults, which
are a priority of 6 for the layer and no other components such as example
recipes:

[trz@empanada build]$ yocto-layer create simplestlayer
Please enter the layer priority you'd like to use for the layer: [default: 6]
Would you like to have an example recipe created? (y/n) [default: n]
Would you like to have an example bbappend file created? (y/n) [default: n]

New layer created in meta-simplestlayer.

Don't forget to add it to your BBLAYERS (for details see meta-simplestlayer\README).

[trz@empanada build]$ find .
.
./meta-simplestlayer
./meta-simplestlayer/conf
./meta-simplestlayer/conf/layer.conf
./meta-simplestlayer/README
./meta-simplestlayer/COPYING.MIT

If you specify a layer_priority directly on the command line,
yocto-layer will not query the user at all but instead will generate
the layer (without example recipes etc) immediately:

[trz@empanada build]$ yocto-layer create simplestlayer 7

New layer created in meta-simplestlayer.

Don't forget to add it to your BBLAYERS (for details see meta-simplestlayer\README).

In the second case, we tell the tool that we do want an example .bb and and
an example .bbappend. We're queried for the recipe name that we want our
recipe to have, and for the .bbappend, the name of the base recipe and its
version. Below you can see the files it generates - please look at the files
themselves to see the contents. For the recipe example, it generates a recipe
based on the example in the Yocto manual, and my own helloworld.c code
(untested so far which is also why this is an RFC). For the .bbappend example,
it just creates an empty .patch file with some instructions on what to do to
modify the parent recipe with a patch:

[trz@empanada build]$ yocto-layer create mylayer
Please enter the layer priority you'd like to use for the layer: [default: 6]
Would you like to have an example recipe created? (y/n) [default: n] y
Please enter the name you'd like to use for your example recipe: [default: example] flork
Would you like to have an example bbappend file created? (y/n) [default: n] y
Please enter the name you'd like to use for your bbappend file: [default: example] chork
Please enter the version number you'd like to use for your bbappend file (this should match the recipe you're appending to): [default: 0.1] 0.22.3

New layer created in meta-mylayer.

Don't forget to add it to your BBLAYERS (for details see meta-mylayer\README).

[trz@empanada build]$ find .
.
./meta-mylayer
./meta-mylayer/recipes-example
./meta-mylayer/recipes-example/example
./meta-mylayer/recipes-example/example/flork-0.1
./meta-mylayer/recipes-example/example/flork-0.1/helloworld.c
./meta-mylayer/recipes-example/example/flork-0.1/example.patch
./meta-mylayer/recipes-example/example/flork_0.1.bb
./meta-mylayer/conf
./meta-mylayer/conf/layer.conf
./meta-mylayer/recipes-example-bbappend
./meta-mylayer/recipes-example-bbappend/example-bbappend
./meta-mylayer/recipes-example-bbappend/example-bbappend/chork-0.22.3
./meta-mylayer/recipes-example-bbappend/example-bbappend/chork-0.22.3/example.patch
./meta-mylayer/recipes-example-bbappend/example-bbappend/chork_0.22.3.bbappend
./meta-mylayer/README
./meta-mylayer/COPYING.MIT

Thanks,

Tom

The following changes since commit 53cc748b93e8af584557d6db5309c3e955182c5c:

linux-libc-headers: fix headers install in long path name environments (2013-01-10 23:53:51 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib.git tzanussi/yocto-layer-v1
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=tzanussi/yocto-layer-v1

Tom Zanussi (7):
scripts/lib/bsp/engine.py: add yocto_layer_create()
yocto-layer: new script
yocto-layer: add help/usage
yocto-layer: add 'layer' template data
scripts/lib/bsp/engine.py: refactor bsp-creation code
yocto-layer: add optional layer priority param
scripts/lib/bsp/engine.py: add handling for JSON strings

scripts/lib/bsp/engine.py | 114 ++++++++--
scripts/lib/bsp/help.py | 238 +++++++++++++++++++++
.../bsp/substrate/target/arch/layer/COPYING.MIT | 17 ++
scripts/lib/bsp/substrate/target/arch/layer/README | 64 ++++++
.../substrate/target/arch/layer/conf/layer.conf | 10 +
.../target/arch/layer/layer-questions.noinstall | 14 ++
.../example.patch" | 12 ++
..._name}}_{{=example_bbappend_version}}.bbappend" | 8 +
.../{{=example_recipe_name}}-0.1/example.patch" | 12 ++
.../{{=example_recipe_name}}-0.1/helloworld.c" | 8 +
.../example/{{=example_recipe_name}}_0.1.bb" | 23 ++
scripts/yocto-layer | 147 +++++++++++++
12 files changed, 645 insertions(+), 22 deletions(-)
create mode 100644 scripts/lib/bsp/substrate/target/arch/layer/COPYING.MIT
create mode 100644 scripts/lib/bsp/substrate/target/arch/layer/README
create mode 100644 scripts/lib/bsp/substrate/target/arch/layer/conf/layer.conf
create mode 100644 scripts/lib/bsp/substrate/target/arch/layer/layer-questions.noinstall
create mode 100644 "scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_bbappend == \"y\": }} recipes-example-bbappend/example-bbappend/{{=example_bbappend_name}}-{{=example_bbappend_version}}/example.patch"
create mode 100644 "scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_bbappend == \"y\": }} recipes-example-bbappend/example-bbappend/{{=example_bbappend_name}}_{{=example_bbappend_version}}.bbappend"
create mode 100644 "scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_recipe == \"y\": }} recipes-example/example/{{=example_recipe_name}}-0.1/example.patch"
create mode 100644 "scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_recipe == \"y\": }} recipes-example/example/{{=example_recipe_name}}-0.1/helloworld.c"
create mode 100644 "scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_recipe == \"y\": }} recipes-example/example/{{=example_recipe_name}}_0.1.bb"
create mode 100755 scripts/yocto-layer

--
1.7.11.4


[PATCH 4/7] yocto-layer: add 'layer' template data

tom.zanussi@...
 

From: Tom Zanussi <tom.zanussi@...>

Add a 'layer' target containing all the data that will be used to
generate a generic yocto layer.

Signed-off-by: Tom Zanussi <tom.zanussi@...>
---
.../bsp/substrate/target/arch/layer/COPYING.MIT | 17 ++++++
scripts/lib/bsp/substrate/target/arch/layer/README | 64 ++++++++++++++++++++++
.../substrate/target/arch/layer/conf/layer.conf | 10 ++++
.../target/arch/layer/layer-questions.noinstall | 14 +++++
.../example.patch" | 12 ++++
..._name}}_{{=example_bbappend_version}}.bbappend" | 8 +++
.../{{=example_recipe_name}}-0.1/example.patch" | 12 ++++
.../{{=example_recipe_name}}-0.1/helloworld.c" | 8 +++
.../example/{{=example_recipe_name}}_0.1.bb" | 23 ++++++++
9 files changed, 168 insertions(+)
create mode 100644 scripts/lib/bsp/substrate/target/arch/layer/COPYING.MIT
create mode 100644 scripts/lib/bsp/substrate/target/arch/layer/README
create mode 100644 scripts/lib/bsp/substrate/target/arch/layer/conf/layer.conf
create mode 100644 scripts/lib/bsp/substrate/target/arch/layer/layer-questions.noinstall
create mode 100644 "scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_bbappend == \"y\": }} recipes-example-bbappend/example-bbappend/{{=example_bbappend_name}}-{{=example_bbappend_version}}/example.patch"
create mode 100644 "scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_bbappend == \"y\": }} recipes-example-bbappend/example-bbappend/{{=example_bbappend_name}}_{{=example_bbappend_version}}.bbappend"
create mode 100644 "scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_recipe == \"y\": }} recipes-example/example/{{=example_recipe_name}}-0.1/example.patch"
create mode 100644 "scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_recipe == \"y\": }} recipes-example/example/{{=example_recipe_name}}-0.1/helloworld.c"
create mode 100644 "scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_recipe == \"y\": }} recipes-example/example/{{=example_recipe_name}}_0.1.bb"

diff --git a/scripts/lib/bsp/substrate/target/arch/layer/COPYING.MIT b/scripts/lib/bsp/substrate/target/arch/layer/COPYING.MIT
new file mode 100644
index 0000000..89de354
--- /dev/null
+++ b/scripts/lib/bsp/substrate/target/arch/layer/COPYING.MIT
@@ -0,0 +1,17 @@
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
diff --git a/scripts/lib/bsp/substrate/target/arch/layer/README b/scripts/lib/bsp/substrate/target/arch/layer/README
new file mode 100644
index 0000000..943dfc4
--- /dev/null
+++ b/scripts/lib/bsp/substrate/target/arch/layer/README
@@ -0,0 +1,64 @@
+This README file contains information on the contents of the
+{{=layer_name}} layer.
+
+Please see the corresponding sections below for details.
+
+
+Dependencies
+============
+
+This layer depends on:
+
+ URI: git://git.openembedded.org/bitbake
+ branch: master
+
+ URI: git://git.openembedded.org/openembedded-core
+ layers: meta
+ branch: master
+
+ URI: git://git.yoctoproject.org/xxxx
+ layers: xxxx
+ branch: master
+
+
+Patches
+=======
+
+Please submit any patches against the {{=layer_name}} layer to the
+xxxx mailing list (xxxx@...) and cc: the maintainer:
+
+Maintainer: XXX YYYYYY <xxx.yyyyyy@...>
+
+
+Table of Contents
+=================
+
+ I. Adding the {{=layer_name}} layer to your build
+ II. Misc
+
+
+I. Adding the {{=layer_name}} layer to your build
+=================================================
+
+--- replace with specific instructions for the {{=layer_name}} layer ---
+
+In order to use this layer, you need to make the build system aware of
+it.
+
+Assuming the {{=layer_name}} layer exists at the top-level of your
+yocto build tree, you can add it to the build system by adding the
+location of the {{=layer_name}} layer to bblayers.conf, along with any
+other layers needed. e.g.:
+
+ BBLAYERS ?= " \
+ /path/to/yocto/meta \
+ /path/to/yocto/meta-yocto \
+ /path/to/yocto/meta-yocto-bsp \
+ /path/to/yocto/meta-{{=layer_name}} \
+ "
+
+
+II. Misc
+========
+
+--- replace with specific information about the {{=layer_name}} layer ---
diff --git a/scripts/lib/bsp/substrate/target/arch/layer/conf/layer.conf b/scripts/lib/bsp/substrate/target/arch/layer/conf/layer.conf
new file mode 100644
index 0000000..84a9abb
--- /dev/null
+++ b/scripts/lib/bsp/substrate/target/arch/layer/conf/layer.conf
@@ -0,0 +1,10 @@
+# We have a conf and classes directory, add to BBPATH
+BBPATH := "${BBPATH}:${LAYERDIR}"
+
+# We have a recipes directory, add to BBFILES
+BBFILES := "${BBFILES} ${LAYERDIR}/recipes-*/*/*.bb \
+ ${LAYERDIR}/recipes-*/*/*.bbappend"
+
+BBFILE_COLLECTIONS += "{{=layer_name}}"
+BBFILE_PATTERN_{{=layer_name}} := "^${LAYERDIR}/"
+BBFILE_PRIORITY_{{=layer_name}} = "{{=layer_priority}}"
diff --git a/scripts/lib/bsp/substrate/target/arch/layer/layer-questions.noinstall b/scripts/lib/bsp/substrate/target/arch/layer/layer-questions.noinstall
new file mode 100644
index 0000000..e2a89c3
--- /dev/null
+++ b/scripts/lib/bsp/substrate/target/arch/layer/layer-questions.noinstall
@@ -0,0 +1,14 @@
+{{ input type:"edit" name:"layer_priority" prio:"20" msg:"Please enter the layer priority you'd like to use for the layer:" default:"6"}}
+
+{{ input type:"boolean" name:"create_example_recipe" prio:"20" msg:"Would you like to have an example recipe created? (y/n)" default:"n"}}
+
+{{ if create_example_recipe == "y": }}
+{{ input type:"edit" name:"example_recipe_name" prio:"20" msg:"Please enter the name you'd like to use for your example recipe:" default:"example"}}
+
+{{ input type:"boolean" name:"create_example_bbappend" prio:"20" msg:"Would you like to have an example bbappend file created? (y/n)" default:"n"}}
+
+{{ if create_example_bbappend == "y": }}
+{{ input type:"edit" name:"example_bbappend_name" prio:"20" msg:"Please enter the name you'd like to use for your bbappend file:" default:"example"}}
+
+{{ if create_example_bbappend == "y": }}
+{{ input type:"edit" name:"example_bbappend_version" prio:"20" msg:"Please enter the version number you'd like to use for your bbappend file (this should match the recipe you're appending to):" default:"0.1"}}
diff --git "a/scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_bbappend == \"y\": }} recipes-example-bbappend/example-bbappend/{{=example_bbappend_name}}-{{=example_bbappend_version}}/example.patch" "b/scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_bbappend == \"y\": }} recipes-example-bbappend/example-bbappend/{{=example_bbappend_name}}-{{=example_bbappend_version}}/example.patch"
new file mode 100644
index 0000000..2000a34
--- /dev/null
+++ "b/scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_bbappend == \"y\": }} recipes-example-bbappend/example-bbappend/{{=example_bbappend_name}}-{{=example_bbappend_version}}/example.patch"
@@ -0,0 +1,12 @@
+#
+# This is a non-functional placeholder file, here for example purposes
+# only.
+#
+# If you had a patch for your recipe, you'd put it in this directory
+# and reference it from your recipe's SRC_URI:
+#
+# SRC_URI += "file://example.patch"
+#
+# Note that you could also rename the directory containing this patch
+# to remove the version number or simply rename it 'files'. Doing so
+# allows you to use the same directory for multiple recipes.
diff --git "a/scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_bbappend == \"y\": }} recipes-example-bbappend/example-bbappend/{{=example_bbappend_name}}_{{=example_bbappend_version}}.bbappend" "b/scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_bbappend == \"y\": }} recipes-example-bbappend/example-bbappend/{{=example_bbappend_name}}_{{=example_bbappend_version}}.bbappend"
new file mode 100644
index 0000000..2e50ff6
--- /dev/null
+++ "b/scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_bbappend == \"y\": }} recipes-example-bbappend/example-bbappend/{{=example_bbappend_name}}_{{=example_bbappend_version}}.bbappend"
@@ -0,0 +1,8 @@
+FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"
+
+#
+# This .bbappend doesn't yet do anything - replace this text with
+# modifications to the example_0.1.bb recipe, or whatever recipe it is
+# that you want to modify with this .bbappend (make sure you change
+# the recipe name (PN) and version (PV) to match).
+#
diff --git "a/scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_recipe == \"y\": }} recipes-example/example/{{=example_recipe_name}}-0.1/example.patch" "b/scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_recipe == \"y\": }} recipes-example/example/{{=example_recipe_name}}-0.1/example.patch"
new file mode 100644
index 0000000..2000a34
--- /dev/null
+++ "b/scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_recipe == \"y\": }} recipes-example/example/{{=example_recipe_name}}-0.1/example.patch"
@@ -0,0 +1,12 @@
+#
+# This is a non-functional placeholder file, here for example purposes
+# only.
+#
+# If you had a patch for your recipe, you'd put it in this directory
+# and reference it from your recipe's SRC_URI:
+#
+# SRC_URI += "file://example.patch"
+#
+# Note that you could also rename the directory containing this patch
+# to remove the version number or simply rename it 'files'. Doing so
+# allows you to use the same directory for multiple recipes.
diff --git "a/scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_recipe == \"y\": }} recipes-example/example/{{=example_recipe_name}}-0.1/helloworld.c" "b/scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_recipe == \"y\": }} recipes-example/example/{{=example_recipe_name}}-0.1/helloworld.c"
new file mode 100644
index 0000000..71f2e46
--- /dev/null
+++ "b/scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_recipe == \"y\": }} recipes-example/example/{{=example_recipe_name}}-0.1/helloworld.c"
@@ -0,0 +1,8 @@
+#include <stdio.h>
+
+int main(int argc, char **argv)
+{
+ printf("Hello World!\n");
+
+ return 0;
+}
diff --git "a/scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_recipe == \"y\": }} recipes-example/example/{{=example_recipe_name}}_0.1.bb" "b/scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_recipe == \"y\": }} recipes-example/example/{{=example_recipe_name}}_0.1.bb"
new file mode 100644
index 0000000..14bf344
--- /dev/null
+++ "b/scripts/lib/bsp/substrate/target/arch/layer/{{ if create_example_recipe == \"y\": }} recipes-example/example/{{=example_recipe_name}}_0.1.bb"
@@ -0,0 +1,23 @@
+#
+# This file was derived from the 'Hello World!' example recipe in the
+# Yocto Project Development Manual.
+#
+
+DESCRIPTION = "Simple helloworld application"
+SECTION = "examples"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
+PR = "r0"
+
+SRC_URI = "file://helloworld.c"
+
+S = "${WORKDIR}"
+
+do_compile() {
+ ${CC} helloworld.c -o helloworld
+}
+
+do_install() {
+ install -d ${D}${bindir}
+ install -m 0755 helloworld ${D}${bindir}
+}
--
1.7.11.4


[PATCH 3/7] yocto-layer: add help/usage

tom.zanussi@...
 

From: Tom Zanussi <tom.zanussi@...>

This is essentially 'the documentation' for the yocto-layer tool.

Signed-off-by: Tom Zanussi <tom.zanussi@...>
---
scripts/lib/bsp/help.py | 228 ++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 228 insertions(+)

diff --git a/scripts/lib/bsp/help.py b/scripts/lib/bsp/help.py
index 3a1f52c..eac172a 100644
--- a/scripts/lib/bsp/help.py
+++ b/scripts/lib/bsp/help.py
@@ -595,6 +595,234 @@ DESCRIPTION
"""

##
+# yocto-layer help and usage strings
+##
+
+yocto_layer_usage = """
+
+ Create a generic Yocto layer.
+
+ usage: yocto-layer [--version] [--help] COMMAND [ARGS]
+
+ Current 'yocto-layer' commands are:
+ create Create a new generic Yocto layer
+ list List available values for input options and properties
+
+ See 'yocto-layer help COMMAND' for more information on a specific command.
+"""
+
+yocto_layer_help_usage = """
+
+ usage: yocto-layer help <subcommand>
+
+ This command displays detailed help for the specified subcommand.
+"""
+
+yocto_layer_create_usage = """
+
+ Create a new generic Yocto layer
+
+ usage: yocto-layer create <layer-name> [-o <DIRNAME> | --outdir <DIRNAME>]
+ [-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]
+
+ This command creates a generic Yocto layer based on the specified
+ parameters. The new layer will be a new Yocto layer contained by
+ default within the top-level directory specified as
+ 'meta-layer-name'. The -o option can be used to place the layer in a
+ directory with a different name and location.
+
+ NOTE: Once created, you should add your new layer to your
+ bblayers.conf file in order for it to be subsequently seen and
+ modified by the yocto-kernel tool. Instructions for doing this can
+ be found in the README file generated in the layer's top-level
+ directory.
+
+ See 'yocto layer help create' for more detailed instructions.
+"""
+
+yocto_layer_create_help = """
+
+NAME
+ yocto-layer create - Create a new generic Yocto layer
+
+SYNOPSIS
+ yocto-layer create <layer-name> [-o <DIRNAME> | --outdir <DIRNAME>]
+ [-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]
+
+DESCRIPTION
+ This command creates a generic Yocto layer based on the specified
+ parameters. The new layer will be a new Yocto layer contained by
+ default within the top-level directory specified as
+ 'meta-layer-name'. The -o option can be used to place the layer
+ in a directory with a different name and location.
+
+ The layer-specific properties that define the values that will be
+ used to generate the layer can be specified on the command-line
+ using the -i option and supplying a JSON object consisting of the
+ set of name:value pairs needed by the layer.
+
+ If the -i option is not used, the user will be interactively
+ prompted for each of the required property values, which will then
+ be used as values for layer generation.
+
+ The set of properties available can be listed using the
+ 'yocto-layer list' command.
+
+ Specifying -c causes the Python code generated and executed to
+ create the layer to be dumped to the 'bspgen.out' file in the
+ current directory, and is useful for debugging.
+
+ NOTE: Once created, you should add your new layer to your
+ bblayers.conf file in order for it to be subsequently seen and
+ modified by the yocto-kernel tool. Instructions for doing this
+ can be found in the README file generated in the layer's top-level
+ directory.
+
+ For example, assuming your poky repo is at /path/to/poky, your new
+ layer is at /path/to/poky/meta-mylayer, and your build directory
+ is /path/to/build:
+
+ $ gedit /path/to/build/conf/bblayers.conf
+
+ BBLAYERS ?= " \\
+ /path/to/poky/meta \\
+ /path/to/poky/meta-yocto \\
+ /path/to/poky/meta-mylayer \\
+ "
+"""
+
+yocto_layer_list_usage = """
+
+ usage: yocto-layer list properties
+ [-o <JSON PROPERTY FILE> | --outfile <JSON PROPERTY_FILE>]
+ yocto-layer list property <xxx>
+ [-o <JSON PROPERTY FILE> | --outfile <JSON PROPERTY_FILE>]
+
+ This command enumerates the complete set of possible values for a
+ specified option or property needed by the layer creation process.
+
+ The first form enumerates all the possible properties that exist and
+ must have values specified for them in the 'yocto-layer create'
+ command.
+
+ The second form enumerates all the possible values that exist and can
+ be specified for any of the enumerable properties in the 'yocto-layer
+ create' command.
+
+ See 'yocto-layer help list' for more details.
+"""
+
+yocto_layer_list_help = """
+
+NAME
+ yocto-layer list - List available values for layer input options and properties
+
+SYNOPSIS
+ yocto-layer list properties
+ [--o <JSON PROPERTY FILE> | -outfile <JSON PROPERTY_FILE>]
+ yocto-layer list property <xxx>
+ [--o <JSON PROPERTY FILE> | -outfile <JSON PROPERTY_FILE>]
+
+DESCRIPTION
+ This command enumerates the complete set of possible values for a
+ specified option or property needed by the layer creation process.
+
+ The first form enumerates all the possible properties that exist
+ and must have values specified for them in the 'yocto-layer
+ create' command. This command is mainly meant to aid the
+ development of user interface alternatives to the default
+ text-based prompting interface. If the -o option is specified,
+ the list of properties, in addition to being displayed, will be
+ written to the specified file as a JSON object. In this case, the
+ object will consist of the set of name:value pairs corresponding
+ to the (possibly nested) dictionary of properties defined by the
+ input statements used by the BSP. Some example output for the
+ 'list properties' command:
+
+ $ yocto-layer list properties
+ "example_bbappend_name" : {
+ "default" : example
+ "msg" : Please enter the name you'd like to use for your bbappend file:
+ "type" : edit
+ "prio" : 20
+ "filename" : /home/trz/yocto/yocto-layer-dev/scripts/lib/bsp/substrate/target/arch/layer/layer-questions.noinstall
+ }
+ "create_example_recipe" : {
+ "default" : n
+ "msg" : Would you like to have an example recipe created? (y/n)
+ "type" : boolean
+ "prio" : 20
+ "filename" : /home/trz/yocto/yocto-layer-dev/scripts/lib/bsp/substrate/target/arch/layer/layer-questions.noinstall
+ }
+ "example_recipe_name" : {
+ "default" : example
+ "msg" : Please enter the name you'd like to use for your example recipe:
+ "type" : edit
+ "prio" : 20
+ "filename" : /home/trz/yocto/yocto-layer-dev/scripts/lib/bsp/substrate/target/arch/layer/layer-questions.noinstall
+ }
+ "layer_priority" : {
+ "default" : 6
+ "msg" : Please enter the layer priority you'd like to use for the layer:
+ "type" : edit
+ "prio" : 20
+ "filename" : /home/trz/yocto/yocto-layer-dev/scripts/lib/bsp/substrate/target/arch/layer/layer-questions.noinstall
+ }
+ "create_example_bbappend" : {
+ "default" : n
+ "msg" : Would you like to have an example bbappend file created? (y/n)
+ "type" : boolean
+ "prio" : 20
+ "filename" : /home/trz/yocto/yocto-layer-dev/scripts/lib/bsp/substrate/target/arch/layer/layer-questions.noinstall
+ }
+ "example_bbappend_version" : {
+ "default" : 0.1
+ "msg" : Please enter the version number you'd like to use for your bbappend file (this should match the recipe you're appending to):
+ "type" : edit
+ "prio" : 20
+ "filename" : /home/trz/yocto/yocto-layer-dev/scripts/lib/bsp/substrate/target/arch/layer/layer-questions.noinstall
+ }
+
+ Each entry in the output consists of the name of the input element
+ e.g. "layer_priority", followed by the properties defined for that
+ element enclosed in braces. This information should provide
+ sufficient information to create a complete user interface. Two
+ features of the scheme provide for conditional input. First, if a
+ Python "if" statement appears in place of an input element name,
+ the set of enclosed input elements apply and should be presented
+ to the user only if the 'if' statement evaluates to true. The
+ test in the if statement will always reference another input
+ element in the list, which means that the element being tested
+ should be presented to the user before the elements enclosed by
+ the if block. Secondly, in a similar way, some elements contain
+ "depends-on" and depends-on-val" tags, which mean that the
+ affected input element should only be presented to the user if the
+ element it depends on has already been presented to the user and
+ the user has selected the specified value for that element.
+
+ The second form enumerates all the possible values that exist and
+ can be specified for any of the enumerable properties in the
+ 'yocto-layer create' command. If the -o option is specified, the
+ list of values for the given property, in addition to being
+ displayed, will be written to the specified file as a JSON object.
+ In this case, the object will consist of the set of name:value
+ pairs corresponding to the array of property values associated
+ with the property.
+
+ $ yocto-layer list property layer_priority
+ [no output - layer_priority is a text field that has no enumerable values]
+
+ The second form as well is meant mainly for developers of
+ alternative interfaces - it allows the developer to fetch the
+ possible values for a given input element on-demand. This
+ on-demand capability is especially valuable for elements that
+ require relatively expensive remote operations to fulfill, such as
+ the example that returns the set of branches available in a remote
+ git tree above.
+
+"""
+
+##
# test code
##

--
1.7.11.4


[PATCH 2/7] yocto-layer: new script

tom.zanussi@...
 

From: Tom Zanussi <tom.zanussi@...>

Implementation of the 'yocto-layer' command-line tool, for creating
generic layers and listing their input properties.

Signed-off-by: Tom Zanussi <tom.zanussi@...>
---
scripts/yocto-layer | 142 ++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 142 insertions(+)
create mode 100755 scripts/yocto-layer

diff --git a/scripts/yocto-layer b/scripts/yocto-layer
new file mode 100755
index 0000000..f759275
--- /dev/null
+++ b/scripts/yocto-layer
@@ -0,0 +1,142 @@
+#!/usr/bin/env python
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+#
+# Copyright (c) 2012, Intel Corporation.
+# All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# DESCRIPTION
+# 'yocto-layer' is the Yocto Tool that helps users create a new Yocto
+# layer. Invoking it without any arguments will display help screens
+# for the 'yocto-layer' command and list the available 'yocto-layer'
+# subcommands. Invoking a subcommand without any arguments will
+# likewise display help screens for the specified subcommand. Please
+# use that interface for detailed help.
+#
+# AUTHORS
+# Tom Zanussi <tom.zanussi (at] intel.com>
+#
+
+__version__ = "0.1.0"
+
+import os
+import sys
+import optparse
+import logging
+
+scripts_path = os.path.abspath(os.path.dirname(os.path.abspath(sys.argv[0])))
+lib_path = scripts_path + '/lib'
+sys.path = sys.path + [lib_path]
+
+from bsp.help import *
+from bsp.engine import *
+
+
+def yocto_layer_create_subcommand(args, usage_str):
+ """
+ Command-line handling for layer creation. The real work is done by
+ bsp.engine.yocto_layer_create()
+ """
+ parser = optparse.OptionParser(usage = usage_str)
+
+ parser.add_option("-o", "--outdir", dest = "outdir", action = "store",
+ help = "name of layer dir to create")
+ parser.add_option("-i", "--infile", dest = "properties_file", action = "store",
+ help = "name of file containing the values for layer input properties as a JSON file")
+ parser.add_option("-c", "--codedump", dest = "codedump", action = "store_true",
+ default = False, help = "dump the generated code to layergen.out")
+ (options, args) = parser.parse_args(args)
+
+ if len(args) != 1:
+ logging.error("Wrong number of arguments, exiting\n")
+ parser.print_help()
+ sys.exit(1)
+
+ layer_name = args[0]
+
+ if options.outdir:
+ layer_output_dir = options.outdir
+ else:
+ layer_output_dir = "meta-" + layer_name
+
+ yocto_layer_create(layer_name, scripts_path, layer_output_dir, options.codedump, options.properties_file)
+
+
+def yocto_layer_list_subcommand(args, usage_str):
+ """
+ Command-line handling for listing available layer properties and
+ values. The real work is done by bsp.engine.yocto_layer_list()
+ """
+ parser = optparse.OptionParser(usage = usage_str)
+
+ parser.add_option("-o", "--outfile", action = "store", dest = "properties_file",
+ help = "dump the possible values for layer properties to a JSON file")
+
+ (options, args) = parser.parse_args(args)
+
+ if not yocto_layer_list(args, scripts_path, options.properties_file):
+ logging.error("Bad list arguments, exiting\n")
+ parser.print_help()
+ sys.exit(1)
+
+
+subcommands = {
+ "create": [yocto_layer_create_subcommand,
+ yocto_layer_create_usage,
+ yocto_layer_create_help],
+ "list": [yocto_layer_list_subcommand,
+ yocto_layer_list_usage,
+ yocto_layer_list_help],
+}
+
+
+def start_logging(loglevel):
+ logging.basicConfig(filname = 'yocto-layer.log', filemode = 'w', level=loglevel)
+
+
+def main():
+ parser = optparse.OptionParser(version = "yocto-layer version %s" % __version__,
+ usage = yocto_layer_usage)
+
+ parser.disable_interspersed_args()
+ parser.add_option("-D", "--debug", dest = "debug", action = "store_true",
+ default = False, help = "output debug information")
+
+ (options, args) = parser.parse_args()
+
+ loglevel = logging.INFO
+ if options.debug:
+ loglevel = logging.DEBUG
+ start_logging(loglevel)
+
+ if len(args):
+ if args[0] == "help":
+ if len(args) == 1:
+ parser.print_help()
+ sys.exit(1)
+
+ invoke_subcommand(args, parser, yocto_layer_help_usage, subcommands)
+
+
+if __name__ == "__main__":
+ try:
+ ret = main()
+ except Exception:
+ ret = 1
+ import traceback
+ traceback.print_exc(5)
+ sys.exit(ret)
+
--
1.7.11.4


[PATCH 1/7] scripts/lib/bsp/engine.py: add yocto_layer_create()

tom.zanussi@...
 

From: Tom Zanussi <tom.zanussi@...>

Add a new yocto_layer_create() function that will be used to generate
a generic yocto layer (for the new 'yocto-layer' command).

Signed-off-by: Tom Zanussi <tom.zanussi@...>
---
scripts/lib/bsp/engine.py | 56 +++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 56 insertions(+)

diff --git a/scripts/lib/bsp/engine.py b/scripts/lib/bsp/engine.py
index 8985544..d406e79 100644
--- a/scripts/lib/bsp/engine.py
+++ b/scripts/lib/bsp/engine.py
@@ -1352,6 +1352,62 @@ def expand_targets(context, bsp_output_dir):
return target_files


+def yocto_layer_create(layer_name, scripts_path, layer_output_dir, codedump, properties_file):
+ """
+ Create yocto layer
+
+ layer_name - user-defined layer name
+ scripts_path - absolute path to yocto /scripts dir
+ bsp_output_dir - dirname to create for BSP
+ codedump - dump generated code to bspgen.out
+ properties_file - use values from here if nonempty i.e no prompting
+
+ arch - the arch for a generic layer is 'generic-layer', defined in
+ scripts/lib/bsp/substrate/target/generic-layer
+ """
+ if os.path.exists(bsp_output_dir):
+ print "\nBSP output dir already exists, exiting. (%s)" % bsp_output_dir
+ sys.exit(1)
+
+ properties = None
+
+ if properties_file:
+ try:
+ infile = open(properties_file, "r")
+ except IOError:
+ print "Couldn't open properties file %s for reading, exiting" % properties_file
+ sys.exit(1)
+
+ properties = json.load(infile)
+
+ os.mkdir(bsp_output_dir)
+
+ context = create_context(machine, arch, scripts_path)
+ target_files = expand_targets(context, bsp_output_dir)
+
+ input_lines = gather_inputlines(target_files)
+
+ program_lines = []
+
+ gen_program_header_lines(program_lines)
+
+ gen_initial_property_vals(input_lines, program_lines)
+
+ if properties:
+ gen_supplied_property_vals(properties, program_lines)
+
+ gen_program_machine_lines(machine, program_lines)
+
+ if not properties:
+ gen_program_input_lines(input_lines, program_lines, context)
+
+ gen_program_lines(target_files, program_lines)
+
+ run_program_lines(program_lines, codedump)
+
+ print "New %s BSP created in %s" % (arch, bsp_output_dir)
+
+
def yocto_bsp_create(machine, arch, scripts_path, bsp_output_dir, codedump, properties_file):
"""
Create bsp
--
1.7.11.4


[PATCH] [Temp-Performance-Patch] Run each command in separate shell and close it accordingly

Ioana Grigoropol <ioanax.grigoropol@...>
 

--needs more testing

Signed-off-by: Ioana Grigoropol <ioanax.grigoropol@...>
---
plugins/org.yocto.bc.ui/META-INF/MANIFEST.MF | 1 +
.../src/org/yocto/bc/bitbake/ShellSession.java | 10 +-
.../org/yocto/bc/remote/utils/RemoteHelper.java | 289 ++++++++++++++++----
.../bc/remote/utils/YoctoRunnableWithProgress.java | 7 +-
.../bc/ui/wizards/NewBitBakeFileRecipeWizard.java | 3 +-
.../ui/wizards/NewBitBakeFileRecipeWizardPage.java | 78 +++---
6 files changed, 295 insertions(+), 93 deletions(-)

diff --git a/plugins/org.yocto.bc.ui/META-INF/MANIFEST.MF b/plugins/org.yocto.bc.ui/META-INF/MANIFEST.MF
index 1f0e63e..4e57f33 100644
--- a/plugins/org.yocto.bc.ui/META-INF/MANIFEST.MF
+++ b/plugins/org.yocto.bc.ui/META-INF/MANIFEST.MF
@@ -29,6 +29,7 @@ Import-Package: org.eclipse.cdt.managedbuilder.core,
org.eclipse.rse.core,
org.eclipse.rse.core.model,
org.eclipse.rse.core.subsystems,
+ org.eclipse.rse.internal.services.local.shells,
org.eclipse.rse.services,
org.eclipse.rse.services.clientserver.messages,
org.eclipse.rse.services.files,
diff --git a/plugins/org.yocto.bc.ui/src/org/yocto/bc/bitbake/ShellSession.java b/plugins/org.yocto.bc.ui/src/org/yocto/bc/bitbake/ShellSession.java
index 6441029..449994c 100644
--- a/plugins/org.yocto.bc.ui/src/org/yocto/bc/bitbake/ShellSession.java
+++ b/plugins/org.yocto.bc.ui/src/org/yocto/bc/bitbake/ShellSession.java
@@ -87,8 +87,11 @@ public class ShellSession {
private void initializeShell(IProgressMonitor monitor) throws IOException {
try {
if (root != null) {
- RemoteHelper.runCommandRemote(projectInfo.getConnection(), new YoctoCommand("source " + initCmd, root.getAbsolutePath(), ""));
- RemoteHelper.runCommandRemote(projectInfo.getConnection(), new YoctoCommand(exportCmd, root.getAbsolutePath(), ""));
+ IHost connection = projectInfo.getConnection();
+// RemoteHelper.runCommandRemote(projectInfo.getConnection(), new YoctoCommand("source " + initCmd, root.getAbsolutePath(), ""));
+ RemoteHelper.handleRunCommandRemote(connection, new YoctoCommand("source " + initCmd, root.getAbsolutePath(), ""), monitor);
+// RemoteHelper.runCommandRemote(projectInfo.getConnection(), new YoctoCommand(exportCmd, root.getAbsolutePath(), ""));
+ RemoteHelper.handleRunCommandRemote(connection, new YoctoCommand(exportCmd, root.getAbsolutePath(), ""), monitor);
} else {
throw new Exception("Root file not found!");
}
@@ -107,7 +110,8 @@ public class ShellSession {

try {
if (projectInfo.getConnection() != null) {
- hasErrors = RemoteHelper.runCommandRemote(projectInfo.getConnection(), new YoctoCommand(command, root.getAbsolutePath() + "/build/", ""));
+// hasErrors = RemoteHelper.runCommandRemote(projectInfo.getConnection(), new YoctoCommand(command, root.getAbsolutePath() + "/build/", ""));
+ RemoteHelper.handleRunCommandRemote(projectInfo.getConnection(), new YoctoCommand(command, root.getAbsolutePath() + "/build/", ""), new NullProgressMonitor());
// return RemoteHelper.getProcessBuffer(projectInfo.getConnection()).getMergedOutputLines();
return root.getAbsolutePath() + "/build/";
}
diff --git a/plugins/org.yocto.bc.ui/src/org/yocto/bc/remote/utils/RemoteHelper.java b/plugins/org.yocto.bc.ui/src/org/yocto/bc/remote/utils/RemoteHelper.java
index 814e3a5..6b8ebfc 100644
--- a/plugins/org.yocto.bc.ui/src/org/yocto/bc/remote/utils/RemoteHelper.java
+++ b/plugins/org.yocto.bc.ui/src/org/yocto/bc/remote/utils/RemoteHelper.java
@@ -10,9 +10,12 @@
********************************************************************************/
package org.yocto.bc.remote.utils;

+import java.io.BufferedReader;
import java.io.File;
+import java.io.InputStreamReader;
import java.net.URI;
import java.net.URISyntaxException;
+import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
@@ -25,16 +28,20 @@ import org.eclipse.core.runtime.OperationCanceledException;
import org.eclipse.core.runtime.Path;
import org.eclipse.core.runtime.Status;
import org.eclipse.core.runtime.SubProgressMonitor;
+import org.eclipse.osgi.util.NLS;
import org.eclipse.ptp.remote.core.IRemoteConnection;
import org.eclipse.rse.core.RSECorePlugin;
import org.eclipse.rse.core.model.IHost;
import org.eclipse.rse.core.model.ISystemRegistry;
import org.eclipse.rse.core.subsystems.ISubSystem;
+import org.eclipse.rse.internal.services.local.shells.LocalShellService;
import org.eclipse.rse.services.IService;
import org.eclipse.rse.services.clientserver.messages.SystemMessageException;
import org.eclipse.rse.services.files.IFileService;
import org.eclipse.rse.services.files.IHostFile;
+import org.eclipse.rse.services.shells.HostShellProcessAdapter;
import org.eclipse.rse.services.shells.IHostShell;
+import org.eclipse.rse.services.shells.IShellService;
import org.eclipse.rse.subsystems.files.core.model.RemoteFileUtility;
import org.eclipse.rse.subsystems.files.core.servicesubsystem.FileServiceSubSystem;
import org.eclipse.rse.subsystems.files.core.servicesubsystem.IFileServiceSubSystem;
@@ -49,6 +56,78 @@ public class RemoteHelper {
public static final int TOTALWORKLOAD = 100;
private static Map<IHost, RemoteMachine> machines;

+ public static Map<String, String> environment;
+ public static final String PROXY = "proxy";
+
+ public static ISubSystem getCmdSubsystem(IHost host) {
+ if (host == null)
+ return null;
+ ISubSystem[] subSystems = host.getSubSystems();
+ for (int i = 0; i < subSystems.length; i++) {
+ if (subSystems[i] instanceof IRemoteCmdSubSystem)
+ return subSystems[i];
+ }
+ return null;
+ }
+ public static ISubSystem getConnectedCmdService(
+ IHost currentConnection, IProgressMonitor monitor) throws Exception {
+ final ISubSystem subsystem = getCmdSubsystem(currentConnection);
+
+ if (subsystem == null)
+ throw new Exception(Messages.ErrorNoSubsystem);
+
+ try {
+ subsystem.connect(monitor, false);
+ } catch (CoreException e) {
+ throw e;
+ } catch (OperationCanceledException e) {
+ throw new CoreException(Status.CANCEL_STATUS);
+ }
+
+ if (!subsystem.isConnected())
+ throw new Exception(Messages.ErrorConnectSubsystem);
+
+
+ return (subsystem);
+ }
+ public static void getRemoteEnvProxyVars(IHost connection, String initialDir, IProgressMonitor monitor){
+ try {
+ if (environment != null && !environment.isEmpty())
+ return;
+
+ environment = new HashMap<String, String>();
+
+ IShellService shellService = (IShellService) getConnectedShellService(connection, new SubProgressMonitor(monitor, 7));
+
+
+ HostShellProcessAdapter p = null;
+ ProcessStreamBuffer buffer = null;
+ try {
+ SubProgressMonitor subMonitor = new SubProgressMonitor(monitor, 3);
+ IHostShell hostShell = shellService.runCommand(initialDir, "env" + " ; echo " + TERMINATOR + "; exit;", new String[]{}, subMonitor);
+ p = new HostShellProcessAdapter(hostShell);
+ buffer = processOutput(p, subMonitor);
+ for(int i = 0; i < buffer.getOutputLines().size(); i++) {
+ String out = buffer.getOutputLines().get(i);
+ String[] tokens = out.split("=");
+ if (tokens.length != 2)
+ continue;
+ String varName = tokens[0];
+ String varValue = tokens[1];
+ if (varName.contains(PROXY))
+ environment.put(varName, varValue);
+ }
+ } catch (Exception e) {
+ if (p != null) {
+ p.destroy();
+ }
+ e.printStackTrace();
+ }
+
+ } catch (Exception e) {
+ e.printStackTrace();
+ }
+ }
public static RemoteMachine getRemoteMachine(IHost connection){
if (!getMachines().containsKey(connection))
getMachines().put(connection, new RemoteMachine(connection));
@@ -198,15 +277,115 @@ public class RemoteHelper {
return getRemoteMachine(connection).getShellService(monitor);
}

- public static ISubSystem getCmdSubsystem(IHost host) {
- if (host == null)
- return null;
- ISubSystem[] subSystems = host.getSubSystems();
- for (int i = 0; i < subSystems.length; i++) {
- if (subSystems[i] instanceof IRemoteCmdSubSystem)
- return subSystems[i];
+ public static void handleRunCommandRemote(IHost connection, YoctoCommand cmd, IProgressMonitor monitor){
+ try {
+ Process process = testRunCommandRemote(connection, cmd, monitor);
+ cmd.setProcessBuffer(processOutput(process, monitor));
+ } catch (Exception e) {
+ e.printStackTrace();
}
- return null;
+ }
+ private static ProcessStreamBuffer processOutput(Process process, IProgressMonitor monitor) throws Exception {
+ if (process == null)
+ throw new Exception("An error has occured while trying to run remote command!");
+ ProcessStreamBuffer processBuffer = new ProcessStreamBuffer();
+
+ BufferedReader inbr = new BufferedReader(new InputStreamReader(process.getInputStream()));
+ BufferedReader errbr = new BufferedReader(new InputStreamReader(process.getErrorStream()));
+ boolean cancel = false;
+ while (!cancel) {
+ if(monitor.isCanceled()) {
+ cancel = true;
+ throw new InterruptedException("User Cancelled");
+ }
+ StringBuffer buffer = new StringBuffer();
+ int c;
+ while ((c = errbr.read()) != -1) {
+ char ch = (char) c;
+ buffer.append(ch);
+ if (ch == '\n'){
+ String str = buffer.toString();
+ processBuffer.addErrorLine(str);
+ System.out.println(str);
+ if (str.trim().equals(TERMINATOR)) {
+ break;
+ }
+ buffer.delete(0, buffer.length());
+ }
+ }
+
+ while ((c = inbr.read()) != -1) {
+ char ch = (char) c;
+ buffer.append(ch);
+ if (ch == '\n'){
+ String str = buffer.toString();
+ processBuffer.addOutputLine(str);
+ System.out.println(str);
+ if (str.trim().equals(TERMINATOR)) {
+ break;
+ }
+ buffer.delete(0, buffer.length());
+ }
+ }
+ cancel = true;
+ }
+ return processBuffer;
+ }
+
+ public static String[] prepareEnvString(IHost connection, String initialDir, IProgressMonitor monitor){
+ IShellService shellService;
+ String[] env = null;
+ try {
+ shellService = (IShellService) getConnectedShellService(connection, new SubProgressMonitor(monitor, 7));
+ if (shellService instanceof LocalShellService) {
+ env = shellService.getHostEnvironment();
+ } else {
+ List<String> envList = new ArrayList<String>();
+ getRemoteEnvProxyVars(connection, initialDir, monitor);
+ String value = "";
+ for (String varName : environment.keySet()){
+ value = varName + "=" + environment.get(varName);
+ envList.add(value);
+ }
+ env = envList.toArray(new String[envList.size()]);
+ }
+ } catch (Exception e) {
+ // TODO Auto-generated catch block
+ e.printStackTrace();
+ }
+
+ return env;
+ }
+ public static Process testRunCommandRemote(IHost connection, YoctoCommand cmd,
+ IProgressMonitor monitor) throws CoreException {
+
+ monitor.beginTask(NLS.bind(Messages.RemoteShellExec_1,
+ cmd, cmd.getArguments()), 10);
+
+ String remoteCommand = cmd + " " + cmd.getArguments() + " ; echo " + TERMINATOR + "; exit ;";
+
+ IShellService shellService;
+ Process p = null;
+// boolean cancel = false;
+ try {
+ shellService = (IShellService) getConnectedShellService(connection, new SubProgressMonitor(monitor, 7));
+
+ String env[] = prepareEnvString(connection, cmd.getInitialDirectory(), monitor);
+
+ try {
+ IHostShell hostShell = shellService.runCommand(cmd.getInitialDirectory(), remoteCommand, env, new SubProgressMonitor(monitor, 3));
+ p = new HostShellProcessAdapter(hostShell);
+
+ } catch (Exception e) {
+ if (p != null) {
+ p.destroy();
+ }
+ e.printStackTrace();
+ }
+ } catch (Exception e1) {
+ e1.printStackTrace();
+ }
+ return p;
}

public static void getRemoteFile(IHost connection, String localExePath, String remoteExePath,
@@ -248,53 +427,53 @@ public class RemoteHelper {
return null;
}

- public static boolean runCommandRemote(final IHost connection, final YoctoCommand cmd) throws Exception {
- final String remoteCommand = cmd.getCommand() + " " + cmd.getArguments();
- final boolean hasErrors = false;
-
- if (!cmd.getInitialDirectory().isEmpty()) {
- writeToShell(connection, "cd " + cmd.getInitialDirectory());
- }
- if (!hasErrors)
- writeToShell(connection, remoteCommand);
-
- return hasErrors;
- }
-
- public static boolean writeToShell(final IHost connection, final String remoteCommand){
- new Thread(new Runnable() {
- @Override
- public void run() {
- try {
- YoctoHostShellProcessAdapter adapter = getHostShellProcessAdapter(connection);
- String fullRemoteCommand = remoteCommand + "; echo " + TERMINATOR + ";";
- adapter.setLastCommand(fullRemoteCommand);
- getHostShell(connection).writeToShell(fullRemoteCommand);
- while (!adapter.isFinished())
- Thread.sleep(2);
- } catch (Exception e) {
- e.printStackTrace();
- }
- }
- }).run();
- return true;
- }
-
- public static void runBatchRemote(IHost connection, List<YoctoCommand> cmds, boolean displayOutput) throws CoreException {
- try {
- String remoteCommand = "";
- for (YoctoCommand cmd : cmds) {
- remoteCommand = cmd.getCommand() + " " + cmd.getArguments();
- if (!cmd.getInitialDirectory().isEmpty()) {
- writeToShell(connection, "cd " + cmd.getInitialDirectory());
- }
- writeToShell(connection, remoteCommand);
- }
-
- } catch (Exception e1) {
- e1.printStackTrace();
- }
- }
+// public static boolean runCommandRemote(final IHost connection, final YoctoCommand cmd) throws Exception {
+// final String remoteCommand = cmd.getCommand() + " " + cmd.getArguments();
+// final boolean hasErrors = false;
+//
+// if (!cmd.getInitialDirectory().isEmpty()) {
+// writeToShell(connection, "cd " + cmd.getInitialDirectory());
+// }
+// if (!hasErrors)
+// writeToShell(connection, remoteCommand);
+//
+// return hasErrors;
+// }
+//
+// public static boolean writeToShell(final IHost connection, final String remoteCommand){
+// new Thread(new Runnable() {
+// @Override
+// public void run() {
+// try {
+// YoctoHostShellProcessAdapter adapter = getHostShellProcessAdapter(connection);
+// String fullRemoteCommand = remoteCommand + "; echo " + TERMINATOR + "; exit ;";
+// adapter.setLastCommand(fullRemoteCommand);
+// getHostShell(connection).writeToShell(fullRemoteCommand);
+// while (!adapter.isFinished())
+// Thread.sleep(2);
+// } catch (Exception e) {
+// e.printStackTrace();
+// }
+// }
+// }).run();
+// return true;
+// }
+
+// public static void runBatchRemote(IHost connection, List<YoctoCommand> cmds, boolean displayOutput) throws CoreException {
+// try {
+// String remoteCommand = "";
+// for (YoctoCommand cmd : cmds) {
+// remoteCommand = cmd.getCommand() + " " + cmd.getArguments();
+// if (!cmd.getInitialDirectory().isEmpty()) {
+// writeToShell(connection, "cd " + cmd.getInitialDirectory());
+// }
+// writeToShell(connection, remoteCommand);
+// }
+//
+// } catch (Exception e1) {
+// e1.printStackTrace();
+// }
+// }

/**
* Throws a core exception with an error status object built from the given
diff --git a/plugins/org.yocto.bc.ui/src/org/yocto/bc/remote/utils/YoctoRunnableWithProgress.java b/plugins/org.yocto.bc.ui/src/org/yocto/bc/remote/utils/YoctoRunnableWithProgress.java
index b6ed2b8..82ba0ba 100644
--- a/plugins/org.yocto.bc.ui/src/org/yocto/bc/remote/utils/YoctoRunnableWithProgress.java
+++ b/plugins/org.yocto.bc.ui/src/org/yocto/bc/remote/utils/YoctoRunnableWithProgress.java
@@ -20,7 +20,7 @@ public class YoctoRunnableWithProgress extends YoctoHostShellProcessAdapter
private String cmd;
private String args;
private IProgressMonitor monitor;
-
+
public YoctoRunnableWithProgress(IHostShell hostShell,
ProcessStreamBuffer processStreamBuffer,
CommandResponseHandler commandResponseHandler) throws IOException {
@@ -33,7 +33,7 @@ public class YoctoRunnableWithProgress extends YoctoHostShellProcessAdapter
try {
this.monitor = monitor;
this.monitor.beginTask(taskName, RemoteHelper.TOTALWORKLOAD);
-
+
if (!remoteConnection.isOpen()) {
try {
remoteConnection.open(monitor);
@@ -48,7 +48,8 @@ public class YoctoRunnableWithProgress extends YoctoHostShellProcessAdapter

try {
IHost connection = RemoteHelper.getRemoteConnectionByName(remoteConnection.getName());
- RemoteHelper.runCommandRemote(connection, new YoctoCommand(cmd, "", args));
+// RemoteHelper.runCommandRemote(connection, new YoctoCommand(cmd, "", args));
+ RemoteHelper.handleRunCommandRemote(connection, new YoctoCommand(cmd, "", args), monitor);
} catch (Exception e) {
e.printStackTrace();
} finally {
diff --git a/plugins/org.yocto.bc.ui/src/org/yocto/bc/ui/wizards/NewBitBakeFileRecipeWizard.java b/plugins/org.yocto.bc.ui/src/org/yocto/bc/ui/wizards/NewBitBakeFileRecipeWizard.java
index 7345b77..66c4b9d 100644
--- a/plugins/org.yocto.bc.ui/src/org/yocto/bc/ui/wizards/NewBitBakeFileRecipeWizard.java
+++ b/plugins/org.yocto.bc.ui/src/org/yocto/bc/ui/wizards/NewBitBakeFileRecipeWizard.java
@@ -204,7 +204,8 @@ public class NewBitBakeFileRecipeWizard extends Wizard implements INewWizard {
public void run(IProgressMonitor monitor) throws InvocationTargetException {
try {
doFinish(element, monitor);
- RemoteHelper.runCommandRemote(connection, new YoctoCommand("rm -rf " + element.getMetaDir() + "/temp", "", ""));
+// RemoteHelper.runCommandRemote(connection, new YoctoCommand("rm -rf " + element.getMetaDir() + "/temp", "", ""));
+ RemoteHelper.handleRunCommandRemote(connection, new YoctoCommand("rm -rf " + element.getMetaDir() + "/temp", "", ""), monitor);
} catch (Exception e) {
throw new InvocationTargetException(e);
} finally {
diff --git a/plugins/org.yocto.bc.ui/src/org/yocto/bc/ui/wizards/NewBitBakeFileRecipeWizardPage.java b/plugins/org.yocto.bc.ui/src/org/yocto/bc/ui/wizards/NewBitBakeFileRecipeWizardPage.java
index a52c2fe..114a484 100644
--- a/plugins/org.yocto.bc.ui/src/org/yocto/bc/ui/wizards/NewBitBakeFileRecipeWizardPage.java
+++ b/plugins/org.yocto.bc.ui/src/org/yocto/bc/ui/wizards/NewBitBakeFileRecipeWizardPage.java
@@ -91,7 +91,6 @@ public class NewBitBakeFileRecipeWizardPage extends WizardPage {
private static final String MIRRORS_FILE = "mirrors.bbclass";
private static final String CLASSES_FOLDER = "classes";
private static final String COPYING_FILE = "COPYING";
- private static final String WHITESPACES = "\\s+";
private static final String CMAKE_LIST = "cmakelists.txt";
private static final String CMAKE = "cmake";
private static final String SETUP_SCRIPT = "setup.py";
@@ -104,9 +103,9 @@ public class NewBitBakeFileRecipeWizardPage extends WizardPage {

private HashMap<String, String> mirrorTable;
private URI extractDir;
- private YoctoCommand licenseChecksumCmd;
- protected YoctoCommand md5YCmd;
- protected YoctoCommand sha256YCmd;
+ protected ProcessStreamBuffer md5Buffer;
+ protected ProcessStreamBuffer sha256Buffer;
+ protected ProcessStreamBuffer md5CopyingBuffer;

public NewBitBakeFileRecipeWizardPage(ISelection selection, IHost connection) {
super("wizardPage");
@@ -331,7 +330,7 @@ public class NewBitBakeFileRecipeWizardPage extends WizardPage {
}

private void handleLocalPopulate(URI srcURI, IProgressMonitor monitor) {
- populateLicenseFileChecksum(srcURI);
+ populateLicenseFileChecksum(srcURI, monitor);
populateInheritance(srcURI, monitor);
}

@@ -345,47 +344,61 @@ public class NewBitBakeFileRecipeWizardPage extends WizardPage {
public void run(IProgressMonitor monitor) throws InvocationTargetException,
InterruptedException {
monitor.beginTask("Populating recipe fields ... ", 100);
- List<YoctoCommand> commands = new ArrayList<YoctoCommand>();
+// List<YoctoCommand> commands = new ArrayList<YoctoCommand>();

try {
String metaDirLocPath = metaDirLoc.getPath();

monitor.subTask("Cleaning environment");
- commands.add(new YoctoCommand("rm -rf " + TEMP_FOLDER_NAME, metaDirLocPath, ""));
- commands.add(new YoctoCommand( "mkdir " + TEMP_FOLDER_NAME, metaDirLocPath, ""));
+// commands.add(new YoctoCommand("rm -rf " + TEMP_FOLDER_NAME, metaDirLocPath, ""));
+ YoctoCommand rmYCmd = new YoctoCommand("rm -rf " + TEMP_FOLDER_NAME, metaDirLocPath, "");
+ RemoteHelper.handleRunCommandRemote(connection, rmYCmd, monitor);
+
+// commands.add(new YoctoCommand( "mkdir " + TEMP_FOLDER_NAME, metaDirLocPath, ""));
+ YoctoCommand mkdirYCmd = new YoctoCommand( "mkdir " + TEMP_FOLDER_NAME, metaDirLocPath, "");
+ RemoteHelper.handleRunCommandRemote(connection, mkdirYCmd, monitor);
+
updateTempFolderPath();
monitor.worked(10);

monitor.subTask("Downloading package sources");
- commands.add(new YoctoCommand("wget " + srcURI.toURL(), tempFolderPath, ""));
+// commands.add(new YoctoCommand("wget " + srcURI.toURL(), tempFolderPath, ""));

updateTempFolderPath();
- RemoteHelper.runBatchRemote(connection, commands, true);
+// RemoteHelper.runBatchRemote(connection, commands, true);

- commands.clear();
+ YoctoCommand wgetYCmd = new YoctoCommand("wget " + srcURI.toURL(), tempFolderPath, "");
+ RemoteHelper.handleRunCommandRemote(connection, wgetYCmd, monitor);
+
+// commands.clear();
monitor.worked(50);

monitor.subTask("Compute package checksums");
String md5Cmd = "md5sum " + srcFileNameExt;
- md5YCmd = new YoctoCommand(md5Cmd, tempFolderPath, "");
- RemoteHelper.runCommandRemote(connection, md5YCmd);
+ YoctoCommand md5YCmd = new YoctoCommand(md5Cmd, tempFolderPath, "");
+// RemoteHelper.runCommandRemote(connection, md5YCmd);
+ RemoteHelper.handleRunCommandRemote(connection, md5YCmd, monitor);
+ md5Buffer = md5YCmd.getProcessBuffer();

monitor.worked(60);

String sha256Cmd = "sha256sum " + srcFileNameExt;
- sha256YCmd = new YoctoCommand(sha256Cmd, tempFolderPath, "");
- RemoteHelper.runCommandRemote(connection, sha256YCmd);
+ YoctoCommand sha256YCmd = new YoctoCommand(sha256Cmd, tempFolderPath, "");
+// RemoteHelper.runCommandRemote(connection, sha256YCmd);
+ RemoteHelper.handleRunCommandRemote(connection, sha256YCmd, monitor);
+ sha256Buffer = sha256YCmd.getProcessBuffer();

monitor.worked(70);

monitor.subTask("Extracting package");
- extractDir = extractPackage(srcURI);
+ extractDir = extractPackage(srcURI, monitor);
monitor.worked(80);

- licenseChecksumCmd = populateLicenseFileChecksum(extractDir);
+ YoctoCommand licenseChecksumCmd = populateLicenseFileChecksum(extractDir, monitor);
+ md5CopyingBuffer = licenseChecksumCmd.getProcessBuffer();

monitor.subTask("Creating mirror lookup table");
- mirrorTable = createMirrorLookupTable();
+ mirrorTable = createMirrorLookupTable(monitor);

monitor.worked(90);
monitor.done();
@@ -397,20 +410,20 @@ public class NewBitBakeFileRecipeWizardPage extends WizardPage {
updateSrcUri(mirrorTable, srcURI);
populateInheritance(extractDir, monitor);

- String md5Val = retrieveSum(srcFileNameExt, md5Pattern);
+ String md5Val = md5Buffer.getOutputLineContaining(srcFileNameExt, md5Pattern);
md5sumText.setText(Pattern.matches(md5Pattern, md5Val) ? md5Val : "");
- String sha256Val = retrieveSum(srcFileNameExt, sha256Pattern);
+ String sha256Val = sha256Buffer.getOutputLineContaining(srcFileNameExt, sha256Pattern);
sha256sumText.setText(Pattern.matches(sha256Pattern, sha256Val) ? sha256Val : "");
- String checkSumVal = retrieveSum(COPYING_FILE, md5Pattern);
+ String checkSumVal = md5CopyingBuffer.getOutputLineContaining(COPYING_FILE, md5Pattern);
checksumText.setText(RemoteHelper.createNewURI(extractDir, COPYING_FILE).toString() + ";md5=" + (Pattern.matches(md5Pattern, checkSumVal) ? checkSumVal : ""));
}

- private String retrieveSum(String arg, String pattern) {
- ProcessStreamBuffer buffer = RemoteHelper.getProcessBuffer(this.connection);
- return buffer.getOutputLineContaining(arg, pattern);
- }
+// private String retrieveSum(String arg, String pattern) {
+// ProcessStreamBuffer buffer = RemoteHelper.getProcessBuffer(this.connection);
+// return buffer.getOutputLineContaining(arg, pattern);
+// }

- private URI extractPackage(URI srcURI) {
+ private URI extractPackage(URI srcURI, IProgressMonitor monitor) {
try {
String path = srcFileNameExt;
String tarCmd = "tar ";
@@ -420,7 +433,8 @@ public class NewBitBakeFileRecipeWizardPage extends WizardPage {
tarCmd += "-xvf ";
}

- RemoteHelper.runCommandRemote(connection, new YoctoCommand(tarCmd + path, tempFolderPath, ""));
+// RemoteHelper.runCommandRemote(connection, new YoctoCommand(tarCmd + path, tempFolderPath, ""));
+ RemoteHelper.handleRunCommandRemote(connection, new YoctoCommand(tarCmd + path, tempFolderPath, ""), new NullProgressMonitor());

return RemoteHelper.createNewURI(metaDirLoc, TEMP_FOLDER_NAME + "/" + srcFileName);

@@ -456,13 +470,14 @@ public class NewBitBakeFileRecipeWizardPage extends WizardPage {
}
}

- private YoctoCommand populateLicenseFileChecksum(URI extractDir) {
+ private YoctoCommand populateLicenseFileChecksum(URI extractDir, IProgressMonitor monitor) {
if (extractDir == null)
throw new RuntimeException("Something went wrong during source extraction!");

try {
YoctoCommand catCmd = new YoctoCommand("md5sum " + COPYING_FILE, extractDir.getPath(), "");
- RemoteHelper.runCommandRemote(connection, catCmd);
+// RemoteHelper.runCommandRemote(connection, catCmd);
+ RemoteHelper.handleRunCommandRemote(connection, catCmd, monitor);
return catCmd;
} catch (Exception e) {
throw new RuntimeException("Unable to process file for MD5 calculation", e);
@@ -491,11 +506,12 @@ public class NewBitBakeFileRecipeWizardPage extends WizardPage {
return "";
}

- private HashMap<String, String> createMirrorLookupTable() throws Exception {
+ private HashMap<String, String> createMirrorLookupTable(IProgressMonitor monitor) throws Exception {
HashMap<String, String> mirrorMap = new HashMap<String, String>();

YoctoCommand cmd = new YoctoCommand("cat " + MIRRORS_FILE, getMetaFolderPath() + CLASSES_FOLDER, "");
- RemoteHelper.runCommandRemote(connection, cmd);
+// RemoteHelper.runCommandRemote(connection, cmd);
+ RemoteHelper.handleRunCommandRemote(connection, cmd, monitor);

if (!cmd.getProcessBuffer().hasErrors()){
String delims = "[\\t]+";
--
1.7.9.5


Re: gcov on target

Robert Berger <gmane@...>
 

Hi Thomas,

On 01/11/2013 06:19 PM, Thomas Petazzoni wrote:
Did someone manage to build/run gcov for an armv7a?
... or let's put it otherwise: Why doesn't it exist on the target?
I supposed the idea is to generate to coverage data on the target, and
then analyze them with gcov on the build machine. See
http://gcc.gnu.org/onlinedocs/gcc/Cross_002dprofiling.html#Cross_002dprofiling.

Yes this should work. Thanks!
It's also documented in the kernel source tree [1].


Best regards,

Thomas
[1]:
http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=blob;f=Documentation/gcov.txt;h=e7ca6478cd93d4e6422d1ae92455b347dbdf7ccb;hb=HEAD

Regards,

Robert
..."About Basic: mentally mutilated potential programmers beyond hope of
regeneration." -- Dijkstra

My public pgp key is available,at:
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x90320BF1


Re: meta-cedartrail - where is grub configured for 'install' option?

Darren Hart <dvhart@...>
 

On 01/11/2013 01:00 AM, Chris Tapp wrote:

BTW the change I need to make to the grub file is partially related -
the boot device is sda and the internal (install target) is sdb. Grub
gets configured to boot from sdb but the device is sda on a clean
boot.
That is definitely a weakness in the installer. Would you please open a
bug for this? An improved installer is something we are working on and
this would be good input to track.

Thanks,

--
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel


Re: gcov on target

Thomas Petazzoni
 

Dear Robert Berger,

On Fri, 11 Jan 2013 12:24:47 +0200, Robert Berger wrote:

I use Yocto Project, 1.3 "danny" for a beagle-xm and would like to show
kernel code coverage with a core-image-sato-sdk.
This image is pretty complete. Unfortunately there is no gcov available
on the target.

It looks like it's only built for the host.

Did someone manage to build/run gcov for an armv7a?
... or let's put it otherwise: Why doesn't it exist on the target?
I supposed the idea is to generate to coverage data on the target, and
then analyze them with gcov on the build machine. See
http://gcc.gnu.org/onlinedocs/gcc/Cross_002dprofiling.html#Cross_002dprofiling.

Best regards,

Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com


gcov on target

Robert Berger <gmane@...>
 

Hi,

I use Yocto Project, 1.3 "danny" for a beagle-xm and would like to show
kernel code coverage with a core-image-sato-sdk.
This image is pretty complete. Unfortunately there is no gcov available
on the target.

It looks like it's only built for the host.

Did someone manage to build/run gcov for an armv7a?
... or let's put it otherwise: Why doesn't it exist on the target?

Regards,

Robert...The most likely way for the world to be destroyed, most experts
agree, is by accident. That's where we come in; we're computer
professionals. We cause accidents.

My public pgp key is available,at:
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x90320BF1


Re: meta-cedartrail - where is grub configured for 'install' option?

Chris Tapp
 

On 10 Jan 2013, at 17:15, Darren Hart wrote:



On 01/10/2013 12:09 AM, Chris Tapp wrote:
Hi Darren,

On 7 Jan 2013, at 16:44, Darren Hart wrote:

On 12/09/2012 01:39 PM, Chris Tapp wrote:
I am trying to change the 'install' behaviour for the meta-cedartrail image so that the installed system has a different grub.cfg file. Is there a file I can bbappend to achieve this?

image_live.bbclass refers to the 'install' syslinux label that triggers the install, but I've not been able to get from this to how/where grub.cfg is created.
This is currently rather crudely implemented here:

meta/recipes-core/initrdscripts/files/init-install.sh

A better deployment mechanism is something being discussed and which
should make this more configurable.

For now, you could bbappend initramfs-live-install_1.0.bb and replace the
init-install.sh script with your own.
Thanks. I've already got to look at that script anyway as it fails to
install to an internal USB drive on a DN2800-MT board as (I think) it
isn't waiting long enough for it to enumerate when booting the
'install' image.
Hrm, I thought it waited indefinitely for the device to appear. Or is it
finding something else and bailing out of the loop? Where is the failure?
The only other device is the USB drive I booted from. I've not had a chance yet to really look at what's going on, but it looks as if it's not waiting long enough as it reports no installable devices. If I then check when it exits to the shell I can see the drive is there and re-running the script then means it does find it (the script then fails for other reasons to do with the failed run).

It will sometimes install, so it looks as if the wait is nearly long enough.

I'm using an ATP AF512SSGH if that makes a difference.

BTW the change I need to make to the grub file is partially related - the boot device is sda and the internal (install target) is sdb. Grub gets configured to boot from sdb but the device is sda on a clean boot.

--
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel
Chris Tapp

opensource@...
www.keylevel.com


Re: [PATCH 0/3][eclipse-poky][branch:windows-build]

Zhang, Jessica
 

Hi Ioana,

I'm still seeing problems:

1. seems the reinstate OEFilesystem etc. changes is not taking effect, I'm still running into heap space error with my metadata directory with huge build data. So can you create such a metadata directory and run toolchain, and sato-sdk build and see whether you can create the project against it?
2. Ran into another issue while playing with these patch set, if I leave the clone box checked, even though I do see the error message telling me the directory/project already a clone directory, I was still able to click on finish and all process continue on instead of being blocked by the error message.

Thanks,
Jessica

-----Original Message-----
From: yocto-bounces@... [mailto:yocto-bounces@...] On Behalf Of Ioana Grigoropol
Sent: Thursday, January 10, 2013 7:25 AM
To: yocto@...
Subject: [yocto] [PATCH 0/3][eclipse-poky][branch:windows-build]

*** BLURB HERE ***

Ioana Grigoropol (1):
Fix variables hoover & remote file system detection

yocto (2):
Allow '-' in project names Signed-off-by: yocto
<yocto@yocto-VirtualBox.(none)>
Reinstate OEFileSystem, OEFile and Ignored Paths for Linux

plugins/org.yocto.bc.ui/META-INF/MANIFEST.MF | 2 +-
plugins/org.yocto.bc.ui/plugin.xml | 3 +-
.../src/org/yocto/bc/bitbake/BBSession.java | 4 +
.../org/yocto/bc/bitbake/ProjectInfoHelper.java | 24 ++--
.../src/org/yocto/bc/bitbake/ShellSession.java | 13 ++-
.../org/yocto/bc/remote/utils/RemoteHelper.java | 7 +-
.../org/yocto/bc/remote/utils/RemoteMachine.java | 8 +-
.../src/org/yocto/bc/ui/Activator.java | 45 +++++---
.../org/yocto/bc/ui/BCResourceChangeListener.java | 1 -
.../bc/ui/editors/bitbake/BBVariableTextHover.java | 12 +-
.../src/org/yocto/bc/ui/filesystem/OEFile.java | 120 ++++----------------
.../org/yocto/bc/ui/filesystem/OEFileSystem.java | 50 ++++++--
.../bc/ui/filesystem/OEFileSystemContributor.java | 7 +-
.../org/yocto/bc/ui/filesystem/OEIgnoreFile.java | 31 ++++-
.../src/org/yocto/bc/ui/model/ProjectInfo.java | 21 +++-
.../src/org/yocto/bc/ui/model/YoctoHostFile.java | 100 ++++++++++++----
.../yocto/bc/ui/wizards/install/OptionsPage.java | 3 +-
.../newproject/CreateBBCProjectOperation.java | 23 ++--
18 files changed, 281 insertions(+), 193 deletions(-)

--
1.7.9.5

_______________________________________________
yocto mailing list
yocto@...
https://lists.yoctoproject.org/listinfo/yocto


Re: Shouldn't linux/version.h appear in the SDK's host sysroot?

Zhang, Jessica
 

Hi Patrick,

The toolchain and sysroot are behaving correctly, the x86_64-pokysdk-linux contains the cross tools for your development host, whereas armv7a-vfp-neon-poky-linux-gnueabi contains your target sysroot, and all your target programs should be using files against that sysroot. By not knowing how your compile your package, but if you look at our ADT manual, you should source the environment setup script before you try to use the toolchain and you also need to pass sysroot option to your compiler and linker for them to go to the target sysroot location for headers and libraries.

Thanks,
Jessica

-----Original Message-----
From: yocto-bounces@... [mailto:yocto-bounces@...] On Behalf Of Patrick Turley
Sent: Thursday, January 10, 2013 2:46 PM
To: yocto@...
Subject: [yocto] Shouldn't linux/version.h appear in the SDK's host sysroot?

I'm using a Yocto-based SDK, produced by:

bitbake meta-toolchain-sdk

This creates the file

poky-eglibc-x86_64-arm-toolchain-gmae-1.3.sh

in the directory

build/tmp/deploy/sdk

I ran the installation script to install the SDK on my build system. Within the SDK installation directory, under "sysroots", I have the following two directories:

armv7a-vfp-neon-poky-linux-gnueabi
x86_64-pokysdk-linux

Unless I'm mistaken, the first one reflects the target environment, and the second one contains what I need to do cross-compilation. Thus, the latter contains the cross-compilation tools, and that's the sysroot to which all my builds are directed.

I'm compiling the lm_sensors package for my target. As part of its work, it tries to discover the version of the target linux kernel. To do this, it compiles a very small file (etc/config.c) that includes the header file <linux/kernel.h>.

As it happens, <linux/kernel.h> does *not* appear in the cross-compilation sysroot, so this simple task fails. That header file *does* appear in the target sysroot, but that's of no use to me in the cross-compilation process.

I believe one of the following is true:

1) The lm_sensors package is doing something entirely legitimate, and the file <linux/kernel.h> *should* appear in the cross-compilation sysroot to support this. The Yocto SDK build target needs a small fix so that <linux/kernel.h> is properly added to the cross-compilation sys root. In the meantime, I can work around this oversight by creating a symbolic link.

2) I've made a mistake in setting up my build environment because I don't understand how cross-compilation *should* be done. If done properly, the cross-compilation process will have access to *both* sysroots and get what it needs from each. If this is the case, please tell me what I appear to have done wrong.

3) Something else.

_______________________________________________
yocto mailing list
yocto@...
https://lists.yoctoproject.org/listinfo/yocto


Shouldn't linux/version.h appear in the SDK's host sysroot?

Patrick Turley <PatrickTurley@...>
 

I'm using a Yocto-based SDK, produced by:

bitbake meta-toolchain-sdk

This creates the file

poky-eglibc-x86_64-arm-toolchain-gmae-1.3.sh

in the directory

build/tmp/deploy/sdk

I ran the installation script to install the SDK on my build system. Within the SDK installation directory, under "sysroots", I have the following two directories:

armv7a-vfp-neon-poky-linux-gnueabi
x86_64-pokysdk-linux

Unless I'm mistaken, the first one reflects the target environment, and the second one contains what I need to do cross-compilation. Thus, the latter contains the cross-compilation tools, and that's the sysroot to which all my builds are directed.

I'm compiling the lm_sensors package for my target. As part of its work, it tries to discover the version of the target linux kernel. To do this, it compiles a very small file (etc/config.c) that includes the header file <linux/kernel.h>.

As it happens, <linux/kernel.h> does *not* appear in the cross-compilation sysroot, so this simple task fails. That header file *does* appear in the target sysroot, but that's of no use to me in the cross-compilation process.

I believe one of the following is true:

1) The lm_sensors package is doing something entirely legitimate, and the file <linux/kernel.h> *should* appear in the cross-compilation sysroot to support this. The Yocto SDK build target needs a small fix so that <linux/kernel.h> is properly added to the cross-compilation sys root. In the meantime, I can work around this oversight by creating a symbolic link.

2) I've made a mistake in setting up my build environment because I don't understand how cross-compilation *should* be done. If done properly, the cross-compilation process will have access to *both* sysroots and get what it needs from each. If this is the case, please tell me what I appear to have done wrong.

3) Something else.


recipes for git gfortran java

Edward Vidal <vidal.develone@...>
 

All,
what is the best way to install git gfortran and  java to beagleboards and pandaboards.
With angstrom builds you use opkg.  Since Yocto is rpm base are there repo that have compiled software.
Any and all help will be appreciated.
Thanks