<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Plain Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:"Calibri","sans-serif";}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1671758149;
mso-list-type:hybrid;
mso-list-template-ids:298582696 -245868780 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Calibri","sans-serif";
mso-fareast-font-family:Calibri;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">I made an example of how we can use the functionality described to conduct 3 of our test cases:<o:p></o:p></span></p>
<p class="MsoNormal"><a href="https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=90">https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=90</a>
<o:p></o:p></p>
<p class="MsoNormal"><a href="https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=93">https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=93</a><o:p></o:p></p>
<p class="MsoNormal"><a href="https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=83">https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=83</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">As mentioned, the structure is very similar to the automated runtime tests configuration files.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">NOTE: ${POKYDIR} does not exist but I use it to refer to the location of the poky directory.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">“<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><i>import unittest<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i>import os<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i>from oeqa import oeSelfTest, skipModule<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i>from oeqa.utils.decorators import *<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i><o:p> </o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i>def setUpModule():<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i># Here we set up the environment for a build directory<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i><o:p> </o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i>class bitbake_layers(oeSelfTest):<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i><o:p> </o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> @classmethod<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> def setUpClass(self):<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i># Here we set up special settings for the build environment, like shared sstate-cache directory<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> <o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> def test_show_overlayed(self):<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> oeSelfTest.execute("cp -R ${POKYDIR}/meta/recipes-graphics/clutter ${POKYDIR}/meta/recipes-core/")<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> (status, output) = oeSelfTest.execute("bitbake-layers show_overlayed | grep clutter", build_dir)<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> oeSelfTest.execute("rm -R ${POKYDIR}/meta/recipes-core/clutter")<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> self.assertEqual(status, 0, msg="Could not find overlayed recipe")<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i><o:p> </o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> def test_show_appends(self):<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> oeSelfTest.execute("echo "" > ${POKYDIR}/meta-yocto/recipes-core/busybox/linux-yocto_3.10.bbappend"/")<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> (status, output) = oeSelfTest.execute("bitbake_layers show_appends | grep \"/meta-yocto/recipes-core/busybox/linux-yocto_3.10.bbappend\"", build_dir)<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> self.assertEqual(status, 0, msg=" 'bitbake_layers show_appends' Could not find the bbappend file")<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i><o:p> </o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> def test_show_layers(self):<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> (ls_status) = oeSelfTest.execute("ls ${POKYDIR} | grep meta-intel")<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> if ls_status == 1:<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> (git_status) = oeSelfTest.execute("git clone git://git.yoctoproject.org/meta-intel ${POKYDIR}/")<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> oeSelfTest.file.var.append("${BUILDDIR}/conf.bblayers.conf", "BBLAYERS", "${POKYDIR}/meta-intel")<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> (status, output) = oeSelfTest.execute("bitbake-layers show_layers | grep meta-intel", build_dir)<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> self.assertEqual(status, 0, msg=" 'bitbake-layers show_layers ' could not find meta-intel layer in output")<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i><o:p> </o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> @classmethod<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> def tearDownClass(self):<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i># Even though now it's not needed, here we can reset the git repo in order to revert to default<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i> oeSelfTest.execute("git reset --hard")<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i><o:p> </o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i>def tearDownModule():<o:p></o:p></i></p>
<p class="MsoNormal" style="text-autospace:none"><i># Here we remove the build directory<o:p></o:p></i></p>
<p class="MsoNormal"><span style="color:#1F497D">“<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Corneliu<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Romania Yocto QA<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> yocto-bounces@yoctoproject.org [mailto:yocto-bounces@yoctoproject.org]
<b>On Behalf Of </b>Stoicescu, CorneliuX<br>
<b>Sent:</b> Friday, October 18, 2013 5:36 PM<br>
<b>To:</b> yocto<br>
<b>Subject:</b> [yocto] oe-selftest proof of concept<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="text-autospace:none">Hello,<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"><o:p> </o:p></p>
<p class="MsoNormal" style="text-autospace:none">After a chat with Richard and Stefan, I came up with an outline of how the oe-selftest feature(<a href="https://bugzilla.yoctoproject.org/show_bug.cgi?id=4740">https://bugzilla.yoctoproject.org/show_bug.cgi?id=4740</a>
) should look like. I made a summary of my proposal below. Please feel free to add your thoughts!<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"><o:p> </o:p></p>
<p class="MsoNormal" style="text-autospace:none">There will be a new script introduced(similar to bitbake-selftest) that will use python unit test to execute tests. Name has not been decided but it can be "oe-selftest".<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none">Running this script will not compromise poky in any way.
<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none">Initially, the script does not need any preparation in order to be run. If this changes in the future, the user will be prompted upon execution with the pre-required tasks.<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none">Oe-selftest can be used together with the automated runtime tests if necessary.
<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"><o:p> </o:p></p>
<p class="MsoNormal" style="text-autospace:none">The following types of tests are targeted for the initial implementation:<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none">- testing the functionality of scripts in poky/scripts (such as: bitbake-layers, yocto-bsp, yocto-kernel, yocto-layers)<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none">- testing of the 'bitbake' command and its output (this includes output data validation such as the sstate-cache/ and tmp/ directories)<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"><o:p> </o:p></p>
<p class="MsoNormal" style="text-autospace:none">In order to achieve the above, the script will need to:<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2;text-autospace:none">
<![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>manipulate configuration files and variables<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2;text-autospace:none">
<![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>issue commands in a certain order<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2;text-autospace:none">
<![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>analyze command output<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2;text-autospace:none">
<![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>analyze output files and directories<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2;text-autospace:none">
<![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>manage shared resources such as sstate-cache<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"><o:p> </o:p></p>
<p class="MsoNormal" style="text-autospace:none">Oe-selftest will use Python Unit Testing as follows:<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> - we will use what has already been done with the ‘unittest’ and ‘oeqa’ classes. This includes ‘decorators’ and methods like ‘skipModules’.<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> - the tests configuration files will be similar to those of the automated runtime tests.<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> - because we are using the same framework and structure, the output will also be similar to that of the automated runtime tests<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> - the build environments used will be set up in the setUpClass() or setUpModule() methods<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> * local.conf may be modified here to use shared resources like sstate-cache and downloads directory with other build environments<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> - the build environments used will be removed in the tearDownClass() or tearDownModule() methods<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> * this ensures no residue is left over that we are not aware of<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> - we will add a new class oeqa.oeSelfTest that contains the following base methods:<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"><o:p> </o:p></p>
<p class="MsoNormal" style="text-autospace:none">1) A few methods to manipulate variables in poky in a standardized way:<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> - oeSelfTest.var_rewrite(filepath_relative_to_poky_location, variable_name, new_variable_value)<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> * resets a variable to the specified value(may be similar to how HOB does to local.conf and bblayers.conf)<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> - oeSelfTest.var_append(filepath_relative_to_poky_location, variable_name, appended_variable_value)<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> * appends data to a variable<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> - oeSelfTest.var_remove(filepath_relative_to_poky_location, variable_name, remove_variable_value)<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> * removes data from a variable<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"><o:p> </o:p></p>
<p class="MsoNormal" style="text-autospace:none">2) We need to be able to execute tasks in a multiple build directories environment.<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> - oeSelfTest.execute(command, build_dir)<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> * this method executes the command 'command' from 'build_dir'<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> * for each execution, the environment is sourced (similar to autobuilder)<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> (if the build directory does not exist, the command will fail. This needs to be taken care of in the setUpClass() method)<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"> * it returns the exit code and the output of the command<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"><o:p> </o:p></p>
<p class="MsoNormal" style="text-autospace:none">Additional information/notes:<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none">- class and method naming and structure is not final and open for suggestions<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none">- poky under git is required for the script to work; we will use git to reset modified files to default.<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none">- a policy needs to be set in place(after experience has been gained from using the tool) for working with a shared sstate-cache directory and multiple build environments at a time.<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none">- coding style is very important in order to get a reasonable execution time. At the moment this is not a priority though.<o:p></o:p></p>
<p class="MsoNormal">- no strategy has been put in place yet in order to deal with multiple poky environments<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size:8.0pt"><o:p> </o:p></span></p>
</div>
</body>
</html>