FW: QA cycle report for 2.4 RC1


libertad
 

Hello All,

Enjoy viewing the full Report for 2.4 RC1:  https://wiki.yoctoproject.org/wiki/WW40_-_2017-10-06_-_Full_Test_Cycle_2.4_RC1

 

======= Summary ========

 

The QA cycle for release 2.4 RC1 is complete.  There are 18 new bugs from which 5 are high priority and are targeted to be fixed in 2.4 M4.  Four of the High Priority bugs (12143[1], 12144[2], 12145[3], 12146[4]) are ptest related and one (12194[5]) is Toaster related.  An additional medium+ priority (12131[6]  Kernel traceback when reboot beaglebone) is targeted to be fixed by 2.4 M4. As an extra note on this report QA wants to highlight that, there are a total of 46 Bugs targeted to be fixed in 2.4 M4 (link [20]).

 

ptest

Results show that python and e2fsprogs are not executing, hence bugs 12143[1] and 12146[4], respectively, were opened and given a high priority.

Bash (bug 12145[3]) had a decrement in the pass rate of  1.27% and  busybox (bug 12144[2]) had a decrement in the pass rate of 0.19%. Although they are high priority, QA sees that the decrement is not very significant, hence this bugs can be lowered in priority.

 

Eclipse

Some good news: testing for Eclipse-Oxygen was  added for the first time to the QA test cycle.

 

Posix

2.4 RC1 Posix results are very similar to last QA cycle 2.4 M3 rc1. They only show that Edgerouter had one more failure if compared with the previous run.

 

                  Failures               2.4 RC1           2.4 M3 RC1

                  Generic86-64           46                      46

                  Beaglebone              45                      45

                  Edgerouter               47                      46

                  Mpc8315e-rdb         45                      45

 

For more details visit:  https://wiki.yoctoproject.org/wiki/Posix_result and https://wiki.yoctoproject.org/wiki/POSIX-results

 

LTP

2.4 RC1 LTP results show that, for Genericx86-64 and Beaglebone, the failures incremented by 1 test. On Edgerouter we have an increment of 2 failures if compared with QA cycle 2.4 M3 rc1.

 

                                                     2.4 RC1               |        2.4 M3 RC1

                                                Total  |  Failures      |      Total   |  Failures

               Genericx86-64         1391 |   27              |      1391   |     26

               Beaglebone              1353 |   18              |      1353   |     17

               Edgerouter               1353 |   34              |      1353   |     32

               Mpc8315e-rdb        1353  |   31              |      1353   |     31

 

For more details: https://wiki.yoctoproject.org/wiki/LTP_result

 

 

Performance

 

In general, the measurements were on the same levels showed up on 2.4_M3. The only noticeable variation was on “rootfs” that is showing an increase of 5% on Ubuntu and 7% on Fedora, although this does not represent an important regression.

 

Ubuntu                 Test       2.4 M3 rc1           2.4_rc1         %

                              sato       1:15:58              1:15:24           -0.75

                              rootfs    2:34                    2:45                 7.14

                              rmwork 1:08:15              1:08:19           0.10

                              kernel    5:36                    5:41                 1.49

                              eSDK      2:39                    2:40                 0.63

                                                                          

                                                                          

Fedora                 Test       2.4 M3 rc1           2.4_rc1         %

                              sato       1:14:13              1:17:02           3.80

                              rootfs    2:35                    2:44                 5.81

                              rmwork 1:09:06              1:10:25           1.91

                              kernel    5:58                    6:04                 1.68

                              eSDK      2:37                    2:42                 3.18

 

 

======= QA-Hints========

 

Due to the fact that, there are 5 high priority bugs and 46 bugs targeted to be fixed and solved by 2.4 M4, QA requests an RC2 so that all the commits/fixes relevant to 2.4 release are integrated.

 

======= Bugs ========

 

       New Bugs

            -12143[1] Ptest-runner does not run python, in 2.4rc1

            -12144[2] ptest-busybox cases failed in 2.4rc1

            -12145[3] ptest-bash cases failed in 2.4rc1

            -12146[4] ptest-e2fsprog cases failed in 2.4rc1

            -12194[5] Toaster server lost "toaster.conf" environment, user settings lost

            -12131[6] Kernel traceback when reboot beaglebone black Medium+

            -12170[7] Add mkfs.ext support to busybox on poky-tiny

            -12138[8] [master] the icon on the connection manager appears without connection in Eclipse Plugin 

            -12168[9] media player - must be warned that it is not possible to play MPEG-1 and MP3 formats

            -12150[10] bug for QA to create Testcases for this new test on testopia AUTO_eSDK_sdkext

            -12133[11] The search results for numbers are incorrect.

            -12140[12] The link and the x button from the search's result did not work as expected

            -12142[13] The color from the table from the project build changed when you use an special character.

            -12164[14] The field from time's task is empty

            -12171[15] After building an image, checking the packages there's nothing.

            -12175[16] [Test Case 946] View detailed configuration information for a build

            -12200[17] The button from New custom image is not working as expected

            -12201[18] The button from New custom image is letting you press it and it’s supposed to be disabled.

 

 

       Not new M+ bugs

            -11994[19]  Graphical qemu doesn’t work over remote X on Fedora 26

 

 

 

Full Bug Report: https://wiki.yoctoproject.org/wiki/WW40_-_2017-10-06_-_Full_Test_Cycle_2.4_RC1#Bugs_Found_during_QA_Test

 

 

======== Links =========

    1.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12143[1]

 

    2.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12144[2]

 

    3.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12145[3]

 

    4.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12146[4]

 

    5.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12194[5]

 

    6.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12131[6]

 

    7.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12170[7]

 

    8.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12138[8]

 

    9.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12168[9]

 

    10.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12150[10]

 

    11.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12133[11]

 

    12.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12140[12]

 

    13.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12142[13]

 

    14.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12164[14]

 

    15.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12171[15]

 

    16.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12175[16]

 

    17.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12200[17]

 

    18.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12201[18]

 

    19.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11994[19]

 

    20.    https://wiki.yoctoproject.org/wiki/WW40_-_2017-10-06_-_Full_Test_Cycle_2.4_RC1#Current_community_Open_Bugs_-_Medium.2B.2FHigh[20]

 

Regards

Libertad G.

 

 

Join yocto@lists.yoctoproject.org to automatically receive all group messages.