Re: [ptest-runner 5/5] main: Do not return number of failed tests when calling ptest-runner


Adrian Freihofer
 

Hi Lukasz

Also our test infrastructure expects an exit value not equal to 0 in
case of a failed test.

Regards,
Adrian

On Wed, 2021-07-21 at 11:46 +0200, ?ukasz Majewski wrote:
Up till now ptest-runner2 returns number of failed tests with its
exit status code. Such use case is not recommended [1] and may cause
issues when there are more than 256 tests to be executed.

To alleviate this issue the number of total tests with number of failed
ones is printed before exit. To be more specific - failure of a single
test doesn't indicate that the ptest-runner itself encounter any issue
during its execution.

One can test this change with executing:
./ptest-runner -d tests/data fail

Links:
[1] - https://www.gnu.org/software/libc/manual/html_node/Exit-Status.html

Signed-off-by: Lukasz Majewski <lukma@denx.de>
---
 main.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/main.c b/main.c
index efa21b2..9f03857 100644
--- a/main.c
+++ b/main.c
@@ -219,6 +219,9 @@ main(int argc, char *argv[])
  ptest_list_remove(run, opts.exclude[i], 1);
 

  rc = run_ptests(run, opts, argv[0], stdout, stderr);
+ fprintf(stdout, "TOTAL: %d FAIL: %d\n", ptest_list_length(run), rc);
+ if (rc > 0)
+ rc = 0;
 

  ptest_list_free_all(&run);
 


Join yocto@lists.yoctoproject.org to automatically receive all group messages.