Date   

[yocto-autobuilder-helper][dunfell 00/15] Pull request (cover letter only)

Steve Sakoman
 

The following changes since commit ef52b284e8cbe90c18fdab6a0d6fa8095a2c4ed9:

send-qa-email: Save the QA email in case it doesn't reach the mailing lists. (2021-02-23 10:24:14 +0000)

are available in the Git repository at:

git://git.yoctoproject.org/yocto-autobuilder-helper contrib/sakoman
http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder-helper/log/?h=contrib/sakoman

Richard Purdie (14):
scripts/run-config: Don't execute steps that don't exist!
scripts/run-config: Ensure stepnum has a value when there are no steps
scripts/run-config: If target is present default to 1 step
run-config: Adapt to two pass execution
scripts/run-config: Improve logfile naming
scripts/run-config: Ensure logging to both logfile and stdout
config.json/run-config: Add human readable descriptions of steps
scripts/run-config: Remove redundant boilerplate json
scripts/shared-repo-unpack: Add flush call to update the output more
regularly before buildtools
config.json/run-config: Add support for shortnames and descriptions
config.json: Unbreak qa-extras locked sigs test
config.json: Add further descriptions
scripts/run-config: Disable output buffering
config.json: Split reproduciblity tests into their own target

Ross Burton (1):
config: build and test SDKs when using package_deb

config.json | 101 +++++++++++++---
scripts/run-config | 235 ++++++++++++++++++++++++++-----------
scripts/shared-repo-unpack | 1 +
scripts/utils.py | 5 +-
4 files changed, 259 insertions(+), 83 deletions(-)

--
2.25.1


Re: To burn 'bitbake package-index' into build process

Ross Burton <ross@...>
 

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=ross/index
is close to what you want, but I didn't quite finish it off!

Ross

On Fri, 26 Mar 2021 at 16:45, keydi <krzysztof.dudziak@...> wrote:

Should it be feasible to burn execution of 'bitbake package-index' into distribution/image build process
by conducting an adaption of meta-data ?
I mean high-level script is used to spawn Bitbake environment then start Bitbake image target.
I am unhappy to mess up high-level script to add that step to build process.
I am unhappy to make detail that level be visible in high-level script.
Rather I prefer to modify meta-data.



Re: [meta-security][PATCH] clamav: upgrade 103.0

Quentin Schulz
 

Hi Armin,

On March 27, 2021 7:04:53 PM UTC, akuster <akuster808@...> wrote:
convert to cmake and general cleanup

include on oe env patch and glibc 2.33 header fixup

if running w/in qemu, need to add qemuparams="-m 2048" to allow
freshclam not to oom

Signed-off-by: Armin Kuster <akuster808@...>
---
.../{clamav_0.101.5.bb => clamav_0.103.0.bb} | 101 +++++++-----------
.../clamav/files/headers_fixup.patch | 58 ++++++++++
.../clamav/files/oe_cmake_fixup.patch | 39 +++++++
3 files changed, 134 insertions(+), 64 deletions(-)
rename recipes-scanners/clamav/{clamav_0.101.5.bb => clamav_0.103.0.bb} (61%)
create mode 100644 recipes-scanners/clamav/files/headers_fixup.patch
create mode 100644 recipes-scanners/clamav/files/oe_cmake_fixup.patch

diff --git a/recipes-scanners/clamav/clamav_0.101.5.bb b/recipes-scanners/clamav/clamav_0.103.0.bb
similarity index 61%
rename from recipes-scanners/clamav/clamav_0.101.5.bb
rename to recipes-scanners/clamav/clamav_0.103.0.bb
index 7dad263..9e50466 100644
--- a/recipes-scanners/clamav/clamav_0.101.5.bb
+++ b/recipes-scanners/clamav/clamav_0.103.0.bb
@@ -4,94 +4,68 @@ HOMEPAGE = "http://www.clamav.net/index.html"
SECTION = "security"
LICENSE = "LGPL-2.1"

-DEPENDS = "libtool db libxml2 openssl zlib curl llvm clamav-native libmspack bison-native"
-DEPENDS_class-native = "db-native openssl-native zlib-native llvm-native curl-native bison-native"
+DEPENDS = "glibc llvm libtool db openssl zlib curl libxml2 bison pcre2 json-c libcheck"

-LIC_FILES_CHKSUM = "file://COPYING.LGPL;beginline=2;endline=3;md5=4b89c05acc71195e9a06edfa2fa7d092"
+LIC_FILES_CHKSUM = "file://COPYING.txt;beginline=2;endline=3;md5=f7029fbbc5898b273d5902896f7bbe17"

-SRCREV = "482fcd413b07e9fd3ef9850e6d01a45f4e187108"
+SRCREV = "5553a5e206ceae5d920368baee7d403f823bcb6f"

-SRC_URI = "git://github.com/vrtadmin/clamav-devel;branch=rel/0.101 \
+SRC_URI = "git://github.com/vrtadmin/clamav-devel;branch=dev/0.104 \
Are you sure it's not dev/0.103 here? Also, there seems to be a 0.103.1 branch available already. I don't really know which version you're targeting. I would say it makes sense to have this branch aligned with the filename of the recipe.

Cheers,
Quentin

file://clamd.conf \
file://freshclam.conf \
file://volatiles.03_clamav \
file://tmpfiles.clamav \
file://${BPN}.service \
- file://freshclam-native.conf \
- "
-
+ file://headers_fixup.patch \
+ file://oe_cmake_fixup.patch \
+"
S = "${WORKDIR}/git"

LEAD_SONAME = "libclamav.so"
-SO_VER = "9.0.4"
+SO_VER = "9.6.0"
+
+BINCONFIG = "${bindir}/clamav-config"

-inherit autotools pkgconfig useradd systemd multilib_header multilib_script
+inherit cmake chrpath pkgconfig useradd systemd multilib_header multilib_script

CLAMAV_UID ?= "clamav"
CLAMAV_GID ?= "clamav"
-INSTALL_CLAMAV_CVD ?= "1"
-
-CLAMAV_USR_DIR = "${STAGING_DIR_NATIVE}/usr"
-CLAMAV_USR_DIR_class-target = "${STAGING_DIR_HOST}/usr"
-
-PACKAGECONFIG_class-target ?= "ncurses bz2"
-PACKAGECONFIG_class-target += " ${@bb.utils.contains("DISTRO_FEATURES", "ipv6", "ipv6", "", d)}"
-PACKAGECONFIG_class-target += "${@bb.utils.contains('DISTRO_FEATURES', 'systemd', 'systemd', '', d)}"
-
-PACKAGECONFIG[pcre] = "--with-pcre=${STAGING_LIBDIR}, --without-pcre, libpcre"
-PACKAGECONFIG[json] = "--with-libjson=${STAGING_LIBDIR}, --without-libjson, json-c,"
-PACKAGECONFIG[ipv6] = "--enable-ipv6, --disable-ipv6"
-PACKAGECONFIG[bz2] = "--with-libbz2-prefix=${CLAMAV_USR_DIR}, --disable-bzip2, bzip2"
-PACKAGECONFIG[ncurses] = "--with-libncurses-prefix=${CLAMAV_USR_DIR}, --without-libncurses-prefix, ncurses, "
-PACKAGECONFIG[systemd] = "--with-systemdsystemunitdir=${systemd_unitdir}/system/, --without-systemdsystemunitdir, "

MULTILIB_SCRIPTS = "${PN}-dev:${bindir}/clamav-config ${PN}-cvd:${localstatedir}/lib/clamav/mirrors.dat"

-EXTRA_OECONF_CLAMAV = "--without-libcheck-prefix --disable-unrar \
- --disable-mempool \
- --program-prefix="" \
- --disable-zlib-vcheck \
- --with-xml=${CLAMAV_USR_DIR} \
- --with-zlib=${CLAMAV_USR_DIR} \
- --with-openssl=${CLAMAV_USR_DIR} \
- --with-libcurl=${CLAMAV_USR_DIR} \
- --with-system-libmspack=${CLAMAV_USR_DIR} \
- --with-iconv=no \
- --enable-check=no \
- "
-
-EXTRA_OECONF_class-native += "${EXTRA_OECONF_CLAMAV}"
-EXTRA_OECONF_class-target += "--with-user=${CLAMAV_UID} --with-group=${CLAMAV_GID} ${EXTRA_OECONF_CLAMAV}"
-
-do_configure () {
- ${S}/configure ${CONFIGUREOPTS} ${EXTRA_OECONF}
-}
+EXTRA_OECMAKE = " -DCMAKE_BUILD_TYPE=Release -DOPTIMIZE=ON -DENABLE_JSON_SHARED=OFF \
+ -DCLAMAV_GROUP=${CLAMAV_GID} -DCLAMAV_USER=${CLAMAV_UID} \
+ -DENABLE_TESTS=OFF -DBUILD_SHARED_LIBS=ON \
+ -DDISABLE_MPOOL=ON -DENABLE_FRESHCLAM_DNS_FIX=ON \
+ "

-do_configure_class-native () {
- ${S}/configure ${CONFIGUREOPTS} ${EXTRA_OECONF}
-}
+PACKAGECONFIG ?= " clamonacc \
+ ${@bb.utils.contains("DISTRO_FEATURES", "systemd", "systemd", "", d)}"

-do_compile_append_class-target() {
- if [ "${INSTALL_CLAMAV_CVD}" = "1" ]; then
- bbnote "CLAMAV creating cvd"
- install -d ${S}/clamav_db
- ${STAGING_BINDIR_NATIVE}/freshclam --datadir=${S}/clamav_db --config=${WORKDIR}/freshclam-native.conf
- fi
-}
+PACKAGECONFIG[milter] = "-DENABLE_MILTER=ON ,-DENABLE_MILTER=OFF, curl, curl"
+PACKAGECONFIG[clamonacc] = "-DENABLE_CLAMONACC=ON ,-DENABLE_CLAMONACC=OFF,"
+PACKAGECONFIG[unrar] = "-DENABLE_UNRAR=ON ,-DENABLE_UNRAR=OFF,"
+PACKAGECONFIG[systemd] = "-DENABLE_SYSTEMD=ON -DSYSTEMD_UNIT_DIR=${systemd_system_unitdir}, -DENABLE_SYSTEMD=OFF, systemd"
+
+export OECMAKE_C_FLAGS += " -I${STAGING_INCDIR} -L ${RECIPE_SYSROOT}${nonarch_libdir} -L${STAGING_LIBDIR} -lpthread"

-do_install_append_class-target () {
+do_install_append () {
install -d ${D}/${sysconfdir}
install -d ${D}/${localstatedir}/lib/clamav
install -d ${D}${sysconfdir}/clamav ${D}${sysconfdir}/default/volatiles

- install -m 644 ${WORKDIR}/clamd.conf ${D}/${sysconfdir}
- install -m 644 ${WORKDIR}/freshclam.conf ${D}/${sysconfdir}
+ install -m 644 ${WORKDIR}/clamd.conf ${D}/${prefix}/${sysconfdir}
+ install -m 644 ${WORKDIR}/freshclam.conf ${D}/${prefix}/${sysconfdir}
install -m 0644 ${WORKDIR}/volatiles.03_clamav ${D}${sysconfdir}/default/volatiles/03_clamav
sed -i -e 's#${STAGING_DIR_HOST}##g' ${D}${libdir}/pkgconfig/libclamav.pc
rm ${D}/${libdir}/libclamav.so
if [ "${INSTALL_CLAMAV_CVD}" = "1" ]; then
install -m 666 ${S}/clamav_db/* ${D}/${localstatedir}/lib/clamav/.
fi
+
+ rm ${D}/${libdir}/libfreshclam.so
+ rm ${D}/${libdir}/libmspack.so
+
if ${@bb.utils.contains('DISTRO_FEATURES','systemd','true','false',d)};then
install -D -m 0644 ${WORKDIR}/clamav.service ${D}${systemd_unitdir}/system/clamav.service
install -d ${D}${sysconfdir}/tmpfiles.d
@@ -114,10 +88,10 @@ pkg_postinst_ontarget_${PN} () {
PACKAGES = "${PN} ${PN}-dev ${PN}-dbg ${PN}-daemon ${PN}-doc ${PN}-cvd \
${PN}-clamdscan ${PN}-freshclam ${PN}-libclamav ${PN}-staticdev"

-FILES_${PN} = "${bindir}/clambc ${bindir}/clamscan ${bindir}/clamsubmit \
+FILES_${PN} = "${bindir}/clambc ${bindir}/clamscan ${bindir}/clamsubmit ${sbindir}/clamonacc \
${bindir}/*sigtool ${mandir}/man1/clambc* ${mandir}/man1/clamscan* \
${mandir}/man1/sigtool* ${mandir}/man1/clambsubmit* \
- ${docdir}/clamav/* "
+ ${docdir}/clamav/* ${libdir}/libmspack* "

FILES_${PN}-clamdscan = " ${bindir}/clamdscan \
${docdir}/clamdscan/* \
@@ -128,12 +102,14 @@ FILES_${PN}-daemon = "${bindir}/clamconf ${bindir}/clamdtop ${sbindir}/clamd \
${mandir}/man1/clamconf* ${mandir}/man1/clamdtop* \
${mandir}/man5/clamd* ${mandir}/man8/clamd* \
${sysconfdir}/clamd.conf* \
+ /usr/etc/clamd.conf* \
${systemd_unitdir}/system/clamav-daemon/* \
${docdir}/clamav-daemon/* ${sysconfdir}/clamav-daemon \
${sysconfdir}/logcheck/ignore.d.server/clamav-daemon "

FILES_${PN}-freshclam = "${bindir}/freshclam \
${sysconfdir}/freshclam.conf* \
+ /usr/etc/freshclam.conf* \
${sysconfdir}/clamav ${sysconfdir}/default/volatiles \
${sysconfdir}/tmpfiles.d/*.conf \
${localstatedir}/lib/clamav \
@@ -148,8 +124,8 @@ FILES_${PN}-dev = " ${bindir}/clamav-config ${libdir}/*.la \

FILES_${PN}-staticdev = "${libdir}/*.a"

-FILES_${PN}-libclamav = "${libdir}/libclamav.so* ${libdir}/libclammspack.so*\
- ${docdir}/libclamav/* "
+FILES_${PN}-libclamav = "${libdir}/libclamav.so* ${libdir}/libclammspack.so* \
+ ${libdir}/libfreshclam.so* ${docdir}/libclamav/* "

FILES_${PN}-doc = "${mandir}/man/* \
${datadir}/man/* \
@@ -169,6 +145,3 @@ RCONFLICTS_${PN} += "${PN}-systemd"
SYSTEMD_SERVICE_${PN} = "${BPN}.service"

RDEPENDS_${PN} = "openssl ncurses-libncurses libxml2 libbz2 ncurses-libtinfo curl libpcre2 clamav-freshclam clamav-libclamav"
-RDEPENDS_${PN}_class-native = ""
-
-BBCLASSEXTEND = "native"
diff --git a/recipes-scanners/clamav/files/headers_fixup.patch b/recipes-scanners/clamav/files/headers_fixup.patch
new file mode 100644
index 0000000..9de0a26
--- /dev/null
+++ b/recipes-scanners/clamav/files/headers_fixup.patch
@@ -0,0 +1,58 @@
+Fixes checks not needed do to glibc 2.33
+
+Upstream-Status: Pending
+Signed-off-by: Armin Kuster <akuster808@...>
+
+Index: git/CMakeLists.txt
+===================================================================
+--- git.orig/CMakeLists.txt
++++ git/CMakeLists.txt
+@@ -374,8 +373,6 @@ check_include_file("stdlib.h"
+ check_include_file("string.h" HAVE_STRING_H)
+ check_include_file("strings.h" HAVE_STRINGS_H)
+ check_include_file("sys/cdefs.h" HAVE_SYS_CDEFS_H)
+-check_include_file("sys/dl.h" HAVE_SYS_DL_H)
+-check_include_file("sys/fileio.h" HAVE_SYS_FILIO_H)
+ check_include_file("sys/mman.h" HAVE_SYS_MMAN_H)
+ check_include_file("sys/param.h" HAVE_SYS_PARAM_H)
+ check_include_file("sys/queue.h" HAVE_SYS_QUEUE_H)
+@@ -410,8 +407,6 @@ endif()
+
+ # int-types variants
+ check_include_file("inttypes.h" HAVE_INTTYPES_H)
+-check_include_file("sys/inttypes.h" HAVE_SYS_INTTYPES_H)
+-check_include_file("sys/int_types.h" HAVE_SYS_INT_TYPES_H)
+ check_include_file("stdint.h" HAVE_STDINT_H)
+
+ # this hack required to silence warnings on systems with inttypes.h
+@@ -539,17 +528,11 @@ check_type_size("time_t" SIZEOF_TIME_T)
+ # Checks for library functions.
+ include(CheckSymbolExists)
+ check_symbol_exists(_Exit "stdlib.h" HAVE__EXIT)
+-check_symbol_exists(accept4 "sys/types.h" HAVE_ACCEPT4)
+ check_symbol_exists(snprintf "stdio.h" HAVE_SNPRINTF)
+-check_symbol_exists(stat64 "sys/stat.h" HAVE_STAT64)
+-check_symbol_exists(strcasestr "string.h" HAVE_STRCASESTR)
+ check_symbol_exists(strerror_r "string.h" HAVE_STRERROR_R)
+-check_symbol_exists(strlcat "string.h" HAVE_STRLCAT)
+-check_symbol_exists(strlcpy "string.h" HAVE_STRLCPY)
+ check_symbol_exists(strndup "string.h" HAVE_STRNDUP)
+ check_symbol_exists(strnlen "string.h" HAVE_STRNLEN)
+-check_symbol_exists(strnstr "string.h" HAVE_STRNSTR)
+-check_symbol_exists(sysctlbyname "sysctl.h" HAVE_SYSCTLBYNAME)
++check_symbol_exists(strcasecmp "string.h" HAVE_STRNCMP)
+ check_symbol_exists(timegm "time.h" HAVE_TIMEGM)
+ check_symbol_exists(vsnprintf "stdio.h" HAVE_VSNPRINTF)
+
+@@ -563,10 +546,9 @@ else()
+ check_symbol_exists(fseeko "stdio.h" HAVE_FSEEKO)
+ check_symbol_exists(getaddrinfo "netdb.h" HAVE_GETADDRINFO)
+ check_symbol_exists(getpagesize "unistd.h" HAVE_GETPAGESIZE)
+- check_symbol_exists(mkstemp "unistd.h" HAVE_MKSTEMP)
+ check_symbol_exists(poll "poll.h" HAVE_POLL)
+- check_symbol_exists(setgroups "unistd.h" HAVE_SETGROUPS)
+ check_symbol_exists(setsid "unistd.h" HAVE_SETSID)
++ set(HAVE_SYSCONF_SC_PAGESIZE 1)
+ endif()
+
+ include(CheckSymbolExists)
diff --git a/recipes-scanners/clamav/files/oe_cmake_fixup.patch b/recipes-scanners/clamav/files/oe_cmake_fixup.patch
new file mode 100644
index 0000000..b284915
--- /dev/null
+++ b/recipes-scanners/clamav/files/oe_cmake_fixup.patch
@@ -0,0 +1,39 @@
+Issue with rpath including /usr/lib and crosscompile checkes causing oe configure to fail
+
+Use oe's cmake rpath framework and exclude some of the cmake checks that fail in our env
+
+Upstream-Status: Inappropriate [configuration]
+Singed-off-by: Armin Kuster <akuster808@...>
+
+Index: git/CMakeLists.txt
+===================================================================
+--- git.orig/CMakeLists.txt
++++ git/CMakeLists.txt
+@@ -162,12 +162,6 @@ endif()
+
+ include(GNUInstallDirs)
+
+-if(CMAKE_INSTALL_FULL_LIBDIR)
+- set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_FULL_LIBDIR}")
+-else()
+- set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib")
+-endif()
+-
+ if(C_LINUX)
+ if(CMAKE_COMPILER_IS_GNUCXX)
+ # Set _GNU_SOURCE for O_LARGEFILE, O_CLOEXEC, O_DIRECTORY, O_NOFOLLOW, etc flags on older systems
+@@ -512,14 +506,8 @@ include(TestInline)
+ include(CheckFileOffsetBits)
+ # Determine how to pack structs on this platform.
+ include(CheckStructPacking)
+-# Check for signed right shift implementation.
+-include(CheckSignedRightShift)
+ # Check if systtem fts implementation available
+ include(CheckFTS)
+-# Check if uname(2) follows POSIX standard.
+-include(CheckUnamePosix)
+-# Check support for file descriptor passing
+-include(CheckFDPassing)
+
+ # Check if big-endian
+ include(TestBigEndian)
--
2.25.1
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.


[meta-security][PATCH] clamav: upgrade 103.0

Armin Kuster
 

convert to cmake and general cleanup

include on oe env patch and glibc 2.33 header fixup

if running w/in qemu, need to add qemuparams="-m 2048" to allow
freshclam not to oom

Signed-off-by: Armin Kuster <akuster808@...>
---
.../{clamav_0.101.5.bb => clamav_0.103.0.bb} | 101 +++++++-----------
.../clamav/files/headers_fixup.patch | 58 ++++++++++
.../clamav/files/oe_cmake_fixup.patch | 39 +++++++
3 files changed, 134 insertions(+), 64 deletions(-)
rename recipes-scanners/clamav/{clamav_0.101.5.bb => clamav_0.103.0.bb} (61%)
create mode 100644 recipes-scanners/clamav/files/headers_fixup.patch
create mode 100644 recipes-scanners/clamav/files/oe_cmake_fixup.patch

diff --git a/recipes-scanners/clamav/clamav_0.101.5.bb b/recipes-scanners/clamav/clamav_0.103.0.bb
similarity index 61%
rename from recipes-scanners/clamav/clamav_0.101.5.bb
rename to recipes-scanners/clamav/clamav_0.103.0.bb
index 7dad263..9e50466 100644
--- a/recipes-scanners/clamav/clamav_0.101.5.bb
+++ b/recipes-scanners/clamav/clamav_0.103.0.bb
@@ -4,94 +4,68 @@ HOMEPAGE = "http://www.clamav.net/index.html"
SECTION = "security"
LICENSE = "LGPL-2.1"

-DEPENDS = "libtool db libxml2 openssl zlib curl llvm clamav-native libmspack bison-native"
-DEPENDS_class-native = "db-native openssl-native zlib-native llvm-native curl-native bison-native"
+DEPENDS = "glibc llvm libtool db openssl zlib curl libxml2 bison pcre2 json-c libcheck"

-LIC_FILES_CHKSUM = "file://COPYING.LGPL;beginline=2;endline=3;md5=4b89c05acc71195e9a06edfa2fa7d092"
+LIC_FILES_CHKSUM = "file://COPYING.txt;beginline=2;endline=3;md5=f7029fbbc5898b273d5902896f7bbe17"

-SRCREV = "482fcd413b07e9fd3ef9850e6d01a45f4e187108"
+SRCREV = "5553a5e206ceae5d920368baee7d403f823bcb6f"

-SRC_URI = "git://github.com/vrtadmin/clamav-devel;branch=rel/0.101 \
+SRC_URI = "git://github.com/vrtadmin/clamav-devel;branch=dev/0.104 \
file://clamd.conf \
file://freshclam.conf \
file://volatiles.03_clamav \
file://tmpfiles.clamav \
file://${BPN}.service \
- file://freshclam-native.conf \
- "
-
+ file://headers_fixup.patch \
+ file://oe_cmake_fixup.patch \
+"
S = "${WORKDIR}/git"

LEAD_SONAME = "libclamav.so"
-SO_VER = "9.0.4"
+SO_VER = "9.6.0"
+
+BINCONFIG = "${bindir}/clamav-config"

-inherit autotools pkgconfig useradd systemd multilib_header multilib_script
+inherit cmake chrpath pkgconfig useradd systemd multilib_header multilib_script

CLAMAV_UID ?= "clamav"
CLAMAV_GID ?= "clamav"
-INSTALL_CLAMAV_CVD ?= "1"
-
-CLAMAV_USR_DIR = "${STAGING_DIR_NATIVE}/usr"
-CLAMAV_USR_DIR_class-target = "${STAGING_DIR_HOST}/usr"
-
-PACKAGECONFIG_class-target ?= "ncurses bz2"
-PACKAGECONFIG_class-target += " ${@bb.utils.contains("DISTRO_FEATURES", "ipv6", "ipv6", "", d)}"
-PACKAGECONFIG_class-target += "${@bb.utils.contains('DISTRO_FEATURES', 'systemd', 'systemd', '', d)}"
-
-PACKAGECONFIG[pcre] = "--with-pcre=${STAGING_LIBDIR}, --without-pcre, libpcre"
-PACKAGECONFIG[json] = "--with-libjson=${STAGING_LIBDIR}, --without-libjson, json-c,"
-PACKAGECONFIG[ipv6] = "--enable-ipv6, --disable-ipv6"
-PACKAGECONFIG[bz2] = "--with-libbz2-prefix=${CLAMAV_USR_DIR}, --disable-bzip2, bzip2"
-PACKAGECONFIG[ncurses] = "--with-libncurses-prefix=${CLAMAV_USR_DIR}, --without-libncurses-prefix, ncurses, "
-PACKAGECONFIG[systemd] = "--with-systemdsystemunitdir=${systemd_unitdir}/system/, --without-systemdsystemunitdir, "

MULTILIB_SCRIPTS = "${PN}-dev:${bindir}/clamav-config ${PN}-cvd:${localstatedir}/lib/clamav/mirrors.dat"

-EXTRA_OECONF_CLAMAV = "--without-libcheck-prefix --disable-unrar \
- --disable-mempool \
- --program-prefix="" \
- --disable-zlib-vcheck \
- --with-xml=${CLAMAV_USR_DIR} \
- --with-zlib=${CLAMAV_USR_DIR} \
- --with-openssl=${CLAMAV_USR_DIR} \
- --with-libcurl=${CLAMAV_USR_DIR} \
- --with-system-libmspack=${CLAMAV_USR_DIR} \
- --with-iconv=no \
- --enable-check=no \
- "
-
-EXTRA_OECONF_class-native += "${EXTRA_OECONF_CLAMAV}"
-EXTRA_OECONF_class-target += "--with-user=${CLAMAV_UID} --with-group=${CLAMAV_GID} ${EXTRA_OECONF_CLAMAV}"
-
-do_configure () {
- ${S}/configure ${CONFIGUREOPTS} ${EXTRA_OECONF}
-}
+EXTRA_OECMAKE = " -DCMAKE_BUILD_TYPE=Release -DOPTIMIZE=ON -DENABLE_JSON_SHARED=OFF \
+ -DCLAMAV_GROUP=${CLAMAV_GID} -DCLAMAV_USER=${CLAMAV_UID} \
+ -DENABLE_TESTS=OFF -DBUILD_SHARED_LIBS=ON \
+ -DDISABLE_MPOOL=ON -DENABLE_FRESHCLAM_DNS_FIX=ON \
+ "

-do_configure_class-native () {
- ${S}/configure ${CONFIGUREOPTS} ${EXTRA_OECONF}
-}
+PACKAGECONFIG ?= " clamonacc \
+ ${@bb.utils.contains("DISTRO_FEATURES", "systemd", "systemd", "", d)}"

-do_compile_append_class-target() {
- if [ "${INSTALL_CLAMAV_CVD}" = "1" ]; then
- bbnote "CLAMAV creating cvd"
- install -d ${S}/clamav_db
- ${STAGING_BINDIR_NATIVE}/freshclam --datadir=${S}/clamav_db --config=${WORKDIR}/freshclam-native.conf
- fi
-}
+PACKAGECONFIG[milter] = "-DENABLE_MILTER=ON ,-DENABLE_MILTER=OFF, curl, curl"
+PACKAGECONFIG[clamonacc] = "-DENABLE_CLAMONACC=ON ,-DENABLE_CLAMONACC=OFF,"
+PACKAGECONFIG[unrar] = "-DENABLE_UNRAR=ON ,-DENABLE_UNRAR=OFF,"
+PACKAGECONFIG[systemd] = "-DENABLE_SYSTEMD=ON -DSYSTEMD_UNIT_DIR=${systemd_system_unitdir}, -DENABLE_SYSTEMD=OFF, systemd"
+
+export OECMAKE_C_FLAGS += " -I${STAGING_INCDIR} -L ${RECIPE_SYSROOT}${nonarch_libdir} -L${STAGING_LIBDIR} -lpthread"

-do_install_append_class-target () {
+do_install_append () {
install -d ${D}/${sysconfdir}
install -d ${D}/${localstatedir}/lib/clamav
install -d ${D}${sysconfdir}/clamav ${D}${sysconfdir}/default/volatiles

- install -m 644 ${WORKDIR}/clamd.conf ${D}/${sysconfdir}
- install -m 644 ${WORKDIR}/freshclam.conf ${D}/${sysconfdir}
+ install -m 644 ${WORKDIR}/clamd.conf ${D}/${prefix}/${sysconfdir}
+ install -m 644 ${WORKDIR}/freshclam.conf ${D}/${prefix}/${sysconfdir}
install -m 0644 ${WORKDIR}/volatiles.03_clamav ${D}${sysconfdir}/default/volatiles/03_clamav
sed -i -e 's#${STAGING_DIR_HOST}##g' ${D}${libdir}/pkgconfig/libclamav.pc
rm ${D}/${libdir}/libclamav.so
if [ "${INSTALL_CLAMAV_CVD}" = "1" ]; then
install -m 666 ${S}/clamav_db/* ${D}/${localstatedir}/lib/clamav/.
fi
+
+ rm ${D}/${libdir}/libfreshclam.so
+ rm ${D}/${libdir}/libmspack.so
+
if ${@bb.utils.contains('DISTRO_FEATURES','systemd','true','false',d)};then
install -D -m 0644 ${WORKDIR}/clamav.service ${D}${systemd_unitdir}/system/clamav.service
install -d ${D}${sysconfdir}/tmpfiles.d
@@ -114,10 +88,10 @@ pkg_postinst_ontarget_${PN} () {
PACKAGES = "${PN} ${PN}-dev ${PN}-dbg ${PN}-daemon ${PN}-doc ${PN}-cvd \
${PN}-clamdscan ${PN}-freshclam ${PN}-libclamav ${PN}-staticdev"

-FILES_${PN} = "${bindir}/clambc ${bindir}/clamscan ${bindir}/clamsubmit \
+FILES_${PN} = "${bindir}/clambc ${bindir}/clamscan ${bindir}/clamsubmit ${sbindir}/clamonacc \
${bindir}/*sigtool ${mandir}/man1/clambc* ${mandir}/man1/clamscan* \
${mandir}/man1/sigtool* ${mandir}/man1/clambsubmit* \
- ${docdir}/clamav/* "
+ ${docdir}/clamav/* ${libdir}/libmspack* "

FILES_${PN}-clamdscan = " ${bindir}/clamdscan \
${docdir}/clamdscan/* \
@@ -128,12 +102,14 @@ FILES_${PN}-daemon = "${bindir}/clamconf ${bindir}/clamdtop ${sbindir}/clamd \
${mandir}/man1/clamconf* ${mandir}/man1/clamdtop* \
${mandir}/man5/clamd* ${mandir}/man8/clamd* \
${sysconfdir}/clamd.conf* \
+ /usr/etc/clamd.conf* \
${systemd_unitdir}/system/clamav-daemon/* \
${docdir}/clamav-daemon/* ${sysconfdir}/clamav-daemon \
${sysconfdir}/logcheck/ignore.d.server/clamav-daemon "

FILES_${PN}-freshclam = "${bindir}/freshclam \
${sysconfdir}/freshclam.conf* \
+ /usr/etc/freshclam.conf* \
${sysconfdir}/clamav ${sysconfdir}/default/volatiles \
${sysconfdir}/tmpfiles.d/*.conf \
${localstatedir}/lib/clamav \
@@ -148,8 +124,8 @@ FILES_${PN}-dev = " ${bindir}/clamav-config ${libdir}/*.la \

FILES_${PN}-staticdev = "${libdir}/*.a"

-FILES_${PN}-libclamav = "${libdir}/libclamav.so* ${libdir}/libclammspack.so*\
- ${docdir}/libclamav/* "
+FILES_${PN}-libclamav = "${libdir}/libclamav.so* ${libdir}/libclammspack.so* \
+ ${libdir}/libfreshclam.so* ${docdir}/libclamav/* "

FILES_${PN}-doc = "${mandir}/man/* \
${datadir}/man/* \
@@ -169,6 +145,3 @@ RCONFLICTS_${PN} += "${PN}-systemd"
SYSTEMD_SERVICE_${PN} = "${BPN}.service"

RDEPENDS_${PN} = "openssl ncurses-libncurses libxml2 libbz2 ncurses-libtinfo curl libpcre2 clamav-freshclam clamav-libclamav"
-RDEPENDS_${PN}_class-native = ""
-
-BBCLASSEXTEND = "native"
diff --git a/recipes-scanners/clamav/files/headers_fixup.patch b/recipes-scanners/clamav/files/headers_fixup.patch
new file mode 100644
index 0000000..9de0a26
--- /dev/null
+++ b/recipes-scanners/clamav/files/headers_fixup.patch
@@ -0,0 +1,58 @@
+Fixes checks not needed do to glibc 2.33
+
+Upstream-Status: Pending
+Signed-off-by: Armin Kuster <akuster808@...>
+
+Index: git/CMakeLists.txt
+===================================================================
+--- git.orig/CMakeLists.txt
++++ git/CMakeLists.txt
+@@ -374,8 +373,6 @@ check_include_file("stdlib.h"
+ check_include_file("string.h" HAVE_STRING_H)
+ check_include_file("strings.h" HAVE_STRINGS_H)
+ check_include_file("sys/cdefs.h" HAVE_SYS_CDEFS_H)
+-check_include_file("sys/dl.h" HAVE_SYS_DL_H)
+-check_include_file("sys/fileio.h" HAVE_SYS_FILIO_H)
+ check_include_file("sys/mman.h" HAVE_SYS_MMAN_H)
+ check_include_file("sys/param.h" HAVE_SYS_PARAM_H)
+ check_include_file("sys/queue.h" HAVE_SYS_QUEUE_H)
+@@ -410,8 +407,6 @@ endif()
+
+ # int-types variants
+ check_include_file("inttypes.h" HAVE_INTTYPES_H)
+-check_include_file("sys/inttypes.h" HAVE_SYS_INTTYPES_H)
+-check_include_file("sys/int_types.h" HAVE_SYS_INT_TYPES_H)
+ check_include_file("stdint.h" HAVE_STDINT_H)
+
+ # this hack required to silence warnings on systems with inttypes.h
+@@ -539,17 +528,11 @@ check_type_size("time_t" SIZEOF_TIME_T)
+ # Checks for library functions.
+ include(CheckSymbolExists)
+ check_symbol_exists(_Exit "stdlib.h" HAVE__EXIT)
+-check_symbol_exists(accept4 "sys/types.h" HAVE_ACCEPT4)
+ check_symbol_exists(snprintf "stdio.h" HAVE_SNPRINTF)
+-check_symbol_exists(stat64 "sys/stat.h" HAVE_STAT64)
+-check_symbol_exists(strcasestr "string.h" HAVE_STRCASESTR)
+ check_symbol_exists(strerror_r "string.h" HAVE_STRERROR_R)
+-check_symbol_exists(strlcat "string.h" HAVE_STRLCAT)
+-check_symbol_exists(strlcpy "string.h" HAVE_STRLCPY)
+ check_symbol_exists(strndup "string.h" HAVE_STRNDUP)
+ check_symbol_exists(strnlen "string.h" HAVE_STRNLEN)
+-check_symbol_exists(strnstr "string.h" HAVE_STRNSTR)
+-check_symbol_exists(sysctlbyname "sysctl.h" HAVE_SYSCTLBYNAME)
++check_symbol_exists(strcasecmp "string.h" HAVE_STRNCMP)
+ check_symbol_exists(timegm "time.h" HAVE_TIMEGM)
+ check_symbol_exists(vsnprintf "stdio.h" HAVE_VSNPRINTF)
+
+@@ -563,10 +546,9 @@ else()
+ check_symbol_exists(fseeko "stdio.h" HAVE_FSEEKO)
+ check_symbol_exists(getaddrinfo "netdb.h" HAVE_GETADDRINFO)
+ check_symbol_exists(getpagesize "unistd.h" HAVE_GETPAGESIZE)
+- check_symbol_exists(mkstemp "unistd.h" HAVE_MKSTEMP)
+ check_symbol_exists(poll "poll.h" HAVE_POLL)
+- check_symbol_exists(setgroups "unistd.h" HAVE_SETGROUPS)
+ check_symbol_exists(setsid "unistd.h" HAVE_SETSID)
++ set(HAVE_SYSCONF_SC_PAGESIZE 1)
+ endif()
+
+ include(CheckSymbolExists)
diff --git a/recipes-scanners/clamav/files/oe_cmake_fixup.patch b/recipes-scanners/clamav/files/oe_cmake_fixup.patch
new file mode 100644
index 0000000..b284915
--- /dev/null
+++ b/recipes-scanners/clamav/files/oe_cmake_fixup.patch
@@ -0,0 +1,39 @@
+Issue with rpath including /usr/lib and crosscompile checkes causing oe configure to fail
+
+Use oe's cmake rpath framework and exclude some of the cmake checks that fail in our env
+
+Upstream-Status: Inappropriate [configuration]
+Singed-off-by: Armin Kuster <akuster808@...>
+
+Index: git/CMakeLists.txt
+===================================================================
+--- git.orig/CMakeLists.txt
++++ git/CMakeLists.txt
+@@ -162,12 +162,6 @@ endif()
+
+ include(GNUInstallDirs)
+
+-if(CMAKE_INSTALL_FULL_LIBDIR)
+- set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_FULL_LIBDIR}")
+-else()
+- set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib")
+-endif()
+-
+ if(C_LINUX)
+ if(CMAKE_COMPILER_IS_GNUCXX)
+ # Set _GNU_SOURCE for O_LARGEFILE, O_CLOEXEC, O_DIRECTORY, O_NOFOLLOW, etc flags on older systems
+@@ -512,14 +506,8 @@ include(TestInline)
+ include(CheckFileOffsetBits)
+ # Determine how to pack structs on this platform.
+ include(CheckStructPacking)
+-# Check for signed right shift implementation.
+-include(CheckSignedRightShift)
+ # Check if systtem fts implementation available
+ include(CheckFTS)
+-# Check if uname(2) follows POSIX standard.
+-include(CheckUnamePosix)
+-# Check support for file descriptor passing
+-include(CheckFDPassing)
+
+ # Check if big-endian
+ include(TestBigEndian)
--
2.25.1


[meta-openssl102] Updated

Mark Hatle
 

Not sure if anyone is still using this, but I did update OpenSSL 1.0.2 to fix
the latest three CVEs:

CVE-2021-23839
CVE-2021-23840
CVE-2021-23841

All branches updated.

Additionally master has been updated for 'hardknott'.

Note, corresponding meta-openssl102-fips does not build. If anyone is
interested in this, they will likely need to help update it.

--Mark


Re: Yocto Technical Team Minutes, Engineering Sync, for March 23, 2021

Alexander Kanavin
 

On Fri, 26 Mar 2021 at 21:44, Trevor Woerner <twoerner@...> wrote:
RP: if anyone knows any RISC-V companies, we’d like to get some of them
    interested in becoming members. i’d really like to add RISC-V as part
    of our matrix, but that would be hard if it doesn’t have a source of
    funding

I guess SiFive is the one you could approach? I actually sent them my CV a few weeks back, as a long shot to try to see if they're interested in Yocto and someone like myself to help them with it.

Alex


Yocto Technical Team Minutes, Engineering Sync, for March 23, 2021

Trevor Woerner
 

Yocto Technical Team Minutes, Engineering Sync, for March 23, 2021
archive: https://docs.google.com/document/d/1ly8nyhO14kDNnFcW2QskANXW3ZT7QwKC5wWVDg9dDH4/edit

== disclaimer ==
Best efforts are made to ensure the below is accurate and valid. However,
errors sometimes happen. If any errors or omissions are found, please feel
free to reply to this email with any corrections.

== attendees ==
Trevor Woerner, Stephen Jolley, Armin Kuster, Michael Opdenacker, Joshua
Watt (JPEW), Saul Wold, Jan-Simon Möller, Scott Murray, Richard Purdie,
Alexandre Belloni, Randy MacLeod, Steve Sakoman, Tim Orling, Trevor
Gamblin, Bruce Ashfield, Michael Halstead, Peter Kjellerstedt (saur), Paul
Barker

== notes ==
- 3.3-m3 rc1 had issue with beaglebone, rc2 to be built as soon as it’s fixed
- layer compatibility bumped to hardknott
- a bunch of version updates were taken (based on guidance from TSC)
- sstate task output before build stats (master-next)
- still lots of “pending” patches which appear to be stale
- still lots of AB-INT issues

== general ==
AlexB: is the stale sstate thing optional? maybe useful cache might be lost
RP: it’s not removing it from cache, it’s removing it if installed in
build directory. see util-linux patch for an example of when/why (the
old util-linux was split into: util-linux-uuid and util-linux). the new
packages have overlapping files that depend on each other. the cache
wasn’t cleaned until -uuid was run which lead to stale data. so this was
proposed as a fix for issues such as this
<various>: sounds like a good change
RP: i’m pretty sure it’s not going to cause issues, but we’re still
leaving it in master-next for the time-being. i’m happy with how fast it
runs and my initial testing seems to indicate there aren’t any follow-on
problems. it’s running on the AB. master-next will become -rc2


RP: TimO: any perl updates? I saw some perl patches on the list and was hoping
for your input
TimO: sorry, haven’t seen the patch
RP: it’s in master-next right now, looks like just a SRC_URI change for now
TimO: i’ve been working on fixing this sort of issue at a higher level.
i’m working on the perl-deps issue, but i don’t want to hold up the
release


TimO: i’ve also been working on recipe-tool for perl modules. currently lost
down a rabbit hole of elastic-search (i.e. the backend for cpan)
ScottM: that goes beyond just using cpan?
TimO: yes, i want to search the cpan for dependencies myself while working on
automated dependency tracking
RP: doing it in self-contained steps is recommended


Randy: anyone heard from Kevin wrt beaglebone blocking -m3
RP: a patch was sent last night, looks like a wic issue (dosutils)
Randy: are you building -rc2 now, or waiting for valgrind
RP: i think i’m going to skip valgrind now. -rc2 already has a lot of
patches (more than usual) so i think we’ll try to cut it off
Randy: one of the things a new valgrind brings is something related to
debuginfod, which might be interesting
RP: yes, given the outreachy project. but given valgrind’s history and
issues we’ve had in the past, i’d rather wait for a new release before
pulling that in, we’re too late now


RP: Randy, any news on AB load issue
Randy: none. still under investigation. maybe tomorrow? we have an in-house
setup that we’re working with
RP: TrevorG do you have a question for MichaelH?
TrevorG: wondering about updates to AB, automated? manual?
MichaelH: for the workers i do them manually. I do some tests to start on each
of them, then start the update manually. we could do something at startup
to look for updates automatically
TrevorG: it works for me with CentOS and Fedora
MichaelH: there are differences between the versions of systemd, so we’re
not sure of the update scripts will work across all workers yet
TrevorG: we just want to make sure that our internal AB setup is as close as
possible to the “real” AB setup


RP: after we started the AB after some maintenance the connection to the
swapbot was lost (the connection wasn’t initialized properly at
startup). i noticed this on Monday and re-init the connection, so it’s
working now, but there seems to be something amiss
MichaelH: okay. we’ll need to add this to the maintenance routine and make
sure that’s working
AlexB: last week was quite quiet, least number of failures i’ve seen recently
RP: probably 2 reasons:
1. we’re reaching a release milestone, so less invasive changes lately
2. fewer changes in the earliest pieces (e.g. gcc patches) leads to more
sstate reuse and tends to cause fewer issues overall
SteveS: it was nice getting builds done fast this week


ScottM: docs status update
RP: MichaelO from Bootlin is on today’s call, and will be helping out with
the docs going forward taking over some of the work Scott Rifenbark used
to do. Nico gave it a good push and MarkM had been working on it, but
it’ll be nice to get some fresh eyes on it.


MichaelO: any plans to support clang
RP: it’s there, but not in core (yet?)
MichaelO: maybe something good to document?
RP: hard to say because it’s not in core. we might have to re-evaluate these
sorts of things (pick specific examples of not-in-core things to document)


Randy: should we consider removing MIPS (now that MIPS has joined RISC-V)
RP: existing hardware isn’t going to disappear, and we do still have members
using it. but we could re-focus our testing away from it moving forward.
e.g. just move to qemu only testing and remove the actual MIPS hardware
being supported (edge router)
PaulB: should we move it out of core into meta-mips (?)
RP: there isn’t much there, so probably not worth moving out (i.e. a couple
machines and some compiler tunings)
PaulB: well, mostly about reducing the test matrix size
RP: i’d like to keep the optics that it is supported, if we cut it out, i
don’t like the message it might send to the embedded community. but we
don’t have to test *everything* that’s in oe-core
PaulB: each machine in oe-core expands to all sorts of builds and build tests
(e.g. reproducibility, ptests, etc…)
RP: there already are areas that aren’t extensively tested across all of
oe-core (e.g. musl)
Armin: ppc64 is in oe-core, but not tested as much as others
RP: we’ve been reducing our coverage, we are in control of the test matrix


RP: if anyone knows any RISC-V companies, we’d like to get some of them
interested in becoming members. i’d really like to add RISC-V as part
of our matrix, but that would be hard if it doesn’t have a source of
funding


TimO: thinking about 5.12 kernel (in rc4 now)
RP: -m3 is delayed and late, so expect -m4 to be very short. if the kernel is
still at only -rc4 then we’re probably still weeks away until a final
kernel release. so i doubt they’ll align in time


Re: [ptest-runner][PATCH 4/4v2] utils.c: wait_child reimplement timeout using alarm

Yi Fan Yu
 

On 3/24/21 2:40 PM, Aníbal Limón wrote:
[Please note: This e-mail is from an EXTERNAL e-mail address]
Since we are using threads to read from child, no complex logic is
needed for handle timeouts by std{out,err} in the child using alarm(2).
[YOCTO #14220]
Signed-off-by: Aníbal Limón <anibal.limon@...>
LGTM, solves my bug and my issue with the old timeout behaviour.

I tested glib-2.0 with the extra debugging output. It works.
Also doesn't timeout.


---
utils.c | 65 ++++++++++++++++++++++++---------------------------------
1 file changed, 27 insertions(+), 38 deletions(-)
diff --git a/utils.c b/utils.c
index 84cb570..b28107d 100644
--- a/utils.c
+++ b/utils.c
@@ -57,6 +57,10 @@
static struct {
int fds[2];
FILE *fps[2];
+
+ int timeout;
+ int timeouted;
+ pid_t pid;
} _child_reader;
static inline char *
@@ -304,6 +308,9 @@ read_child(void *arg)
fwrite(buf, (size_t)n, 1, _child_reader.fps[1]);
}
+ /* Child output reset alarm */
+ alarm(0);
+ alarm(_child_reader.timeout);
I looked at https://linux.die.net/man/2/alarm
I don't think alarm(0) is necessary, but it is fine.

}
fflush(_child_reader.fps[0]);
@@ -335,45 +342,28 @@ run_child(char *run_ptest, int fd_stdout, int fd_stderr)
/* exit(1); not needed? */
}
-static inline int
-wait_child(pid_t pid, int timeout, int *timeouted)
+static void
+timeout_child_handler(int signo)
{
- struct timespec sentinel;
- clockid_t clock = CLOCK_MONOTONIC;
- int looping = 1;
+ _child_reader.timeouted = 1;
+ kill(-_child_reader.pid, SIGKILL);
+}
+static inline int
+wait_child(pid_t pid, int timeout)
+{
int status = -1;
- int waitflags;
-
- if (clock_gettime(clock, &sentinel) == -1) {
- clock = CLOCK_REALTIME;
- clock_gettime(clock, &sentinel);
- }
- *timeouted = 0;
+ _child_reader.timeout = timeout;
+ _child_reader.timeouted = 0;
+ _child_reader.pid = pid;
- while (looping) {
- waitflags = WNOHANG;
+ /* setup alarm to timeout based on std{out,err} in the child */
+ alarm(timeout);
- if (timeout >= 0) {
- struct timespec time;
-
- clock_gettime(clock, &time);
- if ((time.tv_sec - sentinel.tv_sec) > timeout) {
- *timeouted = 1;
- kill(-pid, SIGKILL);
- waitflags = 0;
- }
- }
-
- if (waitpid(pid, &status, waitflags) == pid)
- looping = 0;
-
- clock_gettime(clock, &sentinel);
-
- if (WIFEXITED(status))
- status = WEXITSTATUS(status);
- }
+ waitpid(pid, &status, 0);
+ if (WIFEXITED(status))
+ status = WEXITSTATUS(status);
return status;
}
@@ -438,7 +428,6 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
pid_t child;
int pipefd_stdout[2];
int pipefd_stderr[2];
- int timeouted;
time_t sttime, entime;
time_t duration;
int slave;
@@ -477,6 +466,7 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
close(pipefd_stdout[1]);
break;
}
+ signal(SIGALRM, timeout_child_handler);
fprintf(fp, "START: %s\n", progname);
PTEST_LIST_ITERATE_START(head, p)
@@ -527,8 +517,7 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
fprintf(fp, "BEGIN: %s\n", ptest_dir);
- status = wait_child(child, opts.timeout, &timeouted);
-
+ status = wait_child(child, opts.timeout);
entime = time(NULL);
duration = entime - sttime;
@@ -538,11 +527,11 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
rc += 1;
}
fprintf(fp, "DURATION: %d\n", (int) duration);
- if (timeouted)
+ if (_child_reader.timeouted)
fprintf(fp, "TIMEOUT: %s\n", ptest_dir);
if (opts.xml_filename)
- xml_add_case(xh, status, ptest_dir, timeouted, (int) duration);
+ xml_add_case(xh, status, ptest_dir, _child_reader.timeouted, (int) duration);
fprintf(fp, "END: %s\n", ptest_dir);
fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE, entime));
--
2.31.0


[error-report-web][PATCH v2 5/5] Import parser from the new python3 location

Michael Halstead
 

---
Post/views.py | 2 +-
README | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Post/views.py b/Post/views.py
index 7791201..161d46d 100644
--- a/Post/views.py
+++ b/Post/views.py
@@ -24,7 +24,7 @@ from django.http import JsonResponse
from django.db.models import Q
import json
import urllib
-from urlparse import urlparse
+from urllib.parse import urlparse

class results_mode(object):
LATEST = 0
diff --git a/README b/README
index f759d28..d56fdbd 100644
--- a/README
+++ b/README
@@ -48,7 +48,7 @@ The latest version can be checked out by git cloning:
git://git.yoctoproject.org/error-report-web

Contributions are welcome. Please send patches / pull requests to
-yocto@... with '[error-report-web]' in the subject.
+yocto@... with '[error-report-web]' in the subject.

To manage the size of your database there is a convenience command:

--
2.11.0


[error-report-web][PATCH v2 4/5] Search on contains, enalble admin, and keep more error history.

Michael Halstead
 

---
Post/feed.py | 2 +-
Post/purge.py | 5 +++--
Post/views.py | 3 +--
project/urls.py | 2 +-
4 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/Post/feed.py b/Post/feed.py
index 7e5bfdb..5d57b54 100644
--- a/Post/feed.py
+++ b/Post/feed.py
@@ -27,7 +27,7 @@ class LatestEntriesFeed(Feed):
if self.mode == results_mode.SPECIAL_SUBMITTER and hasattr(settings,"SPECIAL_SUBMITTER"):
#Special submitter mode see settings.py to enable
name = settings.SPECIAL_SUBMITTER['name']
- queryset = BuildFailure.objects.order_by('-BUILD__DATE').filter(BUILD__NAME__istartswith=name)[:self.limit]
+ queryset = BuildFailure.objects.order_by('-BUILD__DATE').filter(BUILD__NAME__icontains=name)[:self.limit]

else:
queryset = BuildFailure.objects.order_by('-BUILD__DATE')[:self.limit]
diff --git a/Post/purge.py b/Post/purge.py
index 1289d13..dd58441 100644
--- a/Post/purge.py
+++ b/Post/purge.py
@@ -15,11 +15,12 @@ def setup_django():
def main():
setup_django()
from Post.models import BuildFailure
- delete_before = timezone.now()-timedelta(days=30)
+ delete_before = timezone.now()-timedelta(days=45)
query = "SELECT bf.id FROM Post_buildfailure bf LEFT JOIN Post_build b ON (bf.BUILD_id = b.id) WHERE bf.REFERER NOT IN ('OTHER','NO_REFERER') AND b.DATE < '{0}'".format(delete_before.date())
- print query
+ #print query
items = BuildFailure.objects.raw(query)
for item in items:
+ print "Deleting: ", item.id
item.delete()

if __name__ == "__main__":
diff --git a/Post/views.py b/Post/views.py
index 0ad58ba..7791201 100644
--- a/Post/views.py
+++ b/Post/views.py
@@ -45,7 +45,6 @@ def common_context(request):

return ret

-
@csrf_exempt
def addData(request, return_json=False):
response = ''
@@ -208,7 +207,7 @@ def search(request, mode=results_mode.LATEST, **kwargs):
if mode == results_mode.SPECIAL_SUBMITTER and hasattr(settings,"SPECIAL_SUBMITTER"):
#Special submitter mode see settings.py to enable
name = settings.SPECIAL_SUBMITTER['name']
- items = items.filter(BUILD__NAME__istartswith=name)
+ items = items.filter(BUILD__NAME__icontains=name)

elif mode == results_mode.SEARCH and "query" in request.GET:
query = request.GET["query"]
diff --git a/project/urls.py b/project/urls.py
index 1f51c7c..c1ac55f 100644
--- a/project/urls.py
+++ b/project/urls.py
@@ -24,7 +24,7 @@ urlpatterns = patterns('',
#url(r'^admin/doc/', include('django.contrib.admindocs.urls')),

# Uncomment the next line to enable the admin:
- #url(r'^admin/', include(admin.site.urls)),
+ url(r'^admin/', include(admin.site.urls)),
#url(r'^accounts/', include('registration.backends.default.urls')),
url(r'^(?i)Errors/Latest/$', 'Post.views.search', { 'mode' : results_mode.LATEST }, name= "latest_errors"),
url(r'^(?i)Errors/Latest/feed$', LatestEntriesFeed(), name="errors_feed"),
--
2.11.0


[error-report-web][PATCH v2 3/5] Allow alternate settings, avoid out of memory condition

Michael Halstead
 

---
Post/purge.py | 19 ++++++++-----------
1 file changed, 8 insertions(+), 11 deletions(-)

diff --git a/Post/purge.py b/Post/purge.py
index 829fada..1289d13 100644
--- a/Post/purge.py
+++ b/Post/purge.py
@@ -1,4 +1,4 @@
-from datetime import datetime
+from datetime import datetime, timedelta
from django.utils import timezone
import os
import sys
@@ -8,22 +8,19 @@ def setup_django():
# Get access to our Django model
newpath = os.path.abspath(os.path.dirname(__file__)) + '/..'
sys.path.append(newpath)
- os.environ['DJANGO_SETTINGS_MODULE'] = 'project.settings'
+ if not os.getenv('DJANGO_SETTINGS_MODULE'):
+ os.environ['DJANGO_SETTINGS_MODULE'] = 'project.settings'
django.setup()

def main():
setup_django()
from Post.models import BuildFailure
- items = BuildFailure.objects.all()
- now = timezone.now()
+ delete_before = timezone.now()-timedelta(days=30)
+ query = "SELECT bf.id FROM Post_buildfailure bf LEFT JOIN Post_build b ON (bf.BUILD_id = b.id) WHERE bf.REFERER NOT IN ('OTHER','NO_REFERER') AND b.DATE < '{0}'".format(delete_before.date())
+ print query
+ items = BuildFailure.objects.raw(query)
for item in items:
- if item.REFERER == 'OTHER' or item.REFERER == 'NO_REFERER':
- continue
- difference = now - item.BUILD.DATE
- if difference.days > 30:
- item.delete()
+ item.delete()

if __name__ == "__main__":
main()
-
-
--
2.11.0


[error-report-web][PATCH v2 2/5] Avoid errors caused by attempting to update referrers for objects that don't exist.

Michael Halstead
 

---
Post/views.py | 24 ++++++++++++------------
1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/Post/views.py b/Post/views.py
index 5737076..0ad58ba 100644
--- a/Post/views.py
+++ b/Post/views.py
@@ -256,23 +256,23 @@ def search(request, mode=results_mode.LATEST, **kwargs):
items = items.order_by()
return render(request, "latest-errors.html", context)

-
def details(request, fail_id):
try:
build_failure = BuildFailure.objects.get(id=fail_id)
except ObjectDoesNotExist:
build_failure = None
- try:
- referer = urlparse(request.META['HTTP_REFERER'])
- referer_hostname = referer.hostname
- if referer.port:
- referer_hostname += ":" + str(referer.port)
- if referer_hostname != request.get_host():
- build_failure.REFERER = 'OTHER'
- except KeyError:
- # There is no referer
- build_failure.REFERER = 'NO_REFERER'
- build_failure.save()
+ if build_failure:
+ try:
+ referer = urlparse(request.META['HTTP_REFERER'])
+ referer_hostname = referer.hostname
+ if referer.port:
+ referer_hostname += ":" + str(referer.port)
+ if referer_hostname != request.get_host():
+ build_failure.REFERER = 'OTHER'
+ except KeyError:
+ # There is no referer
+ build_failure.REFERER = 'NO_REFERER'
+ build_failure.save()

context = {'detail' : build_failure, 'error_types' : ErrorType }

--
2.11.0


[error-report-web][PATCH v2 1/5] purge.py: Create script that will purge database

Michael Halstead
 

From: Amanda Brindle <amanda.r.brindle@...>

The script will be run on a regular basis to get rid of old reports that
we don't need. This will improve the performance of the application
since the database has grown to a huge size.

The script will remove reports older than thirty days if they have not
been referred to by a host other than the Error Reporting Tool.

The function details() in views.py will keep trick of the referer when
a build failure report is accessed. If there is no referer, we will note
that in the database in order to determine how often there is no
referer. It's possible that crawlers will access reports with no
referer; if that's the case, a lack of a referer should not determine
whether we save an older report or not.

Fixes [YOCTO #12332]

Signed-off-by: Amanda Brindle <amanda.r.brindle@...>
---
Post/migrations/0006_buildfailure_referer.py | 19 ++++++++++++++++++
Post/models.py | 10 ++++++++++
Post/purge.py | 29 ++++++++++++++++++++++++++++
Post/views.py | 16 +++++++++++++--
4 files changed, 72 insertions(+), 2 deletions(-)
create mode 100644 Post/migrations/0006_buildfailure_referer.py
create mode 100644 Post/purge.py

diff --git a/Post/migrations/0006_buildfailure_referer.py b/Post/migrations/0006_buildfailure_referer.py
new file mode 100644
index 0000000..5fad048
--- /dev/null
+++ b/Post/migrations/0006_buildfailure_referer.py
@@ -0,0 +1,19 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+ dependencies = [
+ ('Post', '0005_build_error_type'),
+ ]
+
+ operations = [
+ migrations.AddField(
+ model_name='buildfailure',
+ name='REFERER',
+ field=models.CharField(default=b'NOT_VISITED', max_length=14, choices=[(b'NO_REFERER', b'no_referer'), (b'OTHER', b'other'), (b'NOT_VISITED', b'not_visited')]),
+ ),
+ ]
diff --git a/Post/models.py b/Post/models.py
index cf8c1c2..3fa66f2 100644
--- a/Post/models.py
+++ b/Post/models.py
@@ -61,6 +61,16 @@ class BuildFailure(models.Model):
ERROR_DETAILS = models.TextField(max_length=int(settings.MAX_UPLOAD_SIZE))
BUILD = models.ForeignKey(Build)
LEV_DISTANCE = models.IntegerField(blank=True, null=True)
+ REFERER_CHOICES = (
+ ('NO_REFERER', 'no_referer'),
+ ('OTHER', 'other'),
+ ('NOT_VISITED', 'not_visited')
+ )
+ REFERER = models.CharField(
+ max_length = 14,
+ choices = REFERER_CHOICES,
+ default = 'NOT_VISITED'
+ )

def get_similar_fails(self):
if self.LEV_DISTANCE is None:
diff --git a/Post/purge.py b/Post/purge.py
new file mode 100644
index 0000000..829fada
--- /dev/null
+++ b/Post/purge.py
@@ -0,0 +1,29 @@
+from datetime import datetime
+from django.utils import timezone
+import os
+import sys
+
+def setup_django():
+ import django
+ # Get access to our Django model
+ newpath = os.path.abspath(os.path.dirname(__file__)) + '/..'
+ sys.path.append(newpath)
+ os.environ['DJANGO_SETTINGS_MODULE'] = 'project.settings'
+ django.setup()
+
+def main():
+ setup_django()
+ from Post.models import BuildFailure
+ items = BuildFailure.objects.all()
+ now = timezone.now()
+ for item in items:
+ if item.REFERER == 'OTHER' or item.REFERER == 'NO_REFERER':
+ continue
+ difference = now - item.BUILD.DATE
+ if difference.days > 30:
+ item.delete()
+
+if __name__ == "__main__":
+ main()
+
+
diff --git a/Post/views.py b/Post/views.py
index fe7100e..5737076 100644
--- a/Post/views.py
+++ b/Post/views.py
@@ -24,6 +24,7 @@ from django.http import JsonResponse
from django.db.models import Q
import json
import urllib
+from urlparse import urlparse

class results_mode(object):
LATEST = 0
@@ -258,9 +259,20 @@ def search(request, mode=results_mode.LATEST, **kwargs):

def details(request, fail_id):
try:
- build_failure = BuildFailure.objects.get(id=fail_id)
+ build_failure = BuildFailure.objects.get(id=fail_id)
except ObjectDoesNotExist:
- build_failure = None
+ build_failure = None
+ try:
+ referer = urlparse(request.META['HTTP_REFERER'])
+ referer_hostname = referer.hostname
+ if referer.port:
+ referer_hostname += ":" + str(referer.port)
+ if referer_hostname != request.get_host():
+ build_failure.REFERER = 'OTHER'
+ except KeyError:
+ # There is no referer
+ build_failure.REFERER = 'NO_REFERER'
+ build_failure.save()

context = {'detail' : build_failure, 'error_types' : ErrorType }

--
2.11.0


[error-report-web][PATCH 5/5] Import parser from the new python3 location

Michael Halstead
 

From: Michael Halstead <mhalstead@...>

---
Post/views.py | 2 +-
README | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Post/views.py b/Post/views.py
index 7791201..161d46d 100644
--- a/Post/views.py
+++ b/Post/views.py
@@ -24,7 +24,7 @@ from django.http import JsonResponse
from django.db.models import Q
import json
import urllib
-from urlparse import urlparse
+from urllib.parse import urlparse

class results_mode(object):
LATEST = 0
diff --git a/README b/README
index f759d28..d56fdbd 100644
--- a/README
+++ b/README
@@ -48,7 +48,7 @@ The latest version can be checked out by git cloning:
git://git.yoctoproject.org/error-report-web

Contributions are welcome. Please send patches / pull requests to
-yocto@... with '[error-report-web]' in the subject.
+yocto@... with '[error-report-web]' in the subject.

To manage the size of your database there is a convenience command:

--
2.11.0


[error-report-web][PATCH 4/5] Search on contains, enalble admin, and keep more error history.

Michael Halstead
 

From: Michael Halstead <mhalstead@...>

---
Post/feed.py | 2 +-
Post/purge.py | 5 +++--
Post/views.py | 3 +--
project/urls.py | 2 +-
4 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/Post/feed.py b/Post/feed.py
index 7e5bfdb..5d57b54 100644
--- a/Post/feed.py
+++ b/Post/feed.py
@@ -27,7 +27,7 @@ class LatestEntriesFeed(Feed):
if self.mode == results_mode.SPECIAL_SUBMITTER and hasattr(settings,"SPECIAL_SUBMITTER"):
#Special submitter mode see settings.py to enable
name = settings.SPECIAL_SUBMITTER['name']
- queryset = BuildFailure.objects.order_by('-BUILD__DATE').filter(BUILD__NAME__istartswith=name)[:self.limit]
+ queryset = BuildFailure.objects.order_by('-BUILD__DATE').filter(BUILD__NAME__icontains=name)[:self.limit]

else:
queryset = BuildFailure.objects.order_by('-BUILD__DATE')[:self.limit]
diff --git a/Post/purge.py b/Post/purge.py
index 1289d13..dd58441 100644
--- a/Post/purge.py
+++ b/Post/purge.py
@@ -15,11 +15,12 @@ def setup_django():
def main():
setup_django()
from Post.models import BuildFailure
- delete_before = timezone.now()-timedelta(days=30)
+ delete_before = timezone.now()-timedelta(days=45)
query = "SELECT bf.id FROM Post_buildfailure bf LEFT JOIN Post_build b ON (bf.BUILD_id = b.id) WHERE bf.REFERER NOT IN ('OTHER','NO_REFERER') AND b.DATE < '{0}'".format(delete_before.date())
- print query
+ #print query
items = BuildFailure.objects.raw(query)
for item in items:
+ print "Deleting: ", item.id
item.delete()

if __name__ == "__main__":
diff --git a/Post/views.py b/Post/views.py
index 0ad58ba..7791201 100644
--- a/Post/views.py
+++ b/Post/views.py
@@ -45,7 +45,6 @@ def common_context(request):

return ret

-
@csrf_exempt
def addData(request, return_json=False):
response = ''
@@ -208,7 +207,7 @@ def search(request, mode=results_mode.LATEST, **kwargs):
if mode == results_mode.SPECIAL_SUBMITTER and hasattr(settings,"SPECIAL_SUBMITTER"):
#Special submitter mode see settings.py to enable
name = settings.SPECIAL_SUBMITTER['name']
- items = items.filter(BUILD__NAME__istartswith=name)
+ items = items.filter(BUILD__NAME__icontains=name)

elif mode == results_mode.SEARCH and "query" in request.GET:
query = request.GET["query"]
diff --git a/project/urls.py b/project/urls.py
index 1f51c7c..c1ac55f 100644
--- a/project/urls.py
+++ b/project/urls.py
@@ -24,7 +24,7 @@ urlpatterns = patterns('',
#url(r'^admin/doc/', include('django.contrib.admindocs.urls')),

# Uncomment the next line to enable the admin:
- #url(r'^admin/', include(admin.site.urls)),
+ url(r'^admin/', include(admin.site.urls)),
#url(r'^accounts/', include('registration.backends.default.urls')),
url(r'^(?i)Errors/Latest/$', 'Post.views.search', { 'mode' : results_mode.LATEST }, name= "latest_errors"),
url(r'^(?i)Errors/Latest/feed$', LatestEntriesFeed(), name="errors_feed"),
--
2.11.0


[error-report-web][PATCH 3/5] Allow alternate settings, avoid out of memory condition

Michael Halstead
 

From: Michael Halstead <mhalstead@...>

---
Post/purge.py | 19 ++++++++-----------
1 file changed, 8 insertions(+), 11 deletions(-)

diff --git a/Post/purge.py b/Post/purge.py
index 829fada..1289d13 100644
--- a/Post/purge.py
+++ b/Post/purge.py
@@ -1,4 +1,4 @@
-from datetime import datetime
+from datetime import datetime, timedelta
from django.utils import timezone
import os
import sys
@@ -8,22 +8,19 @@ def setup_django():
# Get access to our Django model
newpath = os.path.abspath(os.path.dirname(__file__)) + '/..'
sys.path.append(newpath)
- os.environ['DJANGO_SETTINGS_MODULE'] = 'project.settings'
+ if not os.getenv('DJANGO_SETTINGS_MODULE'):
+ os.environ['DJANGO_SETTINGS_MODULE'] = 'project.settings'
django.setup()

def main():
setup_django()
from Post.models import BuildFailure
- items = BuildFailure.objects.all()
- now = timezone.now()
+ delete_before = timezone.now()-timedelta(days=30)
+ query = "SELECT bf.id FROM Post_buildfailure bf LEFT JOIN Post_build b ON (bf.BUILD_id = b.id) WHERE bf.REFERER NOT IN ('OTHER','NO_REFERER') AND b.DATE < '{0}'".format(delete_before.date())
+ print query
+ items = BuildFailure.objects.raw(query)
for item in items:
- if item.REFERER == 'OTHER' or item.REFERER == 'NO_REFERER':
- continue
- difference = now - item.BUILD.DATE
- if difference.days > 30:
- item.delete()
+ item.delete()

if __name__ == "__main__":
main()
-
-
--
2.11.0


[error-report-web][PATCH 2/5] Avoid errors caused by attempting to update referrers for objects that don't exist.

Michael Halstead
 

From: Michael Halstead <mhalstead@...>

---
Post/views.py | 24 ++++++++++++------------
1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/Post/views.py b/Post/views.py
index 5737076..0ad58ba 100644
--- a/Post/views.py
+++ b/Post/views.py
@@ -256,23 +256,23 @@ def search(request, mode=results_mode.LATEST, **kwargs):
items = items.order_by()
return render(request, "latest-errors.html", context)

-
def details(request, fail_id):
try:
build_failure = BuildFailure.objects.get(id=fail_id)
except ObjectDoesNotExist:
build_failure = None
- try:
- referer = urlparse(request.META['HTTP_REFERER'])
- referer_hostname = referer.hostname
- if referer.port:
- referer_hostname += ":" + str(referer.port)
- if referer_hostname != request.get_host():
- build_failure.REFERER = 'OTHER'
- except KeyError:
- # There is no referer
- build_failure.REFERER = 'NO_REFERER'
- build_failure.save()
+ if build_failure:
+ try:
+ referer = urlparse(request.META['HTTP_REFERER'])
+ referer_hostname = referer.hostname
+ if referer.port:
+ referer_hostname += ":" + str(referer.port)
+ if referer_hostname != request.get_host():
+ build_failure.REFERER = 'OTHER'
+ except KeyError:
+ # There is no referer
+ build_failure.REFERER = 'NO_REFERER'
+ build_failure.save()

context = {'detail' : build_failure, 'error_types' : ErrorType }

--
2.11.0


[error-report-web][PATCH 1/5] purge.py: Create script that will purge database

Michael Halstead
 

From: Amanda Brindle <amanda.r.brindle@...>

The script will be run on a regular basis to get rid of old reports that
we don't need. This will improve the performance of the application
since the database has grown to a huge size.

The script will remove reports older than thirty days if they have not
been referred to by a host other than the Error Reporting Tool.

The function details() in views.py will keep trick of the referer when
a build failure report is accessed. If there is no referer, we will note
that in the database in order to determine how often there is no
referer. It's possible that crawlers will access reports with no
referer; if that's the case, a lack of a referer should not determine
whether we save an older report or not.

Fixes [YOCTO #12332]

Signed-off-by: Amanda Brindle <amanda.r.brindle@...>
---
Post/migrations/0006_buildfailure_referer.py | 19 ++++++++++++++++++
Post/models.py | 10 ++++++++++
Post/purge.py | 29 ++++++++++++++++++++++++++++
Post/views.py | 16 +++++++++++++--
4 files changed, 72 insertions(+), 2 deletions(-)
create mode 100644 Post/migrations/0006_buildfailure_referer.py
create mode 100644 Post/purge.py

diff --git a/Post/migrations/0006_buildfailure_referer.py b/Post/migrations/0006_buildfailure_referer.py
new file mode 100644
index 0000000..5fad048
--- /dev/null
+++ b/Post/migrations/0006_buildfailure_referer.py
@@ -0,0 +1,19 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+ dependencies = [
+ ('Post', '0005_build_error_type'),
+ ]
+
+ operations = [
+ migrations.AddField(
+ model_name='buildfailure',
+ name='REFERER',
+ field=models.CharField(default=b'NOT_VISITED', max_length=14, choices=[(b'NO_REFERER', b'no_referer'), (b'OTHER', b'other'), (b'NOT_VISITED', b'not_visited')]),
+ ),
+ ]
diff --git a/Post/models.py b/Post/models.py
index cf8c1c2..3fa66f2 100644
--- a/Post/models.py
+++ b/Post/models.py
@@ -61,6 +61,16 @@ class BuildFailure(models.Model):
ERROR_DETAILS = models.TextField(max_length=int(settings.MAX_UPLOAD_SIZE))
BUILD = models.ForeignKey(Build)
LEV_DISTANCE = models.IntegerField(blank=True, null=True)
+ REFERER_CHOICES = (
+ ('NO_REFERER', 'no_referer'),
+ ('OTHER', 'other'),
+ ('NOT_VISITED', 'not_visited')
+ )
+ REFERER = models.CharField(
+ max_length = 14,
+ choices = REFERER_CHOICES,
+ default = 'NOT_VISITED'
+ )

def get_similar_fails(self):
if self.LEV_DISTANCE is None:
diff --git a/Post/purge.py b/Post/purge.py
new file mode 100644
index 0000000..829fada
--- /dev/null
+++ b/Post/purge.py
@@ -0,0 +1,29 @@
+from datetime import datetime
+from django.utils import timezone
+import os
+import sys
+
+def setup_django():
+ import django
+ # Get access to our Django model
+ newpath = os.path.abspath(os.path.dirname(__file__)) + '/..'
+ sys.path.append(newpath)
+ os.environ['DJANGO_SETTINGS_MODULE'] = 'project.settings'
+ django.setup()
+
+def main():
+ setup_django()
+ from Post.models import BuildFailure
+ items = BuildFailure.objects.all()
+ now = timezone.now()
+ for item in items:
+ if item.REFERER == 'OTHER' or item.REFERER == 'NO_REFERER':
+ continue
+ difference = now - item.BUILD.DATE
+ if difference.days > 30:
+ item.delete()
+
+if __name__ == "__main__":
+ main()
+
+
diff --git a/Post/views.py b/Post/views.py
index fe7100e..5737076 100644
--- a/Post/views.py
+++ b/Post/views.py
@@ -24,6 +24,7 @@ from django.http import JsonResponse
from django.db.models import Q
import json
import urllib
+from urlparse import urlparse

class results_mode(object):
LATEST = 0
@@ -258,9 +259,20 @@ def search(request, mode=results_mode.LATEST, **kwargs):

def details(request, fail_id):
try:
- build_failure = BuildFailure.objects.get(id=fail_id)
+ build_failure = BuildFailure.objects.get(id=fail_id)
except ObjectDoesNotExist:
- build_failure = None
+ build_failure = None
+ try:
+ referer = urlparse(request.META['HTTP_REFERER'])
+ referer_hostname = referer.hostname
+ if referer.port:
+ referer_hostname += ":" + str(referer.port)
+ if referer_hostname != request.get_host():
+ build_failure.REFERER = 'OTHER'
+ except KeyError:
+ # There is no referer
+ build_failure.REFERER = 'NO_REFERER'
+ build_failure.save()

context = {'detail' : build_failure, 'error_types' : ErrorType }

--
2.11.0


Re: Howto read bitbake -g files?

Mauro Ziliani
 

Thank you for the answer.
I analyze the problem.

opencv_3.1.bb installs many debug packages by PACKAGES variable

Removing unneeded packages for PACKAGES every *-dev  packages disappears from production image.

MZ


On mar 25 2021, at 7:34 pm, Khem Raj <raj.khem@...> wrote:


On 3/25/21 3:37 AM, Mauro Ziliani wrote:
> An example about my question
>
> Inside package-depends.dot I see this
>
> "opencv-dev" [label="opencv-dev(opencv)
> :3.1+gitAUTOINC+92387b1ef8-r0\n/Sources/softfx/krogoth/sources/meta-openembedded/meta-oe/recipes-support/opencv/opencv_3.1.bb"]
>
> I need to remote opencv-dev from final image.
>
> In pn-depends.dot I see
>
> "opencv" -> "opencv-dev" [style=dashed]

this should only happen if opencv-dev has a .so which is not a symlink
and there is a program in opencv package which depends on it. Can you
expand the content of opencv-dev and see whats in there ?
secondly you can also look at content of opencv package and readelf -d
over the binaries in it and see if some binary is dependent on .so from
-dev package.

you can do this inside opencv build tree under packages-split/ area.
of a built tree.

>
> I understand that
> opencv generates opencv-dev from the recipe opencv_3.1.bb
>
>
>
> Sent from Mailspring <https://getmailspring.com/>, the best free email
> app for work
> On mar 25 2021, at 8:39 am, Mauro Ziliani <mauro@...> wrote:
>
> Hi all
>
> I have to look for the recipes put in my image the *-dev packages.
>
> I removed dev-pkgs and dng-pkgs from IMAGE_FEATURES and debug tweaks
> from DISTRO_FEATURES.
> I don't IMAGE_INSTALL_append any *-dev packages
>
> I generate bitbake  -g image, but I don't produce the PDF because is
> unreadable.
>
> So I read the dot source. I use dot and I know the syntax.
>
> But I don't how to read the graph.
>
>
> MZ
>
>
>
>
> Sent from Mailspring <https://getmailspring.com/>, the best free
> email app for work
>
>
>
>
>


Use bitbake command line to override or modify variable

keydi <krzysztof.dudziak@...>
 

1. Where to learn more about above command options (file path the file referred to to be found; file path usage in command option,....)?
2. Set/modify variable in bitbake command line, e.g. BBPATH="xyz" bitbake <target> - does this support all supported kinds of variable expansion?
3. Any build process variable classes exist of which the settlement or modification in way as (3) does not work?
4. Any build process variable classes/types exist for which to set or modify it in way as (1) does not work?

5. Any use-cases exist for which -R, --postread, -r, --read to be preferred choice compared with (3)?
6. Any use-cases exist for (3) be better choice compared with -R, --postread, -r, --read?

4501 - 4520 of 57408