Re: Additional log handler

Konrad Weihmann

Thanks to both of you - seems like I missed out on BB_LOGCONFIG, but it's exactly what I was looking for.

On 24.08.20 15:59, Joshua Watt wrote:
On 8/24/20 8:26 AM, Richard Purdie wrote:
On Mon, 2020-08-24 at 08:10 -0500, Joshua Watt wrote:
The newer log handling uses Python's structure logging configuration
[1], so you should be able to use whatever capabilities it has
(including logging different domains/levels to a separate file). In
fact, this is the only way logging is handled by core bitbake now;
all of the other logging mechanisms used by the UI are additional
configuration specified on top of the user's BB_LOGCONFIG file. If
you are interested, you can see this in bitbake/lib/bb/ui/
In that file is an additional tool that might be helpful. Around line
553, there are two commented lines of code:

     #import logging_tree

uncommenting them and installing the logging_tree python package will
make bitbake dump the entire structured logging configuration to
stdout on startup so you can look at it.
On a slightly different but related note, could we remove the
debug_domains code now? I assume that can all be handled by
Ah, right; you jogged my memory :) Structured logging still co-exists with the debug_domains, so it's not the *only* way the core speicfies logging messages. The main reason for this is that we still need to pass the set of active logging domains to the bitbake workers that they only send back requested logging levels over IPC instead of all log levels (per some code I believe you wrote Richard). It should the theoretically possible to send over the entire structured log config (which isn't very large) to the worker and add in worker side handlers to send back logs over IPC instead of the simplified list of logging domains (as a bonus, this should allow you to log anything on the worker, not just things in the "BitBake" domain). IIRC I tried to get this working in the original change, but it ended up being more complicated and not strictly necessary for the feature at hand (extra logging hashequiv on the AB) so I left in the debug_domain method of filtering in the workers. Since I had to leave that in, I also left in the UI code to directly deal with the debug domains. The log handling itself still all goes through Python logging, and it correctly merges the structured logging and user domains so that the actual logging domain level filter level passed to the workers is the lower of the user specified logging domain and whatever is specified for that domain in the structured config. The UI code could fairly easily be converted to instead map the command line arguments to structured logging configuration, which would remove any knowledge of them on the UI side.
TL;DR No, we can't remove it yet, but we could with some work.



Join to automatically receive all group messages.