What is the best way of excluding a kernel configuration fragment?

I am trying to disable sound support in the kernel.  I can create a fragment for this but then this conflicts with the other existing fragments.
For example when using meta-intel and removing sound support then you will receive warnings like:

WARNING: linux-intel-4.19.73+gitAUTOINC+a7cb57afb9_ca05e9cd64-r0 do_kernel_configcheck: [kernel config]: specified values did not make it into the kernel's final configuration:

---------- CONFIG_USB_AUDIO -----------------
Config: CONFIG_USB_AUDIO
From: /.../build_corei7-64/tmp/work-shared/intel-corei7-64/kernel-source/.kernel-meta/configs/v4.19/standard/intel/features/usb/usb-gadgets.cfg
Requested value:  CONFIG_USB_AUDIO=m
Actual value:  

I haven't had much luck with the documentation finding how to prune machine features and fragments; I may just misunderstand how this is supposed to be done.