What I need is a x86_64-poky-linux toolchain with -m32 set for everything except kernel (and modules).
For the most part the kernel builds do not take any flags from the outside, i.e. they are all generated based on the kernel build structure, not what is calling the build. (unless you bury flags into the definition of $(CC), etc, but I wouldn't recommend that.
So if you have a 64 bit capable toolchain, have a 64 bit configured kernel (i.e. CONFIG_64BIT=y), are passing -m32 to usespace .. the kernel really should build 64bit.
This was my plan, but yocto always create a 32bit toolchain when it set DEFAULTTUNE to i586 for a 64bit machine.
Have you tried that and are seeing the kernel still be built with the 32bit toolchain ?