IMX7D bring up of second core


bartvanderlaan
 

Hi,

I have recently started working on a project that is based on linux-fslc. The target board is a Technexion pico imx7d.
Hopefully I have come to the right place to ask this question and this post provides enough information to paint a complete picture.

I have trouble bringing up the 2nd core of the SoC and have tried to narrow this down. I'm comparing behaviour to a rescue image from Technexion (I have two pico-pi boards) which boots both processors in SVC.

Board with project image: 

[    0.007136] CPU: Testing write buffer coherency: ok
[    0.008708] CPU0: update cpu_capacity 1024
[    0.008738] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
[    0.011558] Setting up static identity map for 0x80100000 - 0x80100078
[    0.012194] rcu: Hierarchical SRCU implementation.
[    0.013650] smp: Bringing up secondary CPUs ...
[    0.016375] smp: Brought up 1 node, 1 CPU
[    0.016405] SMP: Total of 1 processors activated (48.00 BogoMIPS).
[    0.016427] CPU: All CPU(s) started in SVC mode.
Board with rescue image:

CPU: Testing write buffer coherency: ok
CPU0: update cpu_capacity 1024
CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
Setting up static identity map for 0x80100000 - 0x80100058
CPU1: update cpu_capacity 1024
CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
Brought up 2 CPUs
SMP: Total of 2 processors activated (32.00 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
Things I have tried:

- 5.0.7-fslc and 5.4.20-fslc.
- Remove 
CONFIG_ARMV7_BOOT_SEC_DEFAULT in u-boot-fslc, which does boot both processors in HYP mode resulting in some other issues. I don't know much about the differences between secure, non-secure and hypervisor mode and could not find a defenitive answer online which and if a certain mode is mandatory for the 2nd core.
- Debug the boot process in secure mode with pr_info:

SMP is actually trying to get cpu1 up but is failing when invoking callbacks.

Looking at the call below to cpu_up for cpu 1, it returns an error code -38 although the return value is not used in with an error handler.

This traces all the way back to the moment the callback is invoked for CPUHP_BRINGUP_CPU (cpuhp_state #85) where ret = cb(cpu) get’s the value -38, which if I’m not mistaken means ENOSYS.

The callback invocations for this list below did pass without this error just before that for cpu1:

1; CPUHP_CREATE_THREADS
2; CPUHP_PERF_PREPARE
35; CPUHP_WORKQUEUE_PREP
37; CPUHP_HRTIMERS_PREPARE
40; CPUHP_SMPCFD_PREPARE
41; CPUHP_RELAY_PREPARE
44; CPUHP_RCUTREE_PREP
62; CPUHP_TIMERS_PREPARE

Hopefully someone will see my post and share some experience in the matter, or point me to a place where this question might best be answered.

Best regards,
Bart van der Laan


Otavio Salvador
 

Hello,

Please check dunfell branch to check if it works.

Em qua., 30 de set. de 2020 às 03:14, bartvanderlaan
<bartvanderlaan@...> escreveu:


Hi,

I have recently started working on a project that is based on linux-fslc. The target board is a Technexion pico imx7d.
Hopefully I have come to the right place to ask this question and this post provides enough information to paint a complete picture.

I have trouble bringing up the 2nd core of the SoC and have tried to narrow this down. I'm comparing behaviour to a rescue image from Technexion (I have two pico-pi boards) which boots both processors in SVC.

Board with project image:

[ 0.007136] CPU: Testing write buffer coherency: ok
[ 0.008708] CPU0: update cpu_capacity 1024
[ 0.008738] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
[ 0.011558] Setting up static identity map for 0x80100000 - 0x80100078
[ 0.012194] rcu: Hierarchical SRCU implementation.
[ 0.013650] smp: Bringing up secondary CPUs ...
[ 0.016375] smp: Brought up 1 node, 1 CPU
[ 0.016405] SMP: Total of 1 processors activated (48.00 BogoMIPS).
[ 0.016427] CPU: All CPU(s) started in SVC mode.

Board with rescue image:

CPU: Testing write buffer coherency: ok
CPU0: update cpu_capacity 1024
CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
Setting up static identity map for 0x80100000 - 0x80100058
CPU1: update cpu_capacity 1024
CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
Brought up 2 CPUs
SMP: Total of 2 processors activated (32.00 BogoMIPS).
CPU: All CPU(s) started in SVC mode.

Things I have tried:

- 5.0.7-fslc and 5.4.20-fslc.
- Remove CONFIG_ARMV7_BOOT_SEC_DEFAULT in u-boot-fslc, which does boot both processors in HYP mode resulting in some other issues. I don't know much about the differences between secure, non-secure and hypervisor mode and could not find a defenitive answer online which and if a certain mode is mandatory for the 2nd core.
- Debug the boot process in secure mode with pr_info:

SMP is actually trying to get cpu1 up but is failing when invoking callbacks.

Looking at the call below to cpu_up for cpu 1, it returns an error code -38 although the return value is not used in with an error handler.

github.com

Freescale/linux-fslc/blob/c7a0e482a175b632bfd5522bc6d5bfe7785ee9a9/kernel/smp.c#L578

Copy

idle_threads_init();
cpuhp_threads_init();

pr_info("Bringing up secondary CPUs ...\n");

/* FIXME: This should be done in userspace --RR */
for_each_present_cpu(cpu) {
if (num_online_cpus() >= setup_max_cpus)
break;
if (!cpu_online(cpu))
cpu_up(cpu);
}

num_nodes = num_online_nodes();
num_cpus = num_online_cpus();
pr_info("Brought up %d node%s, %d CPU%s\n",
num_nodes, (num_nodes > 1 ? "s" : ""),
num_cpus, (num_cpus > 1 ? "s" : ""));

/* Any cleanup work */
smp_cpus_done(setup_max_cpus);




This traces all the way back to the moment the callback is invoked for CPUHP_BRINGUP_CPU (cpuhp_state #85) where ret = cb(cpu) get’s the value -38, which if I’m not mistaken means ENOSYS.

github.com

Freescale/linux-fslc/blob/c7a0e482a175b632bfd5522bc6d5bfe7785ee9a9/kernel/cpu.c#L168

Copy


return -EAGAIN;
}

if (!step->multi_instance) {
WARN_ON_ONCE(lastp && *lastp);
cb = bringup ? step->startup.single : step->teardown.single;
if (!cb)
return 0;
trace_cpuhp_enter(cpu, st->target, state, cb);
ret = cb(cpu);
trace_cpuhp_exit(cpu, st->state, state, ret);
return ret;
}
cbm = bringup ? step->startup.multi : step->teardown.multi;
if (!cbm)
return 0;

/* Single invocation for instance add/remove */
if (node) {
WARN_ON_ONCE(lastp && *lastp);




The callback invocations for this list below did pass without this error just before that for cpu1:

1; CPUHP_CREATE_THREADS
2; CPUHP_PERF_PREPARE
35; CPUHP_WORKQUEUE_PREP
37; CPUHP_HRTIMERS_PREPARE
40; CPUHP_SMPCFD_PREPARE
41; CPUHP_RELAY_PREPARE
44; CPUHP_RCUTREE_PREP
62; CPUHP_TIMERS_PREPARE

Hopefully someone will see my post and share some experience in the matter, or point me to a place where this question might best be answered.

Best regards,
Bart van der Laan



--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854 Mobile: +1 (347) 903-9750