Familiarize yourself with the general bug reporting procedures first. All of that will apply. When reporting a kernel panic or crash, please remember:
Reports without the above information are useless. This is the minimum we need to be able to track down the issue.
In some situations more information is desirable. Below are outlined some additional steps you can take in certain situations:
Under some circumstances, you may lose the very first message of a panic, stating the reason for the panic.
ddb> show panic 0: kernel: page fault trap, code=0 ddb>
You should get a trace from each processor as part of your report:
ddb{0}> trace
pool_get(d05e7c20,0,dab19ef8,d0169414,80) at pool_get+0x226
fxp_add_rfabuf(d0a62000,d3c12b00,dab19f10,dab19f10) at fxp_add_rfabuf+0xa5
fxp_intr(d0a62000) at fxp_intr+0x1e7
Xintr_ioapic0() at Xintr_ioapic0+0x6d
--- interrupt ---
idle_loop+0x21:
ddb{0}> machine ddbcpu 1
Stopped at Debugger+0x4: leave
ddb{1}> trace
Debugger(d0319e28,d05ff5a0,dab1bee8,d031cc6e,d0a61800) at Debugger+0x4
i386_ipi_db(d0a61800,d05ff5a0,dab1bef8,d01eb997) at i386_ipi_db+0xb
i386_ipi_handler(b0,d05f0058,dab10010,d01d0010,dab10010) at i386_ipi_handler+0x
4a
Xintripi() at Xintripi+0x47
--- interrupt ---
i386_softintlock(0,58,dab10010,dab10010,d01e0010) at i386_softintlock+0x37
Xintrltimer() at Xintrltimer+0x47
--- interrupt ---
idle_loop+0x21:
ddb{1}>
Repeat the machine ddbcpu x followed by trace for each
processor in your machine.
A typical kernel crash on OpenBSD might look like this:
kernel: page fault trap, code=0 Stopped at pf_route+0x263: mov 0x40(%edi),%edx ddb>
This crash happened at offset 0x263 in the function pf_route.
The first command to run from the
ddb(4) prompt is trace:
ddb> trace pf_route(e28cb7e4,e28bc978,2,1fad,d0b8b120) at pf_route+0x263 pf_test(2,1f4ad,e28cb7e4,b4c1) at pf_test+0x706 pf_route(e28cbb00,e28bc978,2,d0a65440,d0b8b120) at pf_route+0x207 pf_test(2,d0a65440,e28cbb00,d023c282) at pf_test+0x706 ip_output(d0b6a200,0,0,0,0) at ip_output+0xb67 icmp_send(d0b6a200,0,1,a012) at icmp_send+0x57 icmp_reflect(d0b6a200,0,1,0,3) at icmp_reflect+0x26b icmp_input(d0b6a200,14,0,0,d0b6a200) at icmp_input+0x42c ipv4_input(d0b6a200,e289f140,d0a489e0,e289f140) at ipv4_input+0x6eb ipintr(10,10,e289f140,e289f140,e28cbd38) at ipintr+0x8d Bad frame pointer: 0xe28cbcac ddb>
This tells us what function calls lead to the crash.
To find out the particular line of C code that caused the crash, you can do the following:
Find the source file where the crashing function is defined.
In this example, that would be pf_route() in /sys/net/pf.c.
Use objdump(1) to get the
disassembly:
$ cd /sys/arch/$(uname -m)/compile/GENERIC $ objdump -dlr obj/pf.o >/tmp/pf.dis
In the output, grep for the function name:
$ grep "<pf_route>:" /tmp/pf.dis 00007d88 <pf_route>:
Take this first hex number 7d88 and add the offset 0x263 from
the Stopped at line:
$ printf '%x\n' $((0x7d88 + 0x263)) 7feb
Scroll down to the line 7feb.
The assembler instruction should match the one quoted in the Stopped at
line.
Then scroll up to the nearest C line number:
$ more /tmp/pf.dis
/sys/net/pf.c:3872
7fe7: 0f b7 43 02 movzwl 0x2(%ebx),%eax
7feb: 8b 57 40 mov 0x40(%edi),%edx
7fee: 39 d0 cmp %edx,%eax
7ff0: 0f 87 92 00 00 00 ja 8088 <pf_route+0x300>
So, it's precisely line 3872 of pf.c that crashes:
$ nl -ba /sys/net/pf.c | sed -n 3872p
3872 if ((u_int16_t)ip->ip_len <= ifp->if_mtu) {
The kernel that produced the crash output and the object file for objdump must be compiled from the exact same source file, otherwise the offsets won't match.
If you provide both the ddb trace output and the relevant objdump section, that's very helpful.