[ Next Article | Previous Article | Book Contents | Library Home | Legal | Search ]
Kernel Extensions and Device Support Programming Concepts

Basic Subcommands for the KDB Kernel Debugger and kdb Command


h Subcommand

Display the list of valid subcommands. The help subcommand can be reduced at only one topic. The actual list of topics is:

Example

KDB(0)> ? ?
help topics:

        basic subcommands
        trace subcommands
        break points/steps
        dumps/display/decode
        modify memory
        namelists/symbols
        kdbx subcommands (dont use directly)
        watch subcommands 
        conditionnal subcommand
        calculator converter
        machine status
        loader subcommands
        address translation
        system table
        net subcommands
        vmm subcommands 
        trampolin subcommands
        SMP subcommands
        bat/Block Address Translation
        btac/BRAT subcommand
        machdep subcommands
KDB(7)> ? step
CMD      ALIAS    ALIAS    FUNCTION                   ARG


                *** break points/steps ***

b        brk               set/list break point(s)    [-p/-v] [addr]
lb       lbrk              set/list local bp(s)       [-p/-v] [addr]
c        cl                clear break point          [slot|[-p/-v] addr]
lc       lcl               clear local bp             [slot|[-p/-v] addr [ctx]]
ca                         clear all break points
r        return            go to end of function
gt                         go until address           [-p/-v] addr
n        nexti             next instruction           [count]
s        stepi             single step                [count]
S                          step on bl/blr
B                          step on branch

his Subcommand

The print history subcommand prints history subcommands. Argument can specify a number of subcommands. Each subcommand can be edited and executed with usual control characters (ala emacs).

Example

KDB(3)> his ?
Usage: hist [line count]
...... CTRL_A go to beginning of the line
...... CTRL_B one char backward
...... CTRL_D delete one char
...... CTRL_E go to end of line
...... CTRL_F one char forward
...... CTRL_N next command
...... CTRL_P previous command
...... CTRL_U kill line
KDB(3)> his
tpid
f
s 11
r
n 11
p proc+001680
c
dc .kforkx+30 11
mw .kforkx+000040
48005402
.
his ?
KDB(3)>

e Subcommand

The exit subcommand leaves KDB session and returns to the system, all break-points are installed in memory. To leave KDB without break-points, the clear all subcommand must be invoked.

When KDB is called after a kernel panic, the dump argument can be specified to force an AIX dump to be processed.

How to take a dump depending on the type of system crash.

You have "_0c9" on leds while the dump is copied on disk (either on hd7 or hd6) If you entered the debugger through panic, then control is returned to the debugger when the dump is over, and LEDS show xxxx If you entered the debugger through halt_display (c20 on the LEDS), the display shows: 888 102 700 0c0 on the leds when the dump is over.


set Subcommand

The setup subcommand may be used to list and set kdb toggles. Current list of toggles is:

Toggles display_stack_frames and display_stacked_regs can be used to find arguments of routines. Arguments are saved in non-volatile registers or in the current stack. It is an easy way to look for them.

Example

KDB(1)> set
 No toggle name               current value

  1 no_symbol                 false
  2 mst_wanted                true
  3 screen_size               24
  4 power_pc_syntax           true
  5 hardware_target           604
  6 Unix symbols start from 3500
  7 hexadecimal_wanted        true
  8 screen_previous           false
  9 display_stack_frames      false
 10 display_stacked_regs      false
 11 64_bit                    false
 12 emacs_window              false
 13 Thread attached local breakpoint
 14 KDB stops all processors
 15 tweq_r1_r1                true
 16 kext_IF_active            true
 17 kext_IF_active            false
 18 origin		      00000000
KDB(1)> dw 000034CC display memory
000034CC: 00000002  00000008  00010006  00000020
KDB(1)> set 6 1000  toggle change
Unix symbols start from 1000
KDB(1)> dw 000034CC display memory
_system_configuration+000000: 00000002  00000008  00010006  00000020

KDB(4)> sw 464
Switch to thread: <thread+015C00>
KDB(4)> sw u to see user code
KDB(4)> dc 1000A14C
1000A14C       bl    <1000A1A4>
KDB(4)> set 17
trace_back_lookup is true
KDB(4)> dc 1000A14C
.get_superblk+00007C       bl    <.validate_super>

KDB(0)> set origin 002C5338
origin = 002C5338
KDB(0)> b init_heap1
.init_heap1+000000 (real address:002C55F4) permanent & global
KDB(0)> e
Breakpoint
.init_heap1+000000 (ORG+000002BC)     stmw    r24,FFFFFFE0(stkp)  <.mainstk+001EB8> r24=00003A60,FFFFFFE0(stkp)=00384B74
KDB(0)>
In the listing you can see ...
      | 000000                           PDEF     init_heap1
     0|                                  PROC     heap_addr,numpages,flags,heapx,pages,gr3-gr8
     0| 0002BC stm      BF01FFE0   8     STM      #stack(gr1,-32)=gr24-gr31
...

f Subcommand

The stack subcommand displays all the stacks from the current instruction as deep as possible. Interrupts and system calls are crossed and the user stack is also seen. In the user space, trace back allows to have symbolic names. But KDB can not access directly these symbols. Use the +x toogle to have binary addresses (by example, to put a break point on one of these addresses). This toggle is valid until -x is entered with another stack subcommand To have shorter display, you can suppressed the mst display with the corresponding toggle mst_wanted. To have longer display, you can set the corresponding toggle display_stack_wanted. It is possible to display the stack of a specified thread (by its slot entry or address).

If running code is -O compiled, routine parameters are not saved in the stack. KDB warns about that with [??] at the end of the line. In this case, showed parameters may be wrong.

Example

In the following example, we set a break point on v_gettlock, and when the break point is encountered, the stack is displayed. Then we try to display the first argument of the open() syscall. Looking at the code, we can see that argument is saved by copen() in register R31, and this register is saved in the stack by openpath(). Looking at memory pointed by register R31, argument is found: /dev/ptc

KDB(2)> f show the stack
thread+012540 STACK:
[0004AC84]v_gettlock+000000 (00012049, C0011E80, 00000080, 00000000 [??])
[00085C18]v_pregettlock+0000B4 (??, ??, ??, ??)
[000132E8]isync_vcs1+0000D8 (??, ??)
____ Exception (2FF3B400) ____
[000131FC].backt+000000 (00012049, C0011E80 [??])
[0004B220]vm_gettlock+000020 (??, ??)
[0019A64C]iwrite+00013C (??)
[0019D194]finicom+0000A0 (??, ??)
[0019D4F0]comlist+0001CC (??, ??)
[0019D5BC]_commit+000030 (00000000, 00000001, 09C6E9E8, 399028AA,
0000A46F, 0000E2AA, 2D3A4EAA, 2FF3A730)
[001E1B18]jfs_setattr+000258 (??, ??, ??, ??, ??, ??)
[001A5ED4]vnop_setattr+000018 (??, ??, ??, ??, ??, ??)
[001E9008]spec_setattr+00017C (??, ??, ??, ??, ??, ??)
[001A5ED4]vnop_setattr+000018 (??, ??, ??, ??, ??, ??)
[01B655C8]pty_vsetattr+00002C (??, ??, ??, ??, ??, ??)
[01B6584C]pty_setname+000084 (??, ??, ??, ??, ??, ??)
[01B60810]pty_create_ptp+0002C4 (??, ??, ??, ??, ??)
[01B60210]pty_open_comm+00015C (??, ??, ??, ??)
[01B5FFC0]call_pty_open_comm+0000B8 (??, ??, ??, ??)
[01B6526C]ptm_open+000140 (??, ??, ??, ??, ??)
(2)> more (^C to quit) ? 
[01A9A124]open_wrapper+0000D0 (??)
[01A8DF74]csq_protect+000258 (??, ??, ??, ??, ??, ??)
[01A96348]osr_open+0000BC (??)
[01A9C1C8]pse_clone_open+000164 (??, ??, ??, ??)
[001ADCC8]spec_clone+000178 (??, ??, ??, ??, ??)
[001B3FC4]openpnp+0003AC (??, ??, ??, ??, ??)
[001B4178]openpath+000064 (??, ??, ??, ??, ??, ??)
[001B43E8]copen+000130 (??, ??, ??, ??, ??)
[001B44BC]open+000014 (??, ??, ??)
[000037D8].sys_call+000000 ()
[10002E74]doit+00003C (??, ??, ??)
[10003924]main+0004CC (??, ??)
[1000014C].__start+00004C ()
KDB(2)> set 10 show saved registers
display_stacked_regs is true
KDB(2)> f show the stack
thread+012540 STACK:
[0004AC84]v_gettlock+000000 (00012049, C0011E80, 00000080, 00000000 [??])
...
[001B3FC4]openpnp+0003AC (??, ??, ??, ??, ??)
r24 : 2FF3B6E0 r25 : 2FF3B400 r26 : 10002E78 r27 : 00000000 r28 : 00000002 
r29 : 2FF3B3C0 r30 : 00000000 r31 : 20000510 
[001B4178]openpath+000064 (??, ??, ??, ??, ??, ??)
[001B43E8]copen+000130 (??, ??, ??, ??, ??)
r27 : 2A22A424 r28 : E3014000 r29 : E6012540 r30 : 0C87B000 r31 : 00000000 
[001B44BC]open+000014 (??, ??, ??)
...
KDB(2)> dc open 6 look for argument R3
.open+000000    stwu    stkp,FFFFFFC0(stkp)
.open+000004    mflr    r0
.open+000008   addic    r7,stkp,38
.open+00000C     stw    r0,48(stkp)
.open+000010      li    r6,0
.open+000014      bl    <.copen>
KDB(2)> dc copen 9 look for argument R3
.copen+000000    stmw    r27,FFFFFFEC(stkp)
.copen+000004    addi    r28,r4,0
.copen+000008    mflr    r0
.copen+00000C     lwz    r4,D5C(toc)         D5C(toc)=audit_flag
.copen+000010     stw    r0,8(stkp)
.copen+000014    stwu    stkp,FFFFFFA0(stkp)
.copen+000018    cmpi    cr0,r4,0
.copen+00001C   mtcrf    cr5,r28
.copen+000020    addi    r31,r3,0
KDB(2)> d 20000510 display memory location @R31
20000510: 2F64 6576  2F70 7463  0000 0000  416C 6C20   /dev/ptc....All 

In the following example, the problem is to find what is lsfs subcommand waiting for. The answer is given with getfssize arguments, and there are saved in the stack.

# ps -ef|grep lsfs
 root  63046  39258   0   Apr 01  pts/1  0:00 lsfs 
# kdb
Preserving 587377 bytes of symbol table
First symbol sys_resource
PFT:
id....................0007
raddr.............01000000 eaddr.............B0000000
size..............01000000 align.............01000000
valid..1 ros....0 holes..0 io.....0 seg....0 wimg...2

PVT:
id....................0008
raddr.............003BC000 eaddr.............B2000000
size..............001FFDA0 align.............00001000
valid..1 ros....0 holes..0 io.....0 seg....0 wimg...2
(0)> dcal 63046 print hexa value of PID
Value decimal: 63046          Value hexa: 0000F646
(0)> tpid 0000F646 show threads of this PID
           SLOT NAME     STATE   TID PRI CPUID CPU FLAGS    WCHAN

thread+025440  795 lsfs     SLEEP 31B31 03C       000 00000004 057DB5BC 
(0)> sw 795 set current context on this thread
Switch to thread: <thread+025440>
(0)> f show the stack
thread+025440 STACK:
[000205C0]e_block_thread+000250 ()
[00020B1C]e_sleep_thread+000040 (??, ??, ??)
[0002AAA0]iowait+00004C (??)
[0002B40C]bread+0000DC (??, ??)
[0020AF4C]readblk+0000AC (??, ??, ??, ??)
[001E90D8]spec_rdwr+00007C (??, ??, ??, ??, ??, ??, ??, ??)
[001A6328]vnop_rdwr+000070 (??, ??, ??, ??, ??, ??, ??, ??)
[00198278]rwuio+0000CC (??, ??, ??, ??, ??, ??, ??, ??)
[001986AC]rdwr+000184 (??, ??, ??, ??, ??, ??)
[001984D4]kreadv+000064 (??, ??, ??, ??)
[000037D8].sys_call+000000 ()
[D0046A18]read+000028 (??, ??, ??)
[1000A0E4]get_superblk+000054 (??, ??, ??)
[100035F8]read_super+000024 (??, ??, ??, ??)
[10005C00]getfssize+0000A0 (??, ??, ??)
[10002D18]prnt_stanza+0001E8 (??, ??, ??)
[1000349C]do_ls+000294 (??, ??)
[10000524]main+0001E8 (??, ??)
[1000014C].__start+00004C ()
(0)> sw u enable user context of the thread
(0)> dc 10005C00-a0 8 look for arguments R3, R4, R5
10005B60    mflr    r0
10005B64     stw    r31,FFFFFFFC(stkp)
10005B68     stw    r0,8(stkp)
10005B6C    stwu    stkp,FFFFFEE0(stkp)
10005B70     stw    r3,108(stkp)
10005B74     stw    r4,104(stkp)
10005B78     stw    r5,10C(stkp)
10005B7C    addi    r3,r4,0
(0)> set 9 print stack frame
display_stack_frames is true
(0)> f show the stack
thread+025440 STACK:
[000205C0]e_block_thread+000250 ()
...
[100035F8]read_super+000024 (??, ??, ??, ??)
=======================================================================
2FF225D0: 2FF2 26F0  2A20 2429  1000 5C04  F071 71C0   /.&.* $)..\..qq.
2FF225E0: 2FF2 2620  2000 4D74  D000 4E18  F071 F83C   /.&  .Mt..N..q.<
2FF225F0: F075 2FF8  F074 36A4  F075 0FE0  F075 1FF8   .u/..t6..u...u..
2FF22600: F071 AE80  8080 8080  0000 0004  0000 0006   .q..............
=======================================================================
[10005C00]getfssize+0000A0 (??, ??, ??)
...
(0)> dw 2FF225D0+104 print arguments (offset 0x104 0x108 0x10c)
2FF226D4: 2000DCC8 2000DC78 00000000 00000004 
(0)> d 2000DC78 20 print first argument
2000DC78: 2F74 6D70  2F73 7472  6970 655F  6673 2E32   /tmp/stripe_fs.2
2000DC88: 3433 3632  0000 0000  0000 0000  0000 0004   4362............
(0)> d 2000DCC8 20 print second argument
2000DCC8: 2F64 6576  2F73 6C76  3234 3336  3200 0000   /dev/slv24362...
2000DCD8: 0000 0000  0000 0000  0000 0000  0000 0004   ................
(0)> q leave debugger
# 

ctx Subcommand

Note: This subcommand is only available within the kdb command; it is not included in the KDB Kernel Debugger.

The context subcommand is used in user mode to analyse memory dump. By default, KDB shows the current AIX context. But it is possible to elect the current kernel KDB context, and to see more information in stack trace subcommand. For instance, the complete stack of a kernel panic is seen. Cpu number is given in argument. No argument allows to restore initial context.

WARNING: KDB context is available only if the running kernel is booted with KDB.

Example

$ kdb dump unix	dump analysis
Preserving 628325 bytes of symbol table
First symbol sys_resource
Component Names:
 1)  proc
 2)  thrd
 3)  errlg
 4)  bos
 5)  vmm
 6)  bscsi
 7)  scdisk
 8)  lvm
 9)  tty
10)  netstat
11)  lent_dd

PFT:
id....................0007
raddr.....0000000001000000 eaddr.....0000000001000000
size..............00800000 align.............00800000
valid..1 ros....0 holes..0 io.....0 seg....1 wimg...2

PVT:
id....................0008
raddr.....00000000004B8000 eaddr.....00000000004B8000
size..............000FFD60 align.............00001000
valid..1 ros....0 holes..0 io.....0 seg....1 wimg...2
Dump analysis on POWER_PC POWER_604 machine with 8 cpu(s)
Processing symbol table...
.......................done
(0)> stat	machine status
RS6K_SMP_MCA POWER_PC POWER_604 machine with 8 cpu(s)
.......... SYSTEM STATUS
sysname... AIX        nodename.. jumbo32
release... 3          version... 4
machine... 00920312A0 nid....... 920312A0
time of crash: Tue Jul 22 09:46:22 1997
age of system: 1 day, 0 min., 35 sec.
.......... PANIC STRING
assert(v_lookup(sid,pno) == -1)
.......... SYSTEM MESSAGES

AIX Version 4.3
Starting physical processor #1 as logical #1... done.
Starting physical processor #2 as logical #2... done.
Starting physical processor #3 as logical #3... done.
Starting physical processor #4 as logical #4... done.
Starting physical processor #5 as logical #5... done.
Starting physical processor #6 as logical #6... done.
Starting physical processor #7 as logical #7... done.
[v_lists.c #727]
<- end_of_buffer
(0)> ctx 0	KDB context of CPU 0
Switch to KDB context of cpu 0
(0)> dr iar	current instruction
iar   : 00009414
.unlock_enable+000110      lwz    r0,8(stkp)          r0=0,8(stkp)=mststack+00AD18
(0)> ctx 1	KDB context of CPU 1
Switch to KDB context of cpu 1
(1)> dr iar	current instruction
iar   : 000BDB68
.kunlockl+000118      blr                        <.ld_usecount+0005BC> r3=0000000B
(1)> ctx 2	KDB context of CPU 2
Switch to KDB context of cpu 2
(2)> dr iar	current instruction
iar   : 00027634
.tstart+000284      blr                        <.sys_timer+000964> r3=00000005
(2)> ctx 3	KDB context of CPU 3
Switch to KDB context of cpu 3
(3)> dr iar	current instruction
iar   : 01B6A580
01B6A580      ori    r3,r31,0            <00000089> r3=50001000,r31=00000089
(3)> ctx 4	KDB context of CPU 4
Switch to KDB context of cpu 4
(4)> dr iar	current instruction
iar   : 00014BFC
.panic_trap+000004       bl    <.panic_dump>       r3=_$STATIC+000294
(4)> f	current stack
__kdb_thread+0002F0 STACK:
[00014BFC].panic_trap+000004 ()
[0003ACAC]v_inspft+000104 (??, ??, ??)
[00048DA8]v_inherit+0004A0 (??, ??, ??)
[000A7ECC]v_preinherit+000058 (??, ??, ??)
[00027BFC]begbt_603_patch_2+000008 (??, ??)

Machine State Save Area [2FF3B400]
iar   : 00027AEC  msr   : 000010B0  cr    : 22222222  lr    : 00243E58
ctr   : 00000000  xer   : 00000000  mq    : 00000000
r0  : 000A7E74  r1  : 2FF3B220  r2  : 002EBC70  r3  : 00013350  r4  : 00000000
r5  : 00000100  r6  : 00009030  r7  : 2FF3B400  r8  : 00000106  r9  : 00000000
r10 : 00243E58  r11 : 2FF3B400  r12 : 000010B0  r13 : 000C1C80  r14 : 2FF22A88
r15 : 20022DB8  r16 : 20006A98  r17 : 20033128  r18 : 00000000  r19 : 0008AD56
r20 : B02A6038  r21 : 0000006A  r22 : 00000000  r23 : 0000FFFF  r24 : 00000100
r25 : 00003262  r26 : 00000000  r27 : B02B8AEC  r28 : B02A9F70  r29 : 00000001
r30 : 00003350  r31 : 00013350
s0  : 00000000  s1  : 007FFFFF  s2  : 0000864B  s3  : 007FFFFF  s4  : 007FFFFF
s5  : 007FFFFF  s6  : 007FFFFF  s7  : 007FFFFF  s8  : 007FFFFF  s9  : 007FFFFF
s10 : 007FFFFF  s11 : 00001001  s12 : 00002002  s13 : 6001F01F  s14 : 00004004
s15 : 007FFFFF
prev      00000000 kjmpbuf   00000000 stackfix  00000000 intpri    0B
curid     0008AD56 sralloc   E01E0000 ioalloc   00000000 backt     00
flags     00 tid       00000000 excp_type 00000000
fpscr     00000000 fpeu            01 fpinfo          00 fpscrx    00000000
o_iar     00000000 o_toc     00000000 o_arg1    00000000
excbranch 00000000 o_vaddr   00000000 mstext    00000000
Except :
 csr   00000000 dsisr 40000000  bit set: DSISR_PFT
 srval 6000864B dar   2FF22FF8 dsirr 00000106

[00027AEC].backt+000000 (00013350, 00000000 [??])
[00243E54]vms_delete+0004DC (??)
[00256838]shmfreews+0000B0 ()
[000732B4]freeuspace+000010 ()
[00072EAC]kexitx+000688 (??)
(4)> ctx    AIX context of CPU 4
Restore initial context
(4)> f	current stack
thread+031920 STACK:
[00027AEC].backt+000000 (00013350, 00000000 [??])
[00243E54]vms_delete+0004DC (??)
[00256838]shmfreews+0000B0 ()
[000732B4]freeuspace+000010 ()
[00072EAC]kexitx+000688 (??)
(4)> 

cdt Subcommand

Note: This subcommand is only available within the kdb command; it is not included in the KDB Kernel Debugger.

The cdt subcommand is used in user mode to analyse memory dump. Any part of component dump can be printed. Usage is cdt [-d] [<index>] [<entry>].

Example

(0)> cdt
1) CDT head name proc, len 001D80E8, entries 96676
2) CDT head name thrd, len 003ABE4C, entries 192489
3) CDT head name errlg, len 00000054, entries 3
4) CDT head name bos, len 00000040, entries 2
5) CDT head name vmm, len 000003D8, entries 30
6) CDT head name sscsidd, len 0000007C, entries 5
7) CDT head name dptSR, len 00000054, entries 3
8) CDT head name scdisk, len 00000130, entries 14
9) CDT head name lvm, len 00000040, entries 2
10) CDT head name SSAGS, len 000000A4, entries 7
11) CDT head name SSAES, len 00000054, entries 3
12) CDT head name ssagateway, len 0000007C, entries 5
13) CDT head name tty, len 00000068, entries 4
14) CDT head name sio_dd, len 00000054, entries 3
15) CDT head name netstat, len 000000E0, entries 10
16) CDT head name ent2104x, len 00000054, entries 3
17) CDT head name cstokdd, len 0000007C, entries 5
18) CDT head name atm_dd_charm, len 00000040, entries 2
19) CDT head name ssadisk, len 000002AC, entries 33
20) CDT head name SSADS, len 00000040, entries 2
21) CDT head name osi_frame, len 0000002C, entries 1
(0)> cdt 12
12) CDT head name ssagateway, len 0000007C, entries 5
CDT    1 name          HashTbl addr 0000000001A25CF0, len 00000040
CDT    2 name         CfgdAdap addr 0000000001A0E044, len 00000004
CDT    3 name         OpenAdap addr 0000000001A0E048, len 00000004
CDT    4 name         LockWord addr 0000000001A0E04C, len 00000004
CDT    5 name             ssa0 addr 0000000001A2D000, len 00000B88
(0)> cdt -d 12 4
12) CDT head name ssagateway, len 0000007C, entries 5
CDT    4 name         LockWord addr 0000000001A0E04C, len 00000004
01A0E04C: FFFFFFFF                                   ....

[ Next Article | Previous Article | Book Contents | Library Home | Legal | Search ]