Compaq AA-RH99A-TE User Manual

Tru64 UNIX

Kernel Debugging

Part Number: AA-RH99A-TE

July 1999

Product Version: Tru64 UNIX Version 5.0 or higher

This manual explains how to use tools to debug a kernel and analyze a
crash dump of the Tru64 UNIX (formerly DIGITAL UNIX) operating
system. Also, this manual explains how to write extensions to the kernel
debugging tools.

Compaq Computer Corporation
Houston, Texas

© 1999 Compaq Computer Corporation

COMPAQ and the Compaq logo Registered in U.S. Patent and Trademark Office. Alpha and Tru64 are
trademarks of Compaq Information Technologies Group, L.P in the United States and other countries.

Microsoft and Windows are trademarks of Microsoft Corporation in the United States and other countries.
UNIX and The Open Group are trademarks of The Open Group.All other product names mentioned herein
may be trademarks of their respective companies.

Confidential computer software. Valid license from Compaq required for possession, use, or copying.
Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software
Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under
vendor’s standard commercial license.

Compaq shall not be liable for technical or editorial errors or omissions contained herein. The information
in this document is provided “as is” without warranty of any kind and is subject to change without
notice. The warranties for Compaq products are set forth in the express limited warranty statements
accompanying such products. Nothing herein should be construed as constituting an additional warranty.

About This Manual

1 Introduction to Kernel Debugging

1.1

1.2

1.3

1.4

Linking a Kernel Image for Debugging ............................ 1–1

Debugging Kernel Programs . .... . . ................................. 1–3

Debugging the Running Kernel . . .. ..... ............................ 1–3

Analyzing a Crash Dump File .. . .. .. . .............................. 1–5

2 Kernel Debugging Utilities

2.1

2.1.1

2.1.2

2.1.3

2.1.4

2.1.5

2.1.6

2.1.7

2.1.8

2.1.9

2.1.10

2.1.11

2.2

2.2.1

2.2.2

2.2.3

2.2.3.1

2.2.3.2

2.2.3.3

2.2.3.4

2.2.3.5

2.2.3.6

2.2.3.7

2.2.3.8

The dbx Debugger . .. .. . ..............................................

Invoking the dbx Debugger for Kernel Debugging .... . . .... 2–2

Debugging Stripped Images ... . .. .. ............................ 2–3

Specifying the Location of Loadable Modules for Crash

Dumps ... . .........................................................

Examining Memory Contents . .... . ............................ 2–5

Printing the Values of Variables and Data Structures .. . . . 2–6

Displaying a Data Structure Format ... . . .................... 2–6

Debugging Multiple Threads .. . .. .............................. 2–7

Examining the Exception Frame .. ............................ 2–7

Examining the User Program Stack .... . . .. . ................. 2–8

Extracting the Preserved Message Buffer . .. .. ............... 2–10

Debugging on SMP Systems . .. ................................. 2–10

The kdbx Debugger . . ................................................. 2–12

Beginning a kdbx Session .. . .. .. .. .............................. 2–12

The kdbx Debugger Commands .. . ............................. 2–13

Using kdbx Debugger Extensions ............................. 2–15

Displaying the Address Resolution Protocol Table .... . 2–16

Performing Commands on Array Elements . .. . .. .... . . . 2–16

Displaying the Buffer Table . .. ............................ 2–18

Displaying the Callout Table and Absolute Callout

Table .. ......................................................... 2–18

Casting Information Stored in a Specific Address . .... 2–19

Displaying Machine Configuration ... .. . ................. 2–19

Converting the Base of Numbers ... . . .. . ................. 2–20

Displaying CPU Use Statistics ............................ 2–20

Contents

2–2

2–4

Contents iii

2.2.3.9

2.2.3.10

2.2.3.11

2.2.3.12

2.2.3.13

2.2.3.14

2.2.3.15

2.2.3.16

2.2.3.17

2.2.3.18

2.2.3.19

2.2.3.20

2.2.3.21

2.2.3.22

2.2.3.23

2.2.3.24

2.2.3.25

2.2.3.26

2.2.3.27

2.2.3.28

2.2.3.29

2.2.3.30

2.2.3.31

2.2.3.32

2.2.3.33

2.3

2.3.1

2.3.2

2.3.3

2.3.4

2.4

Disassembling Instructions ................................

Displaying Remote Exported Entries .... . ............... 2–21

Displaying the File Table . .... . ............................

Displaying the udb and tcb Tables .. . . ..... ............... 2–22

Performing Commands on Lists ... . .. . .. ................. 2–22

Displaying the lockstats Structures ... . . ................. 2–24

Displaying lockinfo Structures ............................ 2–25

Displaying the Mount Table . .............................. 2–26

Displaying the Namecache Structures ... . ............... 2–27

Displaying Processes’ Open Files . . .... . . ................. 2–27

Converting the Contents of Memory to Symbols .... . . . 2–28

Displaying the Process Control Block for a Thread ... . 2–28

Formatting Command Arguments . .... . . .. ............... 2–28

Displaying the Process Table . . ............................ 2–29

Converting an Address to a Procedure name .... . .. .. . . 2–30

Displaying Sockets from the File Table .. . ............... 2–30

Displaying a Summary of the System Information .... 2–30

Displaying a Summary of Swap Space ... . ............... 2–31

Displaying the Task Table ................................. 2–31

Displaying Information About Threads .................. 2–32

Displaying a Stack Trace of Threads .. . . ................. 2–32

Displaying a u Structure ... . . .............................. 2–33

Displaying References to the ucred Structure .. . .... . . . 2–34

Removing Aliases ............................................

Displaying the vnode Table .. . ............................. 2–36

The kdebug Debugger ... . ............................................

Getting Ready to Use the kdebug Debugger ................. 2–39

Invoking the kdebug Debugger ... . ............................ 2–41

Diagnosing kdebug Setup Problems . . .. .. . .... . ............... 2–42

Notes on Using the kdebug Debugger . .. . .... . ............... 2–44

The crashdc Utility .. .. . .............................................. 2–44

2–21

2–21

2–36

2–37

3 Writing Extensions to the kdbx Debugger

3.1

3.2

3.2.1

3.2.2

3.2.3

3.2.4

3.2.5

3.2.6

iv Contents

Basic Considerations for Writing Extensions .. .. ................. 3–1

Standard kdbx Library Functions .. .. .............................. 3–2

Special kdbx Extension Data Types .... . .. .... . ............... 3–2

Converting an Address to a Procedure Name ............... 3–3

Getting a Representation of an Array Element ... . .. .... . . . 3–4

Retrieving an Array Element Value ..... . .. .. ................. 3–4

Returning the Size of an Array .. . ............................. 3–6

Casting a Pointer to a Data Structure .. . . .. . ................. 3–6

3.2.7

3.2.8

3.2.9

3.2.10

3.2.11

3.2.12

3.2.13

3.2.14

3.2.15

3.2.16

3.2.17

3.2.18

3.2.19

3.2.20

3.2.21

3.2.22

3.2.23

3.2.24

3.2.25

3.2.26

3.2.27

3.2.28

3.2.29

3.2.30

3.2.31

3.3

3.4

3.5

Checking Arguments Passed to an Extension ............... 3–7

Checking the Fields in a Structure ............................

Setting the kdbx Context ... . .. .... . ............................

Passing Commands to the dbx Debugger . . .. ................. 3–9

Dereferencing a Pointer .. . .. .. . .... . ............................

Displaying the Error Messages Stored in Fields .. . .. ...... .

Converting a Long Address to a String Address ... . . .... . . . 3–10

Freeing Memory .. . . .. ............................................

Passing Commands to the kdbx Debugger ... . ............... 3–11

Getting the Address of an Item in a Linked List .. . .. ...... . 3–13

Passing an Extension to kdbx . . .. .. ............................ 3–14

Getting the Next Token as an Integer .. . . .... . ............... 3–14

Getting the Next Token as a String .. .. .. . .... . ............... 3–15

Displaying a Message . . .. ...... . .. .............................. 3–16

Displaying Status Messages .. . .. .. ............................. 3–16

Exiting from an Extension .. .... . . .. ............................ 3–17

Reading the Values in Structure Fields . . ..... ............... 3–17

Returning a Line of kdbx Output .............................. 3–18

Reading an Area of Memory ... . .. .. ............................ 3–18

Reading the Response to a kdbx Command .. . ............... 3–19

Reading Symbol Representations . . ............................ 3–20

Reading a Symbol’s Address . .. ................................. 3–20

Reading the Value of a Symbol .. . .............................. 3–21

Getting the Address of a Data Representation . . .. ..... . .. . . 3–21

Converting a String to a Number . ............................. 3–22

Examples of kdbx Extensions . .. . .. ..... ............................ 3–22

Compiling Custom Extensions ... . . ..... ............................ 3–35

Debugging Custom Extensions .. .. .... . ............................ 3–36

3–7
3–8

3–9

3–10

3–11

4 Crash Analysis Examples

4.1

4.2

4.2.1

4.2.2

4.3

4.3.1

4.3.2

4.4

4.5

Guidelines for Examining Crash Dump Files ..... ............... 4–1

Identifying a Crash Caused by a Software Problem .... . . .. . .... 4–2

Using dbx to Determine the Cause of a Software Panic . . . 4–2

Using kdbx to Determine the Cause of a Software Panic . . 4–3

Identifying a Hardware Exception . . . .............................. 4–4

Using dbx to Determine the Cause of a Hardware Error . . 4–4

Using kdbx to Determine the Cause of a Hardware Error 4–7
Finding a Panic String in a Thread Other Than the Current

Thread .. . . .. . ........................................................... 4–8

Identifying the Cause of a Crash on an SMP System . ...... .. . . 4–9

Contents v

A Output from the crashdc Command

Index

Examples

3–1
3–2
3–3
3–4
3–5

Template Extension Using Lists .. . ................................. 3–23

Extension That Uses Linked Lists: callout.c . . .. ................. 3–24

Template Extensions Using Arrays . . .............................. 3–27

Extension That Uses Arrays: file.c . .. .. ............................ 3–28

Extension That Uses Global Symbols: sum.c . .... . ............... 3–34

Figures

2–1

Using a Gateway System During Remote Debugging .... . .. .. . . 2–38

Tables

2–1

The dbx Address Modes ..............................................

2–5

vi Contents

This manual provides information on the tools used to debug a kernel and
analyze a crash dump file of the Tru64™ UNIX (formerly DIGITAL UNIX)
operating system. It also explains how to write extensions to the kernel
debugging tools. You can use extensions to display customized information
from kernel data structures or a crash dump file.

Audience

This manual is intended for system programmers who write programs that
use kernel data structures and are built into the kernel. It is also intended
for system administrators who are responsible for managing the operating
system. System programmers and administrators should have in-depth
knowledge of operating system concepts, commands, and utilities.

New and Changed Features

The following list describes changes that have been made to this manual
for Tru64 UNIX Version 5.0:

• The former Chapter 4, Managing Crash Dumps, has been deleted and
its contents have been moved to the System Administration manual.
All information on that subject is now in one manual. The System
Administration manual was chosen because many aspects of managing
crash dumps (such as storage considerations and default settings) are
handled by a system administrator, often during system installation.

• Crash dumps are now compressed by default and are stored in
compressed crash dump files. These are named
differentiate them from the uncompressed vmcore.n files. Starting with
Version 5.0, all the Tru64 UNIX debugging tools can read vmzcore.n as
well as vmcore.n files. Examples throughout this manual have been
updated to show use of vmzcore.n files.

• When debugging a crash dump with dbx or kdbx, you can examine the
call stack of the user program whose execution precipitated the kernel
crash. For more information, see Section 2.1.9.

• If a loadable kernel module was moved to another location after a kernel
crash, you can specify the directory path where dbx should look for the
module. For more information, see Section 2.1.3.

About This Manual

vmzcore.n to

About This Manual vii

Organization

This manual consists of four chapters and one appendix:

Chapter 1 Introduces the concepts of kernel debugging and

Chapter 2

Chapter 3 Describes how to write a

Chapter 4 Provides background information useful for and examples

Appendix A

crash dump analysis.

Describes the tools used to debug kernels and
analyze crash dump files.

chapter assumes you have purchased and installed a Tru64
UNIX Source Kit and so have access to source files.

of analyzing crash dump files.

Contains example output from the crashdc utility.

Related Documents

For additional information, refer to the following manuals:

• The Alpha Architecture Reference Manual describes how the operating
system interfaces with the Alpha hardware.

• The Alpha Architecture Handbook gives an overview of the Alpha
hardware architecture and describes the 64-bit Alpha RISC (Reduced
Instruction Set Computing) instruction set.

• The Installation Guide and Installation Guide — Advanced Topics
describe how to install your operating system.

• The System Administration manual provides information on managing
and monitoring your system, including managing crash dumps.

kdbx debugger extension. This

• The Programmer’s Guide provides information on the tools, specifically

dbx debugger, for programming on the Tru64 UNIX operating

the
system. This manual also provides information about creating
configurable kernel subsystems.

• The Writing Kernel Modules manual discusses how to code kernel
modules (single binary images) that can be statically loaded as part of
the /vmunix kernel or dynamically loaded into memory, that enhance
the functionality of the Unix kernel.

Icons on Tru64 UNIX Printed Manuals

The printed version of the Tru64 UNIX documentation uses letter icons on
the spines of the manuals to help specific audiences quickly find the manuals
that meet their needs. (You can order the printed documentation from
Compaq.) The following list describes this convention:

viii About This Manual

G Manuals for general users

S Manuals for system and network administrators

P Manuals for programmers

R Manuals for reference page users

Some manuals in the documentation help meet the needs of several
audiences. For example, the information in some system manuals is also
used by programmers. Keep this in mind when searching for information
on specific topics.

The Documentation Overview provides information on all of the manuals in
the Tru64 UNIX documentation set.

Reader’s Comments

Compaq welcomes any comments and suggestions you have on this and
other Tru64 UNIX manuals.

You can send your comments in the following ways:

• Fax: [phone] Attn: UBPG Publications, ZKO3-3/Y32

• Internet electronic mail: [e-mail]

A Reader’s Comment form is located on your system in the following
location:

/usr/doc/readers_comment.txt

Please include the following information along with your comments:

• The full title of the manual and the order number. (The order number
appears on the title page of printed and PDF versions of a manual.)

• The section numbers and page numbers of the information on which
you are commenting.

• The version of Tru64 UNIX that you are using.

• If known, the type of processor that is running the Tru64 UNIX software.

The Tru64 UNIX Publications group cannot respond to system problems
or technical support inquiries. Please address technical questions to your
local system vendor or to the appropriate Compaq technical support office.
Information provided with the software media explains how to send problem
reports to Compaq.

Conventions

The following conventions are used in this manual:

About This Manual ix

%
$

A percent sign represents the C shell system prompt.
A dollar sign represents the system prompt for the
Bourne, Korn, and POSIX shells.

#

% cat

A number sign represents the superuser prompt.

Boldface type in interactive examples indicates
typed user input.

file

Italic (slanted) type indicates variable values,
placeholders, and function argument names.

[|]

{ | } In syntax definitions, brackets indicate items that

are optional and braces indicate items that are
required. Vertical bars separating items inside
brackets or braces indicate that you choose one item
from among those listed.

.
.
.

A vertical ellipsis indicates that a portion of an
example that would normally be present is not
shown.

cat

(1)

A cross-reference to a reference page includes
the appropriate section number in parentheses.
For example, cat

(1) indicates that you can find

information on the cat command in Section 1 of
the reference pages.

Ctrl/x This symbol indicates that you hold down the

x About This Manual

first named key while pressing the key or mouse
button that follows the slash. In examples, this
key combination is enclosed in a box (for example,
Ctrl/C ).

1

Introduction to Kernel Debugging

Kernel debugging is a task normally performed by systems engineers writing
kernel programs. A kernel program is one that is built as part of the kernel
and that references kernel data structures. System administrators might
also debug the kernel in the following situations:

• A process is hung or stops running unexpectedly

• The need arises to examine, and possibly modify, kernel parameters

• The system itself hangs, panics, or crashes

This manual describes how to debug kernel programs and the kernel. It also
includes information about analyzing crash dump files.

In addition to the information provided here, tracing a kernel problem can
require a basic understanding of one or more of the following technical areas:

• The hardware architecture

See the Alpha Architecture Handbook for an overview of the Alpha
hardware architecture and a description of the 64-bit Alpha RISC
instruction set.

• The internal design of the operating system at a source code and data
structure level

See the Alpha Architecture Reference Manual for information on how the
Tru64 UNIX operating system interfaces with the hardware.

This chapter provides an overview of the following topics:

• Linking a kernel image prior to debugging for systems that are running
a kernel built at boot time. (Section 1.1)

• Debugging kernel programs (Section 1.2)

• Debugging the running kernel (Section 1.3)

• Analyzing a crash dump file(Section 1.4)

1.1 Linking a Kernel Image for Debugging

By default, the kernel is a statically linked image that resides in the file
/vmunix. However, your system might be configured so that it is linked
at bootstrap time. Rather than being a bootable image, the boot file is a

Introduction to Kernel Debugging 1–1

text file that describes the hardware and software that will be present on
the running system. Using this information, the bootstrap linker links the
modules that are needed to support this hardware and software. The linker
builds the kernel directly into memory.

You cannot directly debug a bootstrap-linked kernel because you must supply
the name of an image to the kernel debugging tools. Without the image, the
tools have no access to symbol names, variable names, and so on. Therefore,
the first step in any kernel debugging effort is to determine whether your
kernel was linked at bootstrap time. If the kernel was linked at bootstrap
time, you must then build a kernel image file to use for debugging purposes.

The best way to determine whether your system is bootstrap linked or
statically linked is to use the file command to test the type of file from
which your system was booted. If your system is a bootstrap-linked system,
it was booted from an ASCII text file; otherwise, it was booted from an
executable image file. For example, issue the following command to
determine the type of file from which your system was booted:

#/usr/bin/file ‘/usr/sbin/sizer -b‘
/etc/sysconfigtab: ascii text

The sizer -b command returns the name of the file from which the system
was booted. This file name is input to the file command, which determines
that the system was booted from an ASCII text file. The output shown in the
preceeding example indicates that the system is a bootstrap-linked system.
If the system had been booted from an executable image file named vmunix,
the output from the file command would have appeared as follows:

vmunix:COFF format alpha executable or object module

not stripped

If your system is running a bootstrap-linked kernel, build a kernel image
that is identical to the bootstrap-linked kernel your system is running, by
entering the following command:

# /usr/bin/ld -o vmunix.image ‘/usr/sbin/sizer -m‘

The output from the sizer -m command is a list of the exact modules and
linker flags used to build the currently running bootstrap-linked kernel.
This output causes the ld command to create a kernel image that is identical
to the bootstrap-linked kernel running on your system. The kernel image is
written to the file named by the -o flag, in this case the vmunix.image file.

Once you create this image, you can debug the kernel as described in this
manual, using the dbx, kdbx, and kdebug debuggers. When you invoke
the dbx or kdbx debugger, remember to specify the name of the kernel
image file you created with the ld command, such as the vmunix.image
file shown here.

When you are finished debugging the kernel, you can remove the kernel
image file you created for debugging purposes.

1–2 Introduction to Kernel Debugging

1.2 Debugging Kernel Programs

Kernel programs can be difficult to debug because you normally cannot
control kernel execution. To make debugging kernel programs more
convenient, the system provides the
debugger is code that resides inside the kernel and allows you to use the dbx
debugger to control execution of a running kernel in the same manner as
you control execution of a user space program. To debug a kernel program
in this manner, follow these steps:

1. Build your kernel program into the kernel on a test system.

2. Set up the kdebug debugger, as described in Section 2.3.

3. Issue the dbx -remote command on a remote build system, supplying

the pathname of the kernel running on the test system.

4. Set breakpoints and enter dbx commands as you normally would.

Section 2.1 describes some of the commands that are useful during
kernel debugging. For general information about using dbx, see the
Programmer’s Guide.

The system also provides the kdbx debugger, which is designed especially
for debugging kernel code. This debugger contains a number of special
commands, called extensions, that allow you to display kernel data
structures in a readable format. Section 2.2 describes using kdbx and its
extensions. (You cannot use the kdbx debugger with the kdebug debugger.)

kdebug debugger. The kdebug

Another feature of kdbx is that you can customize it by writing your own
extensions. The system contains a set of kdbx library routines that you can
use to create extensions that display kernel data structures in ways that are
meaningful to you. Chapter 3 describes writing kdbx extensions.

1.3 Debugging the Running Kernel

When you have problems with a process or set of processes, you can attempt
to identify the problem by debugging the running kernel. You might also
invoke the debugger on the running kernel to examine the values assigned
to system parameters. (You can modify the value of the parameters using
the debugger, but this practice can cause problems with the kernel and
should be avoided.)

You use the dbx or kdbx debugger to examine the state of processes running
on your system and to examine the value of system parameters. The kdbx
debugger provides special commands, called extensions, that you can use to
display kernel data structures. (Section 2.2.3 describes the extensions.)

To examine the state of processes, you invoke the debugger (as described in
Section 2.1 or Section 2.2) using the following command:

Introduction to Kernel Debugging 1–3

# dbx -k /vmunix /dev/mem

This command invokes dbx with the kernel debugging flag, −k, which
maps kernel addresses to make kernel debugging easier. The /vmunix and
/dev/mem parameters cause the debugger to operate on the running kernel.

Once in the dbx environment, you use dbx commands to display process IDs
and trace execution of processes. You can perform the same tasks using the
kdbx debugger. The following example shows the dbx command you use to
display process IDs:

(dbx) kps

PID COMM
00000 kernel idle
00001 init
00014 kloadsrv
00016 update

.
.
.

If you want to trace the execution of the kloadsrv daemon, use the dbx
command to set the $pid symbol to the process ID of the kloadsrv daemon.
Then, enter the t command:

(dbx) set $pid = 14
(dbx) t
> 0 thread_block() ["/usr/sde/build/src/kernel/kern/sched_prim.c":1623, 0xfffffc0000\
43d77c]

1 mpsleep(0xffffffff92586f00, 0x11a, 0xfffffc0000279cf4, 0x0, 0x0) ["/usr/sde/build\

/src/kernel/bsd/kern_synch.c":411, 0xfffffc000040adc0]

2 sosleep(0xffffffff92586f00, 0x1, 0xfffffc000000011a, 0x0, 0xffffffff81274210) ["/usr/sde\

/build/src/kernel/bsd/uipc_socket2.c":654, 0xfffffc0000254ff8]

3 sosbwait(0xffffffff92586f60, 0xffffffff92586f00, 0x0, 0xffffffff92586f00, 0x10180) ["/usr\

/sde/build/src/kernel/bsd/uipc_socket2.c":630, 0xfffffc0000254f64]

4 soreceive(0x0, 0xffffffff9a64f658, 0xffffffff9a64f680, 0x8000004300000000, 0x0) ["/usr/sde\

/build/src/kernel/bsd/uipc_socket.c":1297, 0xfffffc0000253338]

5 recvit(0xfffffc0000456fe8, 0xffffffff9a64f718, 0x14000c6d8, 0xffffffff9a64f8b8,\
0xfffffc000043d724) ["/usr/sde/build/src/kernel/bsd/uipc_syscalls.c":1002,\
0xfffffc00002574f0]

6 recvfrom(0xffffffff81274210, 0xffffffff9a64f8c8, 0xffffffff9a64f8b8, 0xffffffff9a64f8c8,\
0xfffffc0000457570) ["/usr/sde/build/src/kernel/bsd/uipc_syscalls.c":860,\
0xfffffc000025712c]

7 orecvfrom(0xffffffff9a64f8b8, 0xffffffff9a64f8c8, 0xfffffc0000457570, 0x1, 0xfffffc0000456fe8)\
["/usr/sde/build/src/kernel/bsd/uipc_syscalls.c":825, 0xfffffc000025708c]

8 syscall(0x120024078, 0xffffffffffffffff, 0xffffffffffffffff, 0x21, 0x7d) ["/usr/sde\

/build/src/kernel/arch/alpha/syscall_trap.c":515, 0xfffffc0000456fe4

9 _Xsyscall(0x8, 0x12001acb8, 0x14000eed0, 0x4, 0x1400109d0) ["/usr/sde/build\

/src/kernel/arch/alpha/locore.s":1046, 0xfffffc00004486e4]
(dbx) exit

Often, looking at the trace of a process that is hanging or has unexpectedly
stopped running reveals the problem. Once you find the problem, you can
modify system parameters, restart daemons, or take other corrective actions.

For more information about the commands you can use to debug the running
kernel, see Section 2.1 and Section 2.2.

1–4 Introduction to Kernel Debugging

1.4 Analyzing a Crash Dump File

If your system crashes, you can often find the cause of the crash by using
dbx or kdbx to debug or analyze a crash dump file.

The operating system can crash because one of the following occurs:

• Hardware exception

• Software panic

• Hung system

When a system hangs, it is often necessary to force the system to create
dumps that you can analyze to determine why the system hung. The
System Administration manual describes the procedure for forcing a
crash dump of a hung system.

• Resource exhaustion

The system crashes or hangs because it cannot continue executing. Normally,
even in the case of a hardware exception, the operating system detects
the problem. (For example a machine-checking routine might discover a
hardware problem and begin the process of crashing the system.) In general,
the operating system performs the following steps when it detects a problem
from which it cannot recover:

1. It calls the system panic function.

The panic function saves the contents of registers and sends the panic
string (a message describing the reason for the system panic) to the
error logger and the console terminal.

If the system is a Symmetric Multiprocessing (SMP) system, the panic
function notifies the other CPUs in the system that a panic has
occurred. The other CPUs then also execute the panic function and
record the following panic string:

cpu_ip_intr: panic request

Once each CPU has recorded the system panic, execution continues only
on the master CPU. All other CPUs in the SMP system stop execution.

2. It calls the system boot function.

The boot function records the stack.

3. It calls the dump function.

The dump function copies core memory into swap partitions and the
system stops running or the reboot process begins. Console environment
variables control whether the system reboots automatically. (The
System Administration manual describes these environment variables.)

Introduction to Kernel Debugging 1–5

At system reboot time, the copy of core memory saved in the swap partitions
is copied into a file, called a crash dump file. You can analyze the crash
dump file to determine what caused the crash. By default, the crash dump is
a partial (rather than full) dump and is in compressed form. For complete
information about managing crash dumps and crash dump files, including
how to change default settings, see the

System Administration manual. For

examples of analyzing crash dump files, see Chapter 4.

1–6 Introduction to Kernel Debugging

2

Kernel Debugging Utilities

The Tru64 UNIX system provides several tools you can use to debug the
kernel and kernel programs. The Ladebug debugger (available as an option)
is also capable of debugging the kernel.

This chapter describes three debuggers and a utility for analyzing crash
dumps:

• The dbx debugger, which is described for kernel debugging in Section 2.1.
(For general dbx user information, see the Programmer’s Guide.)

You can use the dbx debugger to display the values of kernel variables
and kernel structures. However, you must understand the structures
and be prepared to follow the address links to find the information you
need. You cannot use dbx alone to control execution of the running
kernel, for example by setting breakpoints.

• The kdbx debugger, which is described in Section 2.2.

The kdbx debugger is an interface to dbx that is tailored specifically
to debugging kernel code. The kdbx debugger has knowledge of the
structure of kernel data and so displays kernel data in a readable format.
Also, kdbx is extensible, allowing you to create commands that are
tailored to your kernel-debugging needs. (Chapter 3 describes how to
tailor the kdbx debugger.) However, you cannot use dbx command line
editing features when you use the kdbx debugger.

• The kdebug debugger, which is described in Section 2.3.

The kdebug debugger is a kernel-debugging program that resides
inside the kernel. Working with a remote version of the dbx debugger,
the kdebug debugger allows you to set breakpoints in and control the
execution of kernel programs and the kernel.

• The crashdc utility, which is described in Section 2.4.

The crashdc utility is a crash dump analysis tool. This utility is useful
when you need to determine why the system is hanging or crashing.

The sections that follow describe how to use these tools to debug the kernel
and kernel programs.

Kernel Debugging Utilities 2–1

______________________ Note _______________________

Starting with Tru64 UNIX Version 5.0, all the previously
mentioned tools can be used with compressed (vmzcore.n) and
uncompressed (vmcore.n) crash dump files. Older versions of
these tools can read only vmcore.n files. If you are using an
older version of a tool, use the expand_dump utility to produce
a vmcore.n file from a vmzcore.n file. For more information
about compressed and uncompressed crash dump files, see

expand_dump

(8) and the System Administration manual.

2.1 The dbx Debugger

The dbx debugger is a symbolic debugger that allows you to examine,
modify, and display the variables and data structures found in stripped or
nonstripped kernel images.

The following sections describe how to invoke the dbx debugger for kernel
debugging (Section 2.1.1) and how to use its commands to perform tasks
such as the following:

• Debugging stripped images (Section 2.1.2)

• Specifying the location of loadable modules for crash dumps
(Section 2.1.3)

• Examining memory contents (Section 2.1.4)

• Displaying the values of kernel variables, and the value and format of
kernel data structures (Section 2.1.5)

• Displaying the format of a data structure (Section 2.1.6)

• Debugging multiple threads (Section 2.1.7)

• Examining the exception frame (Section 2.1.8)

• Examining the user program stack (Section 2.1.9)

• Extracting the preserved message buffer (Section 2.1.10)

• Debugging on SMP systems (Section 2.1.11)

For more information on dbx, see the Programmer’s Guide.

2.1.1 Invoking the dbx Debugger for Kernel Debugging

To debug kernel code with the dbx debugger, you use the −k flag. This flag
causes dbx to map memory addresses. When you use the dbx −k command,
the debugger operates on two separate files that reflect the current state of
the kernel that you want to examine. These files are as follows:

• The disk version of the executable kernel image

2–2 Kernel Debugging Utilities

• The system core memory image

These files may be files from a running system, such as /vmunix and
/dev/mem, or dump files, such as vmunix.n and vmzcore.n (compressed)
or vmcore.n (uncompressed). By default, crash dump files are created in
the /var/adm/crash directory (see the System Administration manual).

______________________ Note _______________________

You might need to be the superuser (root login) to examine the
running system or crash dump files produced by savecore.
Whether you need to be the superuser depends on the directory
and file protections for the files you attempt to examine with
the dbx debugger.

Use the following dbx command to examine the running system:

# dbx −k /vmunix /dev/mem

Use a dbx command similar to the following to examine a compressed or
uncompressed crash dump file, respectively:

# dbx −k vmunix.1 vmzcore.1
# dbx −k vmunix.1 vmcore.1

The version number (.1, in this example) is determined by the value
contained in the bounds file, which is located in the same directory as the
dump files.

2.1.2 Debugging Stripped Images

By default, the kernel is compiled with a debugging flag that does not strip
all of the symbol table information from the executable kernel image. The
kernel is also partially optimized during the compilation process by default.
If the kernel or any other file is fully optimized and stripped of all symbol
table information during compilation, your ability to debug the file is greatly
reduced. However, the dbx debugger provides commands to aid you in
debugging stripped images.

When you attempt to display the contents of a symbol during a debugging
session, you might encounter messages such as the following:

No local symbols.
Undefined symbol.
Inactive symbol.

These messages might indicate that you are debugging a stripped image.

To see the contents of all symbols during a debugging session, you can leave
the debugging session, rebuild all stripped modules (but do not strip them),
and reenter the debugging session. However, on certain occasions, you might

Kernel Debugging Utilities 2–3

want to add a symbol table to your current debugging session rather than
end the session and start a new one. To add a symbol table to your current
debugging session, follow these steps:

1. Go to a window other than the one in which the debugger is running,

or put the debugger in the background, and rebuild the modules for
which you need a symbol table.

2. Once the modules build correctly, use the

ostrip command to strip a

symbol table out of the resulting executable file. For example, if your
executable file is named kernel_program, issue a command such as
the following one:

% /usr/ucb/ostrip -t kernel_program

The -t flag causes the ostrip command to produce two files. One,
named kernel_program, is the stripped executable image. The other,
named kernel_program.stb, contains the symbol table information
for the kernel_program module. (For more information about the
ostrip command, see ostrip(1).)

3. Return to the debugging session and add the symbol table file by issuing

the dbx command stbadd as follows:

dbx> stbadd kernel_program.stb

You can specify an absolute or relative pathname on the stbadd
command line.

Once you issue this command, you can display the contents of symbols
included in the symbol table just as if you had built the module you
are debugging without stripping.

You can also delete symbol tables from a debugging session using the dbx
command stbdel. For more information about this command, see dbx(1).

2.1.3 Specifying the Location of Loadable Modules for Crash Dumps

When a crash dump occurs, the location of any loadable modules used
by the kernel is recorded in the crash dump file, enabling dbx to find the
modules. If the version of a loadable module that was running when the
crash occurred is moved to a different location, dbx will not find it. You can
specify the directory path where dbx should look for loadable modules by
using any one of the following methods (see dbx

• On the dbx command line, specify the directory path with the

-module_path option. For example:

# dbx -k vmunix.1 vmzcore.1 -module_path /project4/mod_dir

• Before invoking dbx, set the environment variable DBX_MODULE_PATH.
For example:

# setenv DBX_MODULE_PATH /project4/mod_dir

2–4 Kernel Debugging Utilities

(1) for complete details):

• During the dbx session, if you want to load a module dynamically, first
set the $module_path dbx variable and then use the addobj command
to load the module, as in the following example:

(dbx) set $module_path /project4/mod_dir
(dbx) addobj kmodC

To verify that modules are being loaded from the correct location, turn on
verbose module-loading using any one of the following methods:

• Specify the -module_verbose dbx command option.

• Set the DBX_MODULE_VERBOSE environment variable to any integer
value.

• Set the $module_verbose dbx variable to a nonzero value.

2.1.4 Examining Memory Contents

To examine memory contents with dbx, use the following syntax:

address/count[mode]

The count argument specifies the number of items that the debugger
displays at the specified address, and the mode argument determines how
dbx displays memory. If you omit the mode argument, the debugger uses
the previous mode. The initial default mode is X (hexadecimal). Table 2–1
lists the dbx address modes.

Table 2–1: The dbx Address Modes

Mode Description

b

c

d

D

f

g

i

n

o

O

s

x

X

Displays a byte in octal.

Displays a byte as a character.

Displays a short word in decimal.

Displays a long word in decimal.

Displays a single precision real number.

Displays a double precision real number.

Displays machine instructions.

Displays data in typed format.

Displays a short word in octal.

Displays a long word in octal.

Displays a string of characters that ends in a null.

Displays a short word in hexadecimal.

Displays a long word in hexadecimal.

Kernel Debugging Utilities 2–5

The following examples show how to use dbx to examine kernel images:

(dbx) _realstart/X
fffffc00002a4008: c020000243c4153e
(dbx) _realstart/i
[_realstart:153, 0xfffffc00002a4008] subq sp, 0x20, sp
(dbx) _realstart/10i

[_realstart:153, 0xfffffc00002a4008] subq sp, 0x20, sp
[_realstart:154, 0xfffffc00002a400c] br r1, 0xfffffc00002a4018
[_realstart:156, 0xfffffc00002a4010] call_pal 0x4994e0
[_realstart:157, 0xfffffc00002a4014] bgt r31, 0xfffffc00002a3018
[_realstart:171, 0xfffffc00002a4018] ldq gp, 0(r1)
[_realstart:172, 0xfffffc00002a401c] stq r31, 24(sp)
[_realstart:177, 0xfffffc00002a4020] bis r16, r31, r9
[_realstart:178, 0xfffffc00002a4024] bis r17, r31, r10
[_realstart:179, 0xfffffc00002a4028] bis r18, r31, r11
[_realstart:181, 0xfffffc00002a402c] bis r19, r31, r12

2.1.5 Printing the Values of Variables and Data Structures

You can use the print command to examine values of variables and data
structures. The print command has the following syntax:

print expression

p expression

For example:

(dbx) print utsname
struct {

sysname = "OSF1"
nodename = "system.dec.com"
release = "V5.0"
version = "688.2"
machine = "alpha"

}

Note that dbx has a default alias of p for print:

(dbx) p utsname

2.1.6 Displaying a Data Structure Format

You can use the whatis command to display the format for many of the
kernel data structures. The whatis command has the following syntax:

whatis typename

The following example displays the itimerval data structure:

(dbx) whatis struct itimerval
struct itimerval {

struct timeval {

int tv_sec;
int tv_usec;

} it_interval;

2–6 Kernel Debugging Utilities

struct timeval {

int tv_sec;
int tv_usec;

} it_value;

};

2.1.7 Debugging Multiple Threads

You can use the dbx debugger to examine the state of the kernel’s threads
with the querying and scoping commands described in this section. You
use these commands to show process and thread lists and to change the
debugger’s context (by setting its current process and thread variables)
so that a stack trace for a particular thread can be displayed. Use these
commands to examine the state of the kernel’s threads:

print $tid

print $pid

where

tlist

kps

set $pid=process_id

tset thread_id

tstack

Display the thread ID of the current
thread

Display the process ID of the current
process

Display a stack trace for the current
thread

Display a list of kernel threads for the
current process

Display a list of processes (not available
when used with kdebug)

Change the context to another process (a
process ID of 0 changes context to the
kernel)

Change the context to another thread

Displays the stack trace for all threads.

2.1.8 Examining the Exception Frame

When you work with a crash dump file to debug your code, you can use
dbx to examine the exception frame. The exception frame is a stack frame
created during an exception. It contains the registers that define the state
of the routine that was running at the time of the exception. Refer to the

Kernel Debugging Utilities 2–7

/usr/include/machine/reg.h header file to determine where registers
are stored in the exception frame.

The savedefp variable contains the location of the exception frame. (Note
that no exception frames are created when you force a system to dump, as
described in the

System Administration manual.) The following example

shows an example exception frame:

(dbx) print savedefp/33X
ffffffff9618d940: 0000000000000000 fffffc000046f888
ffffffff9618d950: ffffffff86329ed0 0000000079cd612f
ffffffff9618d960: 000000000000007d 0000000000000001
ffffffff9618d970: 0000000000000000 fffffc000046f4e0
ffffffff9618d980: 0000000000000000 ffffffff9618a2f8
ffffffff9618d990: 0000000140012b20 0000000000000000
ffffffff9618d9a0: 000000014002ee10 0000000000000000
ffffffff9618d9b0: 00000001400075e8 0000000140026240
ffffffff9618d9c0: ffffffff9618daf0 ffffffff8635af20
ffffffff9618d9d0: ffffffff9618dac0 00000000000001b0
ffffffff9618d9e0: fffffc00004941b8 0000000000000000
ffffffff9618d9f0: 0000000000000001 fffffc000028951c
ffffffff9618da00: 0000000000000000 0000000000000fff
ffffffff9618da10: 0000000140026240 0000000000000000
ffffffff9618da20: 0000000000000000 fffffc000047acd0
ffffffff9618da30: 0000000000901402 0000000000001001
ffffffff9618da40: 0000000000002000

2.1.9 Examining the User Program Stack

When debugging a crash dump with dbx, you can examine the call stack of
the user program whose execution precipitated the kernel crash. To examine
a crash dump and also view the user program stack, you must invoke dbx
using the following command syntax:

dbx -k vmunix.n vm[z]core.n path/user-program

The version number (n) is determined by the value contained in the
bounds file, which is located in the same directory as the dump files. The
user-program parameter specifies the user program executable.

The crash dump file must contain a full crash dump. For information on
setting system defaults for full or partial crash dumps, see the System
Administration manual. You can use the assign command in dbx, as shown
in the following example, to temporarily specify a full crash dump. This
setting stays in effect until the system is rebooted.

# dbx -k vmunix.3
dbx version 5.0

.
.
.

(dbx) assign partial_dump=0

2–8 Kernel Debugging Utilities

To specify a full crash dump permanently so that this setting remains in
effect after a reboot, use the patch command in dbx, as shown in the
following example:

(dbx) patch partial_dump=0

With either command, a partial_dump value of 1 specifies a partial dump.

The following example shows how to examine the state of a user program
named test1 that purposely precipitated a kernel crash with a syscall
after several recursive calls:

# dbx -k vmunix.1 vmzcore.1 /usr/proj7/test1
dbx version 5.0
Type ’help’ for help.

stopped at [boot:1890 ,0xfffffc000041ebe8] Source not available

warning: Files compiled -g3: parameter values probably wrong
(dbx) where
> 0 boot() ["../../../../src/kernel/arch/alpha/machdep.c":1890,
0xfffffc000041ebe8]

1 panic(0xfffffc000051e1e0, 0x8, 0x0, 0x0, 0xffffffff888c3a38)

["../../../../src/kernel/bsd/subr_prf.c":824, 0xfffffc0000281974]

2 syscall(0x2d, 0x1, 0xffffffff888c3ce0, 0x9aa1e00000000, 0x0)

["../../../../src/kernel/arch/alpha/syscall_trap.c":593, 0xfffffc0000423be4]

3 _Xsyscall(0x8, 0x3ff8010f9f8, 0x140008130, 0xaa, 0x3ffc0097b70)

["../../../../src/kernel/arch/alpha/locore.s":1409, 0xfffffc000041b0f4]

4 __syscall(0x0, 0x0, 0x0, 0x0, 0x0) [0x3ff8010f9f4]
5 justtryme(scall = 170, cpu = 0, levels = 25) ["test1.c":14,

0x120001310]

6 recurse(inbox = (...)) ["test1.c":28, 0x1200013c4]
7 recurse(inbox = (...)) ["test1.c":30, 0x120001400]
8 recurse(inbox = (...)) ["test1.c":30, 0x120001400]
9 recurse(inbox = (...)) ["test1.c":30, 0x120001400]
.
.

.
30 recurse(inbox = (...)) ["test1.c":30, 0x120001400]
31 main(argc = 3, argv = 0x11ffffd08) ["test1.c":52, 0x120001518]

(dbx) up 8
recurse: 30 if (r.a[2] > 0) recurse(r);
(dbx) print r
struct {

a={

(dbx) print r.a[511]
25
(dbx)

1

2

3

[0] 170
[1] 0
[2] 2
[3] 0
.
.
.

4

1 The where command displays the kernel stack followed by the user

program stack at the time of the crash. In this case, the kernel stack
has 4 activation levels; the user program stack starts with the fifth level
and includes several recursive calls.

Kernel Debugging Utilities 2–9

2 The up 8 command moves the debugging context 8 activation levels up

the stack to one of the recursive calls within the user program code.

3 The print r command displays the current value of the variable r,

which is a structure of array elements. Full symbolization is available
for the user program, assuming it was compiled with the -g option.

4 The print r.a[511] command displays the current value of array

element 511 of structure r.

2.1.10 Extracting the Preserved Message Buffer

The preserved message buffer (pmsgbuf) contains information such as
the firmware version, operating system version, pc value, and device
configuration. You can use dbx to extract the preserved message buffer from
a running system or dump files. For example:

(dbx) print *pmsgbuf
struct {

msg_magic = 405601
msg_bufx = 1537
msg_bufr = 1537

msg_bufc = "Alpha boot: available memory from 0x7c6000 to 0x6000000
Tru64 UNIX V5.0; Sun Jan 03 11:20:36 EST 1999
physical memory = 96.00 megabytes.
available memory = 84.57 megabytes.
using 360 buffers containing 2.81 megabytes of memory
tc0 at nexus
scc0 at tc0 slot 7
asc0 at tc0 slot 6
rz1 at scsi0 target 1 lun 0 (LID=0) (DEC RZ25 (C) DEC 0700)
rz2 at scsi0 target 2 lun 0 (LID=1) (DEC RZ25 (C) DEC 0700)
rz3 at scsi0 target 3 lun 0 (LID=2) (DEC RZ26 (C) DEC T384)
rz4 at scsi0 target 4 lun 0 (LID=3) (DEC RRD42 (C) DEC 4.5d)
tz5 at scsi0 target 5 lun 0 (DEC TLZ06 (C)DEC 0374)
scsi1 at tc0 slot 7
fb0 at tc0 slot 8

1280X1024
ln0: DEC LANCE Module Name: PMAD-BA
ln0 at tc0 slot 7

.
.
.

2.1.11 Debugging on SMP Systems

Debugging in an SMP environment can be difficult because an SMP system
optimized for performance keeps the minimum of lock debug information.

The Tru64 UNIX system supports a lock mode to facilitate debugging SMP
locking problems. The lock mode is implemented in the lockmode boot
time system attribute. By default, the lockmode attribute is set to a value
between 0 and 3, depending upon whether the system is an SMP system and
whether the RT_PREEMPTION_OPT attribute is set. (This attribute optimizes
system performance.)

2–10 Kernel Debugging Utilities

For debugging purposes, set the lockmode attribute to 4. Follow these steps
to set the lockmode attribute to 4:

1. Create a stanza-formatted file named, for example, generic.stanza

that appears as follows:

generic:

lockmode=4

The contents of this file indicate that you are modifying the lockmode
attribute of the generic subsystem.

2. Add the new definition of lockmode to the /etc/sysconfigtab

database:

sysconfigdb -a -f generic.stanza generic

#

3. Reboot your system.

Some of the debugging features provided with lockmode set to 4 are as
follows:

• Automatic lock hierarchy checking and minimum spl checking when

any kernel lock is acquired (assuming a lockinfo structure exists
for the lock class in question). This checking helps you find potential
deadlock situations.

• Lock initialization checking.

• Additional debug information maintenance, including information about

simple and complex locks.

For simple locks, the system records an array of the last 32 simple locks
which were acquired on the system (slock_debug). The system creates
a slock_debug array for each CPU in the system.

For complex locks, the system records the locks owned by each thread in
the thread structure (up to eight complex locks).

To get a list of the complex locks a thread is holding use these commands:

# dbx -k /vmunix
(dbx) print thread->lock_addr
{

[0] 0xe4000002a67e0030
[1] 0xc3e0005b47ff0411
[2] 0xb67e0030a6130048
[3] 0xa67e0030d34254e5
[4] 0x279f0200481e1617
[5] 0x4ae33738a7730040
[6] 0x477c0101471c0019

[7] 0xb453004047210402
}
(dbx) print slock_debug
{

Kernel Debugging Utilities 2–11

[0] 0xfffffc000065c580

[1] 0xfffffc000065c780
}

• Lock statistics are recorded to allow you to determine what kind of

contention you have on a particular lock. Use the kdbx lockstats
extension as shown in the following example to display lock statistics:

# kdbx /vmunix
(kdbx) lockstats

Lockstats li_name cpu count tries misses %misses waitsum waitmax waitmin trmax
=========== ===================== === ====== ========== ======= ====== ============ ======= ======= ======
k0x00657d40 inode.i_io_lock 1 1784 74268 1936 2.61 110533 500 6 10
k0x00653400 nfs_daemon_lock 0 1 7 0 0.00 0 0 0 0
k0x00657d80 nfs_daemon_lock 1 1 0 0 0.00 0 0 0 0
k0x00653440 lk_lmf 0 1 0 0 0.00 0 0 0 0
k0x00657dc0 lk_lmf 1 1 2 0 0.00 0 0 0 0
k0x00653480 procfs_global_lock 0 1 3 0 0.00 0 0 0 0
k0x00657e00 procfs_global_lock 1 1 5 0 0.00 0 0 0 0
k0x006534c0 procfs.pr_trace_lock 0 40 0 0 0.00 0 0 0 0
k0x00657e40 procfs.pr_trace_lock 1 40 0 0 0.00 0 0 0

2.2 The kdbx Debugger

The kdbx debugger is a crash analysis and kernel debugging tool; it serves
as a front end to the dbx debugger. The kdbx debugger is extensible,
customizable, and insensitive to changes to offsets and field sizes in
structures. The only dependencies on kernel header files are for bit
definitions in flag fields.

The kdbx debugger has facilities for interpreting various symbols and kernel
data structures. It can format and display these symbols and data structures
in the following ways:

• In a predefined form as specified in the source code modules that

currently accompany the kdbx debugger

• As defined in user-written source code modules according to a

standardized format for the contents of the kdbx modules

All dbx commands (except signals such as Ctrl/P) are available when you
use the kdbx debugger. In general, kdbx assumes hexadecimal addresses for
commands that perform input and output.

As with dbx, you can use kdbx to examine the call stack of the user program
whose execution precipitated a kernel crash (see Section 2.1.9).

The sections that follow explain using kdbx to debug kernel programs.

2.2.1 Beginning a kdbx Session

Using the kdbx debugger, you can examine the running kernel or dump files
created by the savecore utility. In either case, you examine an object file
and a core file. For running systems, these files are usually /vmunix and

2–12 Kernel Debugging Utilities

/dev/mem, respectively. By default, crash dump files are created in the
/var/adm/crash directory (see the System Administration manual).

Use the following kdbx command to examine a running system:

# kdbx −k /vmunix /dev/mem

Use a kdbx command similar to the following to examine a compressed or
uncompressed crash dump file, respectively:

# kdbx −k vmunix.1 vmzcore.1
# kdbx −k vmunix.1 vmcore.1

The version number (.1 in this example) is determined by the value contained
in the bounds file, which is located in the same directory as the dump files.

To examine a crash dump file and also view the call stack of the user
program whose execution precipitated the kernel crash, you must invoke
kdbx using the following command syntax:

kdbx -k vmunix.

n vm[z]core.n path/user-program

For more information, see Section 2.1.9.

When you begin a debugging session, kdbx reads and executes the
commands in the system initialization file /var/kdbx/system.kdbxrc.
The initialization file contains setup commands and alias definitions. (For
a list of kdbx aliases, see the kdbx

(1) reference page.) You can further

customize the kdbx environment by adding commands and aliases to:

• The /var/kdbx/site.kdbxrc file

This file contains customized commands and alias definitions for a
particular system.

• The ~/.kdbxrc file

This file contains customized commands and alias definitions for a
specific user.

• The ./.kdbxrc file

This file contains customized commands and alias definitions for a
specific project. This file must reside in the current working directory
when kdbx is invoked.

2.2.2 The kdbx Debugger Commands

The kdbx debugger provides the following commands:
alias [name] [command-string]

Sets or displays aliases. If you omit all arguments, alias displays all
aliases. If you specify the variable name, alias displays the alias for
name, if one exists. If you specify name and command-string, alias
establishes name as an alias for command-string.

Kernel Debugging Utilities 2–13

context proc | user

Sets context to the user’s aliases or the extension’s aliases. This
command is used only by the extensions.

coredata start_address end_address

Dumps, in hexadecimal, the contents of the core file starting at
start_address and ending before end_address.

dbx command-string

Passes the command-string to dbx. Specifying dbx is optional; if
kdbx does not recognize a command, it automatically passes that

command to dbx. See the dbx

(1) reference page for a complete

description of dbx commands.

help [-long] [args]

Prints help text.

pr [flags] [extensions] [arguments]

Executes an extension and gives it control of the kdbx session until it
quits. You specify the name of the extension in extension and pass
arguments to it in arguments.

−debug

−pipe in_pipe

out_pipe

−print_output

−redirect_output

−tty

2–14 Kernel Debugging Utilities

Causes kdbx to display input to and output
from the extension on the screen.

Used in conjunction with the dbx debugger
for debugging extensions. See Chapter 3 for
information on using the −pipe flag.

Causes the output of the extension to be
sent to the invoker of the extension without
interpretation as kdbx commands.

Used by extensions that execute other
extensions to redirect the output from the
called extensions; otherwise, the user receives
the output.

Causes kdbx to communicate with the
subprocess through a terminal line instead
of pipes. If you specify the −pipe flag, proc
ignores it.