[ Next Article | Previous Article | Book Contents | Library Home | Legal | Search ]
Base Operating System and Extensions Technical Reference, Volume 1

ptrace, ptracex Subroutine

Purpose

Traces the execution of another process.

Library

Standard C Library (libc.a)

Syntax

#include <sys/reg.h>
#include <sys/ptrace.h>
#include <sys/ldr.h>

int ptrace (Request, Identifier,Address, Data, Buffer)
int Request;
int Identifier;
int *Address;
int Data;
int *Buffer;
int ptracex (request, identifier, long long addr, data, buff)
int request;
int identifier, long long addr;
int data;
int *buff;

Description

The ptrace subroutine allows a 32-bit process to trace the execution of another process. The ptrace subroutine is used to implement breakpoint debugging.

A debugged process executes normally until it encounters a signal. Then it enters a stopped state and its debugging process is notified with the wait subroutine.

Exception: If the process encounters the SIGTRAP signal, a signal handler for SIGTRAP exists, and fast traps have been enabled for the process, then the signal handler is called and the debugger is not notified. This exception only applies to AIX Version 4.3.3 and later releases.

While the process is in the stopped state, the debugger examines and modifies the memory image of the process being debugged by using the ptrace subroutine. For multi-threaded processes, the getthrds subroutine identifies each kernel thread in the debugged process. Also, the debugging process can cause the debugged process to terminate or continue, with the possibility of ignoring the signal that caused it to stop.

As a security measure, the ptrace subroutine inhibits the set-user-ID facility on subsequent exec subroutines.

(This paragraph only applies to AIX Version 4.3.1 and previous releases.) When a process is executing under ptrace control, portions of the process's address space are recopied after load, unload, and loadbind calls. For a 32-bit process, the main program text (loaded in segment 1) and shared library modules (loaded in segment 13) is recopied. Any breakpoints or other modifications to these segments must be reinserted after load, unload, or loadbind. Changes to privately loaded modules persist. For a 64-bit process, shared library modules are recopied after load and unload are called. (For AIX Version 4.3 and AIX Version 4.3.1, these segments have a virtual address of 0x09000000xxxxxxxx, where x denotes any value.) The segments for the main programs and the segments containing privately loaded modules are not recopied. When a 64-bit process calls loadbind, no segments are recopied and the debugger is not notified.

(This paragraph only applies to AIX Version 4.3.2 and later releases.) When a process executing under ptrace control calls load or unload, the debugger is notified and the W_SLWTED flag is set in the status returned by wait. (A 32-bit process calling loadbind is stopped as well.) If the process being debugged has added modules in the shared library to its address space, the modules are added to the process's private copy of the shared library segments. If shared library modules are removed from a process's address space, the modules are deleted from the process's private copy of the library text segment by freeing the pages that contain the module. No other changes to the segment are made, and existing breakpoints do not have to be reinserted.

To allow a debugger to generate code more easily (in order to handle fast trap instructions, for example), memory from the end of the main program up to the next segment boundary can be modified. That memory is read-only to the process but can be modified by the debugger.

When a process being traced forks, the child process is initialized with the unmodified main program and shared library segment, effectively removing breakpoints in these segments in the child process. If multiprocess debugging is enabled, new copies of the main program and shared library segments are made. Modifications to privately loaded modules, however, are not affected by a fork. These breakpoints will remain in the child process, and if these breakpoints are executed, a SIGTRAP signal will be generated and delivered to the process.

If a traced process initiates an exec subroutine, the process stops before executing the first instruction of the new image and returns the SIGTRAP signal.

Note: ptrace and ptracex are not supported in 64-bit mode.

Fast Trap Instructions

Note: The "Fast Trap Instructions" section only applies to AIX Version 4.3.3 and later releases.

Sometimes, allowing the process being debugged to handle certain trap instructions is useful, instead of causing the process to stop and notify the debugger. You can use this capability to patch running programs or programs whose source codes are not available. For a process to use this capability, you must enable fast traps, which requires you to make a ptrace call from a debugger on behalf of the process.

To let a process handle fast traps, a debugger uses the ptrace (PT_SET, pid, 0, PTFLAG_FAST_TRAP, 0) subroutine call. Cancel this capability with the ptrace (PT_CLEAR, pid, 0, PTFLAG_FAST_TRAP, 0) subroutine call. If a process is able to handle fast traps when the debugger detaches, the fast trap capability remains in effect. Consequently, when another debugger attaches to that process, fast trap processing is still enabled. When no debugger is attached to a process, SIGTRAP signals are handled in the same manner, regardless of whether fast traps are enabled.

A fast trap instruction is an unconditional trap immediate instruction in the form twi 14,r13,0xNXXX. This instruction has the binary form 0x0ddfNXXX, where N is a hex digit >=8 and XXX are any three hex digits. By using different values of 0xNXXX, a debugger can generate different fast trap instructions, allowing a signal handler to quickly determine how to handle the signal. (The fast trap instruction is defined by the macro _PTRACE_FASTTRAP. The _PTRACE_FASTTRAP_MASK macro can be used to check whether a trap is a fast trap.)

Usually, a fast trap instruction is treated like any other trap instruction. However, if a process has a signal handler for SIGTRAP, the signal is not blocked, and the fast trap capability is enabled, then the signal handler is called and the debugger is not notified.

A signal handler can logically AND the trap instruction with _PTRACE_FASTTRAP_NUM (0x7FFF) to obtain an integer identifying which trap instruction was executed.

For the 64-bit Process

Use ptracex where the debuggee is a 64-bit process and the operation requested uses the third (Address) parameter to reference the debuggee's address space or is sensitive to register size.

If returning or passing an int doesn't work for a 64-bit debuggee (for example, PT_READ_GPR), the buffer parameter takes the address for the result. Thus, with the ptracex subroutine, PT_READ_GPR and PT_WRITE_GPR take a pointer to an 8 byte area representing the register value.

In general, ptracex supports all the calls that ptrace does when they are modified for any that are extended for 64-bit addresses (for example, GPRs, LR, CTR, IAR, and MSR). Anything whose size increases for 64-bit processes must be allowed for in the obvious way (for example, PT_REGSET must be an array of long longs for a 64-bit debuggee).

Parameters

Request Determines the action to be taken by the ptrace subroutine and has one of the following values:
PT_ATTACH This request allows a debugging process to attach a current process and place it into trace mode for debugging. This request cannot be used if the target process is already being traced. The Identifier parameter is interpreted as the process ID of the traced process. The Address, Data, and Buffer parameters are ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to one the following codes:

ESRCH Process ID is not valid; the traced process is a kernel process; the process is currently being traced; or, the debugger or traced process already exists.
EPERM Real or effective user ID of the debugger does not match that of the traced process, or the debugger does not have root authority.
EINVAL The debugger and the traced process are the same.
PT_CLEAR This request clears an internal flag or capability. The Data parameter specifies which flags to clear. The following flag can be cleared:
PTFLAG_FAST_TRAP Disables the special handling of a fast trap instruction. This allows all fast trap instructions causing an interrupt to generate a SIGTRAP signal.

The Identifier parameter specifies the process ID of the traced process. The Address parameter, Buffer parameter, and the unused bits in the Data parameter are reserved for future use and should be set to 0.

PT_CONTINUE This request allows the process to resume execution. If the Data parameter is 0, all pending signals, including the one that caused the process to stop, are concealed before the process resumes execution. If the Data parameter is a valid signal number, the process resumes execution as if it had received that signal. If the Address parameter equals 1, the execution continues from where it stopped. If the Address parameter is not 1, it is assumed to be the address at which the process should resume execution. Upon successful completion, the value of the Data parameter is returned to the debugging process. The Identifier parameter is interpreted as the process ID of the traced process. The Buffer parameter is ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO The signal to be sent to the traced process is not a valid signal number.

Note: For the PT_CONTINUE request, use ptracex with a 64-bit debuggee because the resume address needs 64 bits.

PTT_CONTINUE This request asks the scheduler to resume execution of the kernel thread specified by Identifier. This kernel thread must be the one that caused the exception. The Data parameter specifies how to handle signals:
  • If the Data parameter is 0, the kernel thread which caused the exception will be resumed as if the signal never occurred.
  • If the Data parameter is a valid signal number, the kernel thread which caused the exception will be resumed as if it had received that signal.

The Address parameter specifies where to resume execution:

  • If the Address parameter is 1, execution resumes from the address where it stopped.
  • If the Address parameter contains an address value other than 1, execution resumes from that address.

The Buffer parameter should point to a PTTHREADS structure, which contains a list of kernel thread identifiers to be started. This list should be NULL terminated if it is smaller than the maximum allowed.

On successful completion, the value of the Data parameter is returned to the debugging process. On unsuccessful completion, the value -1 is returned, and the errno global variable is set as follows:

EINVAL The Identifier parameter names the wrong kernel thread.
EIO The signal to be sent to the traced kernel thread is not a valid signal number.
ESRCH The Buffer parameter names an invalid kernel thread. Each kernel thread in the list must be stopped and belong to the same process as the kernel thread named by the Identifier parameter.

Note: For the PTT_CONTINUE request, use ptracex with a 64-bit debuggee because the resume address needs 64 bits.

PT_DETACH This request allows a debugged process, specified by the Identifier parameter, to exit trace mode. The process then continues running, as if it had received the signal whose number is contained in the Data parameter. The process is no longer traced and does not process any further ptrace calls. The Address and Buffer parameters are ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO Signal to be sent to the traced process is not a valid signal number.
PT_KILL This request allows the process to terminate the same way it would with an exit subroutine.
PT_LDINFO This request retrieves a description of the object modules that were loaded by the debugged process. The Identifier parameter is interpreted as the process ID of the traced process. The Buffer parameter is ignored. The Address parameter specifies the location where the loader information is copied. The Data parameter specifies the size of this area. The loader information is retrieved as a linked list of ld_info structures. The ld_info structures are defined in the /usr/include/sys/ldr.h file. The linked list is implemented so that the ldinfo_nxt field of each element gives the offset of the next element from this element. The ldinfo_nxt field of the last element has the value 0.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

ENOMEM Either the area is not large enough to accommodate the loader information, or there is not enough memory to allocate an equivalent buffer in the kernel.

Note: For the PT_LDINFO request, use ptracex with a 64-bit debuggee because the source address needs 64 bits.

PT_MULTI This request turns multiprocess debugging mode on and off, to allow debugging to continue across fork and exec subroutines. A 0 value for the Data parameter turns multiprocess debugging mode off, while all other values turn it on. When multiprocess debugging mode is in effect, any fork subroutine allows both the traced process and its newly created process to trap on the next instruction. If a traced process initiated an exec subroutine, the process stops before executing the first instruction of the new image and returns the SIGTRAP signal. The Identifier parameter is interpreted as the process ID of the traced process. The Address and Buffer parameters are ignored.

Also, when multiprocess debugging mode is enabled, the following values are returned from the wait subroutine:

W_SEWTED Process stopped during execution of the exec subroutine.
W_SFWTED Process stopped during execution of the fork subroutine.
PT_READ_BLOCK This request reads a block of data from the debugged process address space. The Address parameter points to the block of data in the process address space, and the Data parameter gives its length in bytes. The value of the Data parameter must not be greater than 1024. The Identifier parameter is interpreted as the process ID of the traced process. The Buffer parameter points to the location in the debugging process address space where the data is copied. Upon successful completion, the ptrace subroutine returns the value of the Data parameter.

If this request is unsuccessful, -1 is returned and the errno global variable is set to one of the following codes:

EIO The Data parameter is less than 1 or greater than 1024.
EIO The Address parameter is not a valid pointer into the debugged process address space.
EFAULT The Buffer parameter does not point to a writable location in the debugging process address space.

Note: For the PT_READ_BLOCK request, use ptracex with a 64-bit debuggee because the source address needs 64 bits.

PT_READ_FPR This request stores the value of a floating-point register into the location pointed to by the Address parameter. The Data parameter specifies the floating-point register, defined in the sys/reg.h file for the machine type on which the process is executed. The Identifier parameter is interpreted as the process ID of the traced process. The Buffer parameter is ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO The Data parameter is not a valid floating-point register. The Data parameter must be in the range 256-287.
PTT_READ_FPRS This request writes the contents of the 32 floating point registers to the area specified by the Address parameter. This area must be at least 256 bytes long. The Identifier parameter specifies the traced kernel thread. The Data and Buffer parameters are ignored.
PT_READ_GPR This request returns the contents of one of the general-purpose or special-purpose registers of the debugged process. The Address parameter specifies the register whose value is returned. The value of the Address parameter is defined in the sys/reg.h file for the machine type on which the process is executed. The Identifier parameter is interpreted as the process ID of the traced process. The Data and Buffer parameters are ignored. The buffer points to long long target area.
Note: If ptracex with a 64-bit debugee is used for this request, the register value is instead returned to the 8-byte area pointed to by the buffer pointer.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO The Address is not a valid general-purpose or special-purpose register. The Address parameter must be in the range 0-31 or 128-136.
PTT_READ_GPRS This request writes the contents of the 32 general purpose registers to the area specified by the Address parameter. This area must be at least 128 bytes long.
Note: If ptracex with a 64-bit debugee is used for the PTT_READ_GPRS request, there must be at least a 256 byte target area. The Identifier parameter specifies the traced kernel thread. The Data and Buffer parameters are ignored.
PT_READ_I or PT_READ_D These requests return the word-aligned address in the debugged process address space specified by the Address parameter. On all machines currently supported by the Version 4 operating system, the PT_READ_I and PT_READ_D instruction and data requests can be used with equal results. The Identifier parameter is interpreted as the process ID of the traced process. The Data parameter is ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO The Address is not word-aligned, or the Address is not valid. User blocks, kernel segments, and kernel extension segments are not considered as valid addresses.

Note: For the PT_READ_I or the PT_READ_D request, use ptracex with a 64-bit debuggee because the source address needs 64 bits.

PTT_READ_SPRS This request writes the contents of the special purpose registers to the area specified by the Address parameter, which points to a ptsprs structure. The Identifier parameter specifies the traced kernel thread. The Data and Buffer parameters are ignored.

Note: For the PTT_READ_SPRS request, use ptracex with the 64-bit debuggee because the new ptxsprs structure must be used.

PT_REATT This request allows a new debugger, with the proper permissions, to trace a process that was already traced by another debugger. The Identifier parameter is interpreted as the process ID of the traced process. The Address, Data, and Buffer parameters are ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to one the following codes:

ESRCH The Identifier is not valid; or the traced process is a kernel process.
EPERM Real or effective user ID of the debugger does not match that of the traced process, or the debugger does not have root authority.
EINVAL The debugger and the traced process are the same.
PT_REGSET This request writes the contents of all 32 general purpose registers to the area specified by the Address parameter. This area must be at least 128 bytes for the 32-bit debuggee or 256 bytes for the 64-bit debuggee. The Identifier parameter is interpreted as the process ID of the traced process. The Data and Buffer parameters are ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO The Address parameter points to a location outside of the allocated address space of the process.

Note: For the PT_REGSET request, use ptracex with the 64-bit debuggee because 64-bit registers requiring 256 bytes are returned.

PT_SET This request sets an internal flag or capability. The Data parameter indicates which flags are set. The following flag can be set:
PTFLAG_FAST_TRAP Enables the special handling of a fast trap instruction. When a fast trap instruction is executed in a process that has a signal handler for SIGTRAP, the signal handler will be called even if the process is being traced.

The Identifier parameter specifies the process ID of the traced process. The Address parameter, Buffer parameter, and the unused bits in the Data parameter are reserved for future use and should be set to 0.

PT_TRACE_ME This request must be issued by the debugged process to be traced. Upon receipt of a signal, this request sets the process trace flag, placing the process in a stopped state, rather than the action specified by the sigaction subroutine. The Identifier, Address, Data, and Buffer parameters are ignored. Do not issue this request if the parent process does not expect to trace the debugged process.

As a security measure, the ptrace subroutine inhibits the set-user-ID facility on subsequent exec subroutines, as shown in the following example:

if((childpid = fork()) == 0)
{ /* child process */
 ptrace(PT_TRACE_ME,0,0,0,0);
   execlp(          )/* your favorite exec*/
   }
else
{    /* parent                          */
     /* wait for child to stop          */
        rc = wait(status) 
Note: This is the only request that should be performed by the child. The parent should perform all other requests when the child is in a stopped state.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

ESRCH Process is debugged by a process that is not its parent.
PT_WRITE_BLOCK This request writes a block of data into the debugged process address space. The Address parameter points to the location in the process address space to be written into. The Data parameter gives the length of the block in bytes, and must not be greater than 1024. The Identifier parameter is interpreted as the process ID of the traced process. The Buffer parameter points to the location in the debugging process address space where the data is copied. Upon successful completion, the value of the Data parameter is returned to the debugging process.

If this request is unsuccessful, -1 is returned and the errno global variable is set to one of the following codes:

EIO The Data parameter is less than 1 or greater than 1024.
EIO The Address parameter is not a valid pointer into the debugged process address space.
EFAULT The Buffer parameter does not point to a readable location in the debugging process address space.

Note: For the PT_WRITE_BLOCK request, use ptracex with the 64-bit debuggee because 64-bit registers requiring 256 bytes are returned.

PT_WRITE_FPR This request sets the floating-point register specified by the Data parameter to the value specified by the Address parameter. The Identifier parameter is interpreted as the process ID of the traced process. The Buffer parameter is ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO The Data parameter is not a valid floating-point register. The Data parameter must be in the range 256-287.
PTT_WRITE_FPRS This request updates the contents of the 32 floating point registers with the values specified in the area designated by the Address parameter. This area must be at least 256 bytes long. The Identifier parameter specifies the traced kernel thread. The Data and Buffer parameters are ignored.
PT_WRITE_GPR This request stores the value of the Data parameter in one of the process general-purpose or special-purpose registers. The Address parameter specifies the register to be modified. Upon successful completion, the value of the Data parameter is returned to the debugging process. The Identifier parameter is interpreted as the process ID of the traced process. The Buffer parameter is ignored.
Note: If ptracex with a 64-bit debugee is used for the PT_WRITE_GPR request, the new register value is NOT passed via the Data parameter, but is instead passed via the 8-byte area pointed to by the buffer parameter.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO The Address parameter is not a valid general-purpose or special-purpose register. The Address parameter must be in the range 0-31 or 128-136.
PTT_WRITE_GPRS This request updates the contents of the 32 general purpose registers with the values specified in the area designated by the Address parameter. This area must be at least 128 bytes long. The Identifier parameter specifies the traced kernel thread. The Data and Buffer parameters are ignored.

Note: For the PTT_WRITE_GPRS request, use ptracex with the 64-bit debuggee because 64-bit registers requiring 256 bytes are returned. The buffer points to long long source area.

PT_WRITE_I or PT_WRITE_D These requests write the value of the Data parameter into the address space of the debugged process at the word-aligned address specified by the Address parameter. On all machines currently supported by the AIX Version 4 operating system, instruction and data address spaces are not separated. The PT_WRITE_I and PT_WRITE_D instruction and data requests can be used with equal results. Upon successful completion, the value written into the address space of the debugged process is returned to the debugging process. The Identifier parameter is interpreted as the process ID of the traced process. The Buffer parameter is ignored.

If this request is unsuccessful, -1 is returned and the errno global variable is set to the following code:

EIO The Address parameter points to a location in a pure procedure space and a copy cannot be made; the Address is not word-aligned; or, the Address is not valid. User blocks, kernel segments, and kernel extension segments are not considered valid addresses.

Note: For the or PT_WRITE_I or PT_WRITE_D request, use ptracex with a 64-bit debuggee because the target address needs 64 bits.

PTT_WRITE_SPRS This request updates the special purpose registers with the values in the area specified by the Address parameter, which points to a ptsprs structure. The Identifier parameter specifies the traced kernel thread. The Data and Buffer parameters are ignored.
Identifier Determined by the value of the Request parameter.
Address Determined by the value of the Request parameter.
Data Determined by the value of the Request parameter.
Buffer Determined by the value of the Request parameter.

Note: For the PTT_READ_SPRS request, use ptracex with the 64-bit debuggee because the new ptxsprs structure must be used.

Error Codes

The ptrace subroutine is unsuccessful when one of the following is true:

EFAULT The Buffer parameter points to a location outside the debugging process address space.
EINVAL The debugger and the traced process are the same; or the Identifier parameter does not identify the thread that caused the exception.
EIO The Request parameter is not one of the values listed, or the Request parameter is not valid for the machine type on which the process is executed.
ENOMEM Either the area is not large enough to accommodate the loader information, or there is not enough memory to allocate an equivalent buffer in the kernel.
EPERM The Identifier parameter corresponds to a kernel thread which is stopped in kernel mode and whose computational state cannot be read or written.
ESRCH The Identifier parameter identifies a process or thread that does not exist, that has not executed a ptrace call with the PT_TRACE_ME request, or that is not stopped.

For ptrace: If the debuggee is a 64-bit process, the options that refer to GPRs or SPRs fail with errno = EIO, and the options that specify addresses are limited to 32-bits.

For ptracex: If the debuggee is a 32-bit process, the options that refer to GPRs or SPRs fail with errno = EIO, and the options that specify addresses in the debuggee's address space that are larger than 2**32 - 1 fail with errno set to EIO.

Also, the options PT_READ_U and PT_WRITE_U are not supported if the debuggee is a 64-bit program (errno = ENOTSUP).

Implementation Specifics

This subroutine is part of Base Operating System (BOS) Runtime.

Related Information

The exec, getprocs, getthrds, load, sigaction, unload, wait, waitpid, or wait3 subroutine.

The dbx command.


[ Next Article | Previous Article | Book Contents | Library Home | Legal | Search ]