SMP Scheduling

AIX Versions 3.2 and 4 Performance Tuning Guide

SMP Scheduling

Thread support, added to AIX Version 4, divides program-execution control into two elements:

A process is a collection of physical resources required to run the program, such as memory and access to files.
A thread is the execution state of an instance of the program, such as the current contents of the instruction-address register and the general-purpose registers. Each thread runs within the context of a given process, and uses that process's resources. Multiple threads can run within a single process, sharing its resources.

In previous versions of AIX, the CPU scheduler dispatched processes. In AIX Version 4, the scheduler dispatches threads.

In the SMP environment, the availability of thread support makes it easier and less expensive to implement SMP-exploiting applications. Forking multiple processes to create multiple flows of control is cumbersome and expensive, since each process has its own set of memory resources and requires considerable system processing to set up. Creating multiple threads within a single process requires less processing and uses less memory.

Thread support exists at two levels:

libpthreads.a support in the application program environment
kernel thread support.

Default Scheduler Processing of Migrated Workloads

The new division between processes and threads is invisible to existing programs. In fact, workloads migrated directly from earlier releases of AIX create processes as before. Each new process is created with a single thread (the initial thread) that contends for the CPU with the threads of other processes. The default attributes of the initial thread, in conjunction with the new scheduler algorithms, minimize changes in system dynamics for unchanged workloads.

Priorities can be manipulated with the nice and renice commands and the setpri and setpriority system calls, as before. The scheduler allows a given thread to run for at most one time slice (normally 10ms) before forcing it to yield to the next dispatchable thread of the same or higher priority.

Scheduling Algorithm Variables

Several variables affect the scheduling of threads. Some are unique to thread support; others are elaborations of process-scheduling considerations:

Priority	A thread's priority value is the basic indicator of its precedence in the contention for processor time.
Scheduler run queue position	A thread's position in the scheduler's queue of dispatchable threads reflects a number of preceding conditions.
Scheduling policy	This thread attribute determines what happens to a running thread at the end of the time slice.
Contention scope	A thread's contention scope determines whether it competes only with the other threads within its process or with all threads in the system. A pthread created with process contention scope is scheduled by the library, while those created with system scope are scheduled by the kernel. The library scheduler utilizes a pool of kernels threads to schedule pthreads with process scope. Generally, pthreads should be created with system scope, if they are performing I/O. Process scope is useful, when there is a lot of intra-process synchronizations. Contention scope is a libpthreads.a concept.
Processor affinity	The degree to which affinity is enforced affects performance

The combinations of these considerations can seem complex, but there are essentially three distinct approaches from which to choose in managing a given process:

Default	The process has one thread, whose priority varies with CPU consumption and whose scheduling policy, SCHED_OTHER, is comparable to the AIX Version 3 algorithm.
Process-level control	The process can have one or more threads, but the scheduling policy of those threads is left as the default SCHED_OTHER, which permits the use of the existing AIX Version 3 methods of controlling nice values and fixed priorities. All of these methods affect all of the threads in the process identically. If setpri() is used, the scheduling policy of all of the threads in the process is set to SCHED_RR
Thread-level control	The process can have one or more threads. The scheduling policy of these threads is set to SCHED_RR or SCHED_FIFO, as appropriate. The priority of each thread is fixed, and is manipulated with thread-level subroutines.

Scheduling Environment Variables

Within the libpthreads.a framework, a series of tuning knobs have been provided that may impact the performance of the application. These environment variables are:

SPINLOOPTIME=n, where n is the number of times to retry a busy lock before yielding to another processor. n must be a positive value.
YIELDLOOPTIME=n, where n is the number of times to yield the processor before blocking on a busy lock. n must be a positive value. The processor is yielded to another kernel thread, assuming there is another runnable one with sufficient priority.
AIXTHREAD_SCOPE={P|S}, where P signifies process based contention scope and S signifies system based contention scope. Either "P" or "S" should be specified. The braces are provided for syntactic reasons only. The use of this environment variable impacts only those threads created with the default attribute. The default attribute is employed, when the attr parameter to pthread_create is NULL.

The following environment variables impact the scheduling of pthreads created with process based contention scope.

AIXTHREAD_MNRATIO=p:k, where k is the number of kernel threads that should be employed to handle p runnable pthreads. This environment variable controls the scaling factor of the library. This ratio is used when creating and terminating pthreads.
AIXTHREAD_SLPRATIO=k:p, where k is the number of kernel threads that should be held in reserve for p sleeping pthreads. In general, fewer kernel threads are required to support sleeping pthreads, since they are generally woken one at a time when processing locks and/or events. This conserves kernel resources.
AIXTHREAD_MINKTHREADS=n, where n is the minimum number of kernel threads that should be used. The library scheduler will not reclaim kernel threads below this figure. A kernel thread may be reclaimed at virtually any point. Generally, a kernel thread is targeted as a result of a pthread terminating.