DISPATCHER_HEADER

Every kernel object that can be waited on, e.g., by giving its address as the first argument to KeWaitForSingleObject, begins with a DISPATCHER_HEADER. Microsoft’s symbol files for the kernel identify the following structures as beginnning with a DISPATCHER_HEADER:

• KEVENT;
• KGATE (5.2 SP1, and higher);
• KMUTANT;
• KPRIQUEUE (6.3 and higher);
• KPROCESS;
• KQUEUE;
• KSEMAPHORE;
• KTHREAD;
• KTIMER;
• KTIMER2 (6.3 and higher).

For each, the DISPATCHER_HEADER is named Header (not that it must be, just that it is). For the KEVENT and KGATE, the Header is the only member, and so these objects are in effect nothing but a DISPACHER_HEADER. Some types of object add only one or a few members to the Header. Others, notably the KPROCESS and KTHREAD, are among the largest and most complex structures in all of kernel-mode programming.

It is through their common beginning that these objects pick up some common behaviour of being waitable. When the address of a waitable object is given to a function such as KeWaitForSingleObject, the function does not return until the object has been signalled. Meanwhile, the calling thread is left in an efficient wait state, with the kernel knowing not to waste time on the thread’s execution. Some of the behaviour varies with the type of object. For instance, signalling ordinarily releases all the waiting threads, but for some types signalling releases only one waiter. How an object even gets signalled varies with the type of object. Some can be signalled explicitly, others only implicitly. For instance, a KEVENT is signalled by passing its address to KeSetEvent but a KPROCESS is signalled from within the kernel just as a consequence of the corresponding process’s termination.

Documentation Status

The DISPATCHER_HEADER is not formally documented but even the Device Driver Kit (DDK) for Windows NT 3.1 supplies a C-language definition in NTDDK.H, and each new DDK or Windows Driver Kit (WDK) has continued to, though the definition soon moved to WDM.H.

Of the objects that build on the DISPATCHER_HEADER, Microsoft publishes C-language definitions for some but not all. The main distinction concerns the allocation of memory for the object. For some types, such as KEVENT, memory for the object can be supplied by its intended user, such as a kernel-mode driver, who then calls a kernel function to initialise the object. Other types, such as KPROCESS, only ever exist in memory that is allocated by the kernel, the intended user of the object having called a kernel function (including indirectly from user mode) that creates the object. The former have C-language definitions in public header files and the latter don’t.

Variability

Although its layout is public, the DISPATCHER_HEADER surely is intended to be treated as opaque outside the kernel. The definition seems to have been published only so that in those cases where drivers and other kernel-mode modules create the waitable object they can know how much space to allocate. If only in principle, how much space, and what sort of space, is all they need to know. What happens in the space is entirely in the hands of kernel functions that are provided for initialising and then working with the object. Microsoft might therefore have defined the DISPATCHER_HEADER as an array of bytes, with no consequences for programmers at large except if the size ever changed.

In all versions, the DISPATCHER_HEADER is 0x10 and 0x18 bytes in 32-bit and 64-bit Windows respectively. Constancy of size is not strictly required, but is more or less required for compatbility, given the expectation of opacity in caller-supplied memory. The same opacity, however, means that interpretation within the constant size could change completely even between builds, if only in principle.

It happens that the DISPATCHER_HEADER has varied hardly at all if understood as a common header for all waitable objects. In this sense, the simple structure from the early kits is all that any kernel-mode programmer needs to keep in mind, e.g., to recognise when debugging. In another sense, the layout has changed significantly as the structure has been elaborated here and there for different types of object. At first, and mostly since, the mechanism has been that the object picks up more functionality as Windows evolves but new members that support the new functionality are squeezed into the header because the object as a whole is constrained to its original size.

Layout

Type-specific members are all in the first four bytes. Their proliferation has greatly complicated the header’s description, both for this page and in Microsoft’s own programming. This article distinguishes three layouts according to the organisation of the first four bytes:

• originally, individual members at the same level as the subsequent members;
• in versions 5.2 to 6.3, a union of a four-byte integer and a sub-structure mostly of unions for different object types;
• in Windows 10, a union of a four-byte integer and four-byte structures for different object types.

Please bear in mind that the difference between layouts is mostly cosmetic. It is Microsoft’s description in C that changes, not (with a few exceptions) the position and interpretation of the members. Importantly, there has been no change at all, ever, in the two members for whether the object is yet signalled and for which threads currently wait on the object:  

/*  integral members, see below  */

union {
    struct {
        /*  four single-byte members, see below  */
    };
    LONG volatile Lock;
};

union {
    struct {
        /*  four single-byte members, see below  */
    };
    union {
        LONG volatile Lock;
        LONG LockNV;
    };
};

union {
    union {
        LONG volatile Lock;
        LONG LockNV;
    };
    /*  four-byte members, see below  */
};

LONG SignalState;

LIST_ENTRY WaitListHead;

Though different types of object have different ways of changing the SignalState, what they have in common is that the object is regarded as signalled when the SignalState is positive. Indeed, for many types of object, the SignalState is only ever 0 (not signalled) or 1 (signalled).

Any object can have multiple threads waiting on it simultaneously. Any thread can be waiting on multiple objects concurrently. For each object that it waits on, a thread needs a KWAIT_BLOCK structure. This may be (and, in practice, almost always is) one of three such structures that are built-in to the KTHREAD or it can be provided from outside, as must be done by callers of KeWaitForMultipleObjects when multiple is more than three. Each thread that waits on the same object gets a wait block appended to the double-linked list that begins with the object’s WaitListHead and links through the wait block’s WaitListEntry member.

Original (3.10 to 5.1)

Before Windows Server 2003, the first four bytes of the DISPATCHER_HEADER are straightforwardly defined as 1-byte and 2-byte integers. They were originally divided equally into a Type and a Size. That space might be squeezed from these seems to have been anticipated for version 3.51 but was not acted on until version 4.0.

SHORT Type;

UCHAR Type;

UCHAR Spare;

UCHAR Absolute;

SHORT Size;

USHORT Size;

UCHAR Size;

UCHAR Inserted;

Object Type

Though no DISPATCHER_HEADER definition in any DDK or WDK ever says so, the Type—or, to be precise, its low 7 bits except in early versions—comes from the undocumented KOBJECTS enumeration.

Though the enumeration and its possible values are disclosed in the public symbol files for the kernel, Microsoft has plainly not intended that any of this shows in the usual headers for programming. All that slips out nowadays in the WDK headers is a comment in WDM.H that the Type is “accessible via KOBJECT_TYPE” but the latter, which is presumably a macro, is not defined in any of the WDK headers.

This comment first appeared in WDM.H from the WDK for Windows 7. It, but not any later WDK, has macros such as ASSERT_EVENT that show KOBJECT_TYPE in use for testing that an object is the type it’s expected to be. These in turn show that the result of applying KOBJECT_TYPE to an object is some such symbol as GateObject or EventSynchronizationObject (from the enumeration). I pick out these two because the macro that tests for these, ASSERT_GATE, looks to be the oldest of this sort. It shows first in the DDK for Windows Server 2003 SP1. There, it does not use KOBJECT_TYPE but instead computes the object’s type by extracting bits, such that the presumably later macro is defined something like:

#define KOBJECT_TYPE(object) ((object) -> Header.Type & KOBJECT_TYPE_MASK)

The name KOBJECT_TYPE_MASK suggests an intention that it applies to an arbitrary object to extract its type. And it does indeed work this way, starting with Windows 7. Then and since, the type of any object is only the low 7 bits of the Type because the high bit was given a special meaning. At first, however, this was true only for particular types of object. The WDM.H from the WDK for Windows Vista has some macros test against the low 7 bts but most compare the whole 8. The high bit had meaning only for gate objects in version 5.2 SP1 and then also for queue objects in version 6.0. In these cases, and then for all types of object in version 6.1 and higher, the high bit of the Type acts as a lock. No changes are ever made to an object’s DISPATCH_HEADER without setting the high bit, having waited in a spin, if necessary, for the high bit to have become clear.

By the way, there is other interpretation of the Type in bits, and it has even earlier history, going back to version 4.0. When this version extended the ancient differentiation of notification and synchronization event objects to apply also to timer objects, it coded for generality. Any object, not just events and timers, might be a synchronization object in the sense that when the object is signalled and a waiting thread is released from its wait, the object is automatically reset to non-signalled such that no other waiting thread is released until the object is signalled again. What distinguishes that an object gets this synchronization behaviour is that the low three bits of its Type are exactly 1.

Object Size

The Size is that of the whole object, including the header—but only of the kernel object, not of an even larger object such as managed by the Object Manager and exposed even to user mode through a handle. For instance, for a thread object, the Size is that of the KTHREAD, not the ETHREAD.

The original 16-bit Size counted bytes. When narrowed to 8 bits for version 4.0, it changed to counting dwords.

Timer-Specific

The Absolute and Inserted members are the chronologically first examples of the DISPATCHER_HEADER being specialised for a type of object. The motivation in this case came when Windows NT 4.0 improved the functionality of the KTIMER enough to need new members. Because timer objects are among the types that can be in caller-supplied memory, these new members would ideally be squeezed in without enlarging the KTIMER.

The boolean Absolute was genuinely new, and brought into use a byte that version 3.51 had explicitly set aside as Spare. The other, Inserted, had been at the end of the KTIMER beyond the header. As a side-effect of alignment, moving its one byte to the header freed four bytes for the new use of allowing timers to be periodic. Space for Inserted in the header was found by taking from the Size.

Nested Unions (5.2 to 6.3)

Starting with Windows Server 2003 the role of the high bit of the Type as the object’s lock is formalised as one 32-bit Lock laid over the first four bytes so that all four bytes, but especially the high bit of the Type, can be accessed together with one instruction such as cmpxchg or bts that can take the lock prefix (and which anyway can’t operate on single bytes).

struct {
    /*  varying members, see below  */
};

LONG volatile Lock;

union {
    LONG volatile Lock;
    LONG LockNV;
};

Except for the Type, each of the first four bytes—in the structure within the union—soon becomes a union of different members for different object types:

UCHAR Type;

UCHAR Absolute;

union {
    /*  varying members, see below (Byte 1)  */
};

UCHAR Size;

union {
    /*  varying members, see below (Byte 2)  */
};

union {
    /*  varying members, see below (Byte 3)  */
};

Better might have been to define the last three of these bytes as a union of three-byte structures for the different types, but the business of this article is to describe, not prescribe, however messy the description has to be. That said, the definition in WDM.H seems to have got messy enough that Microsoft redid it for Windows 10: you may prefer to skip ahead.

Byte 1

union {
    UCHAR TimerControlFlags;
    struct {
        /*  bit fields, follow link  */
    };
};

UCHAR Abandoned;

union {
    UCHAR QueueControlFlags;
    struct {
        /*  bit fields, follow link  */
    };
};

UCHAR Absolute;

UCHAR NpxIrql;

BOOLEAN Signalling;

union {
    UCHAR Timer2Flags;
    struct {
        /*  bit fields, follow link  */
    };
};

Byte 2

Windows Server 2003 SP1 made a simple union of what had just been the Size, adding Hand for timer objects—in that order. Windows 7 swapped them, and put them both after a new set of bit fields for thread objects.

UCHAR Size;

union {
    UCHAR ThreadControlFlags;
    struct {
        /*  bit fields, follow link  */
    };
};

UCHAR Hand;

UCHAR Reserved3;

UCHAR Size;

See that as pressure mounted for finding new space for new functionality in timer objects, keeping a size in the header was ruled extravagant. Timer objects can be in memory supplied by their intended user and Microsoft apparently regarded this as fixing the size in stone. Starting with version 5.2 SP1, timer objects do not have a size in their header.

Starting with version 6.1, thread objects also lose the size from their header. Thread objects, however, are created only by the kernel and evidently do not have their size constrained by any considerations of compatibility. It is presently not known what is so important about the ThreadControlFlags that they needed to be in the header, rather than be defined elsewhere in the KTHREAD.

Byte 3

The last of the first four bytes had been used as Inserted for timer objects since version 4.0. Windows Server 2003 added DebugActive for thread objects, Windows Vista added DpcActive for mutants, and then Windows 7 made bit fields of the first two. For DebugActive this merely formalised an earlier interpretation in bits, despite the formal declaration as a BOOLEAN—and then these bit fields are dropped from the definition for x86 builds in Windows 8 and higher.

UCHAR Inserted;

union {
    UCHAR TimerMiscFlags;
    struct {
        /*  bit fields, follow link  */
    };
};

BOOLEAN DebugActive;

 union {
    BOOLEAN DebugActive;
    struct {
        /*  bit fields, follow link  */
    };
};

union {
    BOOLEAN DebugActive;
};

BOOLEAN DpcActive;

UCHAR Reserved5;

Union of Structures (10.0 and Higher)

Windows 10 reorganises the first four bytes into a union of different four-byte structures for the different object types:

 union {
    union {
        LONG volatile Lock;
        LONG LockNV;
    };
    /*  varying unnamed structures, see below  */
};

General Objects

The objects that have no special interpretation are events, processes, semaphores and gates.

UCHAR Type;

UCHAR Signalling;

UCHAR Size;

UCHAR Reserved1;

Timer Objects

UCHAR TimerType;

union {
    UCHAR TimerControlFlags;
    struct {
        /*  bit fields, follow link  */
    };
};

UCHAR Hand;

union {
    UCHAR TimerMiscFlags;
    struct {
        /*  bit fields, follow link  */
    };
};

Timer2 Objects

UCHAR Timer2Type;

union {
    UCHAR Timer2Flags;
    struct {
        /*  bit fields, follow link  */
    };
};

UCHAR Timer2Reserved1;

UCHAR Timer2ComponentId;

UCHAR Timer2Reserved2;

UCHAR Timer2RelativeId;

Queue Objects

UCHAR QueueType;

union {
    UCHAR QueueControlFlags;
    struct {
        /*  bit fields, follow link  */
    };
};

UCHAR QueueSize;

UCHAR QueueReserved;

Thread Objects

UCHAR ThreadType;

UCHAR ThreadReserved;

union {
    UCHAR ThreadSpecControl;
    struct {
        UCHAR SpecControlIbrs : 1;
        UCHAR SpecControlStibp : 1;
        UCHAR SpecControlReserved : 6;
    };
};

UCHAR ThreadReserved;

union {
    UCHAR ThreadControlFlags;
    struct {
        /*  bit fields, follow link  */
    };
};

union {
    UCHAR DebugActive;
    struct {
        /*  bit fields, follow link  */
    };
};

The ThreadSpecControl, including the bit fields and the union, is from public symbol files for the kernel. The C-language definition in WDM.H from the WDK for the 1803 release of Windows 10 persists with ThreadReserved.

Mutant Objects

For mutant objects, the DpcActive member was at offset 0x03 in versions 6.0 to 6.2. Windows 10 actually does shift it to offset 0x02 (and the Size to offset 0x01). Whether this was intentional, or is just a transcription error in Microsoft’s reorganisation of the surprisingly complex structure that the DISPATCH_HEADER had become, is not known.

UCHAR MutantType;

UCHAR MutantSize;

BOOLEAN DpcActive;

 UCHAR MutantReserved;