This page is being prepared for a substantial reworking. Some content may be temporarily missing. Temporarily may turn into indefinitely. New content may be more than usually defective. The page is now published only as a revision in progress. Use with caution.

EVENT_TRACE_HEADER

The EVENT_TRACE_HEADER is one of several types of fixed-size header that introduce variable-size data for events that are logged through Event Tracing for Windows (ETW). As with other types of event, those that begin with an EVENT_TRACE_HEADER accumulate first in trace buffers. To have these events persist in this raw form for ready inspection, configure the event tracing session to flush the trace buffers to an Event Trace Log (ETL) file.

Availability

The EVENT_TRACE_HEADER dates from version 5.0, when ETW was not yet named ETW but was arguably just a side-line to the then-new and much-touted Windows Management Infrastructure (WMI). Indeed, the EVENT_TRACE_HEADER of ETW Is very similar to, though certainly not the same as, the WNODE_HEADER of WMI.

Since the substantial reworking of ETW for Windows Vista, the EVENT_TRACE_HEADER is largely superseded by the EVENT_HEADER. Events that start with an EVENT_TRACE_HEADER can still be written, presumably to support not just old tools but also diagnostic techniques that have not been entirely displaced by the supposedly new and improved. For event consumption (see below), the structure is obsolete, i.e., is properly ignored in favour of the later structure, unless the event consumer may run on older Windows versions.

Usage

Apparently for surviving from the simpler architecture of ETW in Windows 2000, the EVENT_TRACE_HEADER can be involved in event tracing all the way from provider to consumer.

Provision

If an event begins with an EVENT_TRACE_HEADER when seen in the trace buffers, then it typically originated with this header. The usual way that it will have been provided to the ETW machinery is through the documented API function TraceEvent which takes an EVENT_TRACE_HEADER as its input. Though the similarly old kernel export IoWMIWriteEvent is documented as taking a WNODE_HEADER as its input, it can accept an EVENT_TRACE_HEADER instead.

Consumption

The EVENT_TRACE_HEADER can also turn up when retrieving events from the ETW machinery, even events that have some other type of trace header while in the trace buffers (or ETL file). The usual way is that an event consumer’s call to the documented API function OpenTrace registers one or more routines that are to be called back, typically many times over, during a later call to the documented API function ProcessTrace. One of these routines delivers events, one on each call back. The older form of this routine presents its event as an EVENT_TRACE structure which begins with an EVENT_TRACE_HEADER. (The newer form of the routine replaces these with the EVENT_RECORD and EVENT_HEADER, respectively.)

Documentation Status

The EVENT_TRACE_HEADER structure is documented. It has been since at least an MSDN Library dated January 2000 (albeit marked as preliminary documentation). So too is the TraceEvent function that can take an EVENT_TRACE_HEADER as input.

As ever, the picture for kernel-mode use is a little different. Documentation that IoWMIWriteEvent accepts an EVENT_TRACE_HEADER has always been, shall we say, obscure. To this day, 10th December 2018, Microsoft’s online page for IoWMIWriteEvent makes no mention of the EVENT_TRACE_HEADER. Documentation has instead always been either by hint or in reverse: the programmer can follow links from the function to learn that if a particular flag is set in a WNODE_HEADER, then the latter is actually an EVENT_TRACE_HEADER; or the programmer can know to start with the kernel-mode documentation of the EVENT_TRACE_HEADER and then learn which function can write one.

Microsoft’s C-language definition of the EVENT_TRACE_HEADER is in the EVNTRACE.H header for both kernel-mode and user-mode programming, starting with development kits for Windows 2000.

Layout

The EVENT_TRACE_HEADER is 0x30 bytes in both 32-bit and 64-bit Windows in all versions that are known to have it, i.e., 5.0 and higher:

USHORT Size;

UCHAR HeaderType;

union {
    USHORT FieldTypeFlags;
    struct {
        UCHAR HeaderType;
        UCHAR MarkerFlags;
    };
};

UCHAR MarkerFlags;

union {
    ULONG Version;
    struct {
        UCHAR Type;
        UCHAR Level;
        USHORT Version;
    } Class;
};

ULONGLONG ThreadId;

ULONG ThreadId;

ULONG ProcessId;

LARGE_INTEGER TimeStamp;

union {
    GUID Guid;
    ULONGLONG GuidPtr;
};

union {
    struct {
        ULONG ClientContext;
        ULONG Flags;
    };
    struct {
        ULONG KernelTime;
        ULONG UserTime;
    };
    ULONG64 ProcessorTime;
};

union {
    struct {
        ULONG KernelTime;
        ULONG UserTime;
    };
    ULONG64 ProcessorTime;
    struct {
        ULONG ClientContext;
        ULONG Flags;
    };
};

The EVENT_TRACE_HEADER developed as a specialised WNODE_HEADER. The adaptation was made overt by a comment which precedes the C-language definition in EVNTRACE.H before Windows Vista:

// Trace header for all (except kernel) events. This is used to overlay
// to bottom part of WNODE_HEADER to conserve space.

Both structures have the same size and have TimeStamp, Guid, ClientContext and Flags members at the same offsets. Functions that might take either structure distinguish them from the Flags.

Size

The Size is the total, in bytes, of the fixed-size EVENT_TRACE_HEADER and all the variable-size event data that follows. The WNODE_HEADER has its size as a 32-bit BufferSizeThe Size is among the members that must be set on input by the event provider.

Header Type and Marker Flags

The first 4 bytes have common elements in all the various trace headers. Some begin with a 16-bit Size, some not, but if the first four bytes are taken as one 32-bit marker (which some do formally define as a member named Marker), then all types of trace header have the high two bits set. For the EVENT_TRACE_HEADER, given its particular definition of the first four bytes, this means that the high two bits are necessarily set in the 8-bit MarkerFlags at offset 0x03. What distinguishes a trace header as continuing specifically as an EVENT_TRACE_HEADER is that the 8-bit HeaderType at offset 0x02 is either:

Microsoft’s names for these values are known from a header, named NTWMI.H, which might otherwise have been internal to Microsoft except for possibly accidental disclsoure in the original and Version 1511 editions of the WDK for Windows 10. Another otherwise unpublished header, NTETW.H, defines corresponding masks to combine with the 16-bit Size in the 32-bit marker:

Flags

As noted above, the Flags at offset 0x2C are shared with the WNODE_HEADER. At first, it was only as Flags of the WNODE_HEADER that any of the applicable bits were named. A comment notes—it is still there—that

// The second byte, except for the first bit is used exclusively for tracing

and is followed by definitions of bits in the third byte. Names that showed their meaningfulness to the EVENT_TRACE_HEADER had to wait for Windows Vista.

Though WNODE_FLAG_USE_TIMESTAMP is defined for the WNODE_HEADER as early as version 5.0, no use of it is known for the EVENT_TRACE_HEADER until version 5.1. Its effect then and after is to signify that the event is already time-stamped by the event provider. Ordinarily, the TimeStamp is generated when the event is written to a trace buffer. With this flag set, the input TimeStamp is instead retained.

TO BE CONTINUED