This page is being prepared for a substantial reworking. Some content may be temporarily missing. Temporarily may turn into indefinitely. New content may be more than usually defective. The page is now published only as a revision in progress. Use with caution.

MESSAGE_TRACE_HEADER

The MESSAGE_TRACE_HEADER is one of several types of fixed-size header that introduce variable-size data for events that are logged through Event Tracing for Windows (ETW). As with other types of event, those that begin with a MESSAGE_TRACE_HEADER accumulate first in trace buffers. To have these events persist in this raw form for ready inspection, configure the event tracing session to flush the trace buffers to an Event Trace Log (ETL) file.

Usage

The events that get a MESSAGE_TRACE_HEADER, rather than one of the other types of trace header, are generated through a handful of interfaces that are mostly used for Windows Pre-Processor (WPP) software tracing. The kernel-mode interface is the pair of documented kernel exports WmiTraceMessage and WmiTraceMessageVa. The kernel exposes the functionality to user mode through the undocumented NtTraceEvent function. In user mode, this path to the kernel-mode implementation goes through the the undocumented NTDLL functions EtwTraceMessage and EtwTraceMessageVa, which are in turn accessed as forwards from the documented ADVAPI32 functions TraceMessage and TraceMessageVa. Much of the magic of WPP is that it hides all this behind what looks to the programmer to be a programmer-defined function in the style of printf from the C Run-Time library.

Documentation Status

The MESSAGE_TRACE_HEADER structure is not documented. However, Microsoft has published a C-language definition in the NTWMI.H from the Enterprise edition of the Windows Driver Kit (WDK) for Windows 10 version 1511.

Were it not for this relatively recent and possibly unintended disclosure, much would anyway be known from type information in symbol files. Curiously though, type information for this structure has never appeared in any public symbol files for the kernel or for the obvious low-level user-mode DLLs. In the whole of Microsoft’s packages of public symbol files, at least to the original Windows 10, relevant type information is unknown before Windows 8 and appears in symbol files only for AppXDeploymentClient.dll, CertEnroll.dll (before Windows 10) and Windows.Storage.ApplicationData.dll.

Layout

The MESSAGE_TRACE_HEADER is 8 bytes in both 32-bit and 64-bit Windows in all known versions.

union {
    ULONG Marker;
    struct {
        USHORT Size;
        UCHAR Reserved;
        UCHAR Version;
    };
};

union {
    ULONG Header;
    WMI_TRACE_MESSAGE_PACKET Packet;
};

What distinguishes the MESSAGE_TRACE_HEADER from other Trace Headers is a particular combination of high bits in the Marker. NTWMI.H defines them as:

While the WMI_TRACE_MESSAGE_PACKET is not known to have any other use, it is as well presented here:

USHORT MessageNumber;

USHORT OptionFlags;

Both are vital for interpreting the event-specific data that follows the header. The MessageNumber is the primary identifier of what event occurred. It is what the kernel received as the MessageNumber argument via the documented API functions. The OptionFlags tell what data follow the header. There can be any selection of the following items in the following order:

• a 32-bit sequence number;
• either a 32-bit component ID or a 16-byte GUID (but not both);
• an 8-byte time stamp;
• 32-bit thread and process IDs (in that order) as so-called system information;
• arbitrary data, as message arguments.

The Size in the low word of the Marker is the total size in bytes of the header and all these items. Note that although the MESSAGE_TRACE_HEADER always has 8-byte alignment in a trace buffer, the 8-byte time stamp need not have (and typically hasn’t, since a sequence number is usual).

The first six OptionFlags come directly from the MessageFlags argument of the relevant API functions and indicate which items are present between the header and the arbitrary data:

In all versions, TRACE_MESSAGE_COMPONENT_ID has precedence over TRACE_MESSAGE_GUID. The MessageGuid argument can have supplied only one or the other, not both. If both flags are set, what’s present is the component ID, not the GUID. Note that Microsoft somehow documents TRACE_MESSAGE_COMPONENTID with TraceMessage but not with TraceMessageVa, and seems never to have documented it for kernel-mode use.

Interpretation of the time-stamping flags is version-dependent. All versions calculate space for a time stamp if either flag is set, but in no known version does an event actually receive a time stamp unless TRACE_MESSAGE_TIMESTAMP is set. The intention seems to be that TRACE_MESSAGE_TIMESTAMP indicates that a time stamp is present and then TRACE_MESSAGE_PERFORMANCE_TIMESTAMP is a refinement to tell which type, but only in version 5.1 does a set TRACE_MESSAGE_PERFORMANCE_TIMESTAMP indicate certainly that the time stamp is from the performance counter (in contrast to the system time). In later versions, the event’s time stamp comes from whatever clock the logger ordinarily uses and TRACE_MESSAGE_PERFORMANCE_TIMESTAMP is meaningless. Note that Microsoft already had TRACE_MESSAGE_PERFORMANCE_TIMESTAMP documented as obsolete in the Windows XP Device Driver Kit (DDK).

The arbitrary data are copied from the addresses and sizes that are given to the API functions as variable arguments or via the MessageArgList argument. See that no types or sizes are recorded. All data from the argument list is simply aggregated as one blob. Its useful interpretation as placeholders for resolving arguments must be the shared knowledge of provider and consumer, presumably varying according to event-specific identifiers such as the GUID or component ID, and the MessageNumber—and, where pointer and size_t types are expected in the data, on the last two of the OptionFlags.