MMPTE_HARDWARE

The MMPTE_HARDWARE structure is Microsoft’s representation of a Page Table Entry (PTE) such as the processor can use when translating a linear address to a physical address.

The processor finds these PTEs by following a chain of physical addresses that begin from what’s in the cr3 register. The translation algorithm iterates through as many as four levels. Since this is all documented by Intel and our concern is less with the processor than the Memory Manager, a brief summary suffices here. Successively less significant parts of the linear address each provide an index into successively lower levels of page tables. These get successively more clumsy names—page directory, page directory pointer table and page map level 4 (PML4) table—but each page table at whatever level is a page-sized array of PTEs. The indexed PTE at each level provides the physical address of the next (lower) level of page table until the iteration completes. The physical address from the lowest-level PTE is the physical address of the page that contains the given linear address.

There are presently three translation algorithms. Their mechanisms differ in the size of the PTE and the maximum possible depth of the iteration. The effect is of translating between different sizes of linear and physical address spaces. The choice is made even before the kernel is loaded. Indeed, the kernel exists in different forms which each implement only one of the translation algorithms:

Access

The Memory Manager, of course, accesses PTEs by linear addresses, not physical. It greatly eases this work by preparing the highest-level page table to have one of its PTEs give the physical address of that same page table. This creates a linear address whose translation to a physical address uses just this one PTE at all levels, but its larger consequence is that the PTEs for successive pages in the whole linear address space become addressable as one array. Through most of the history of Windows, and still in 32-bit Windows, the base address of this PTE array is preset:

Microsoft’s macro definitions for the x64 values of these magic addresses have been published in the NTDDK.H and other headers from all versions of the Device Driver Kit (DDK) and Windows Driver Kit (WDK) from as far back as Windows XP. That Microsoft has matching definitions for the x86 values is no surprise but public confirmation is known only from the NTOSP.H file from the Enterprise WDK for the 1511 release of Windows 10. Starting with the 1607 release of Windows 10, the x64 kernel is still built with these addresses hard-coded, e.g., as immediate data in instructions, but in a continuing programme of Address Space Layout Randomization (ASLR) they all get changed at load time through the Dynamic Value Relocation Table in the kernel’s load configuration.

The reverse engineer or even the programmer who’s doing kernel-mode debugging will not get far into the Memory Manager’s code without encountering sequences that compute the address of the PTE for an arbitrary linear address. Let MmPteBase be an internal variable or the hard-coded PTE_BASE, but either way a pointer to the whole MMPTE array. Then the address of the PTE for the linear address p is

&MmPteBase [(ULONG_PTR) (p) >> PAGE_SHIFT]

in 32-bit Windows, with or without Physical Address Extension (PAE), but the computation for 64-bit Windows needs just a little more because only the low 48 bits of a 64-bit linear address are meaningful:

&MmPteBase [((ULONG_PTR) (p) & 0x0000FFFFFFFFFFFF) >> PAGE_SHIFT]

See that the computation is recursive. For instance, the address of the PDE for a given address is the address of the PTE for the PTE for that address. This recursion of course produces more formulae that the programmer or reverse engineer may do well to recognise. For the magic addresses in particular, the recursion is that the PTE for PTE_BASE is at PDE_BASE, the PTE for which is at PPE_BASE, and so on, as far as applicable to the architecture. The PTE for the self-mapping PTE is itself.

Layout

That a PTE is intended for the processor’s interpretation is presumably what makes it a hardware PTE for the Memory Manager. In Intel’s terminology for the processor, the translation from linear address to physical address can complete only if each PTE along the way has a set P bit (masked by 0x01). Only then is anything else in a PTE meaningful to the processor. Encountering a hardware PTE with a clear P bit causes the processor to raise an exception. The effect is to hand the PTE to software for interpretation as if the PTE were instead a software PTE. It is here thought that the MMPTE_HARDWARE is the Memory Manager’s one structure for a PTE that is intended to be interpreted by the processor. An MMPTE_HARDWARE may have a clear P bit but its continued interpretation is then as an MMPTE_SOFTWARE.

Names and types in the following tables are from public symbol files for the kernel, starting with Windows 2000 SP3 in general. The exception is that symbol files for the single-processor kernel without PAE support don’t have type information before Windows XP. This mostly doesn’t matter and is left unspecified almost everywhere that this website says that information is from public symbol files for Windows 2000, but it matters for the MMPTE_HARDWARE because the kernel manages some bits differently if it has the complication of executing on multiple processors.

32-Bit PTE

For the x86 builds that do not use PAE, page table entries are four bytes. The whole MMPTE_HARDWARE is a structure of ULONG bit fields:

ULONG Valid : 1;

ULONG Write : 1;

ULONG Writable : 1;

ULONG Dirty1 : 1;

ULONG Owner : 1;

ULONG WriteThrough : 1;

ULONG CacheDisable : 1;

ULONG Accessed : 1;

ULONG Dirty : 1;

ULONG LargePage : 1;

ULONG Global : 1;

ULONG CopyOnWrite : 1;

ULONG Prototype : 1;

ULONG reserved : 1;

ULONG Write : 1;

ULONG PageFrameNumber : 20;

Remember that “all” and “higher” reach only to version 6.1. Though kernels without support for PAE may exist in theory for version 6.2 and higher, none are known to have been distributed.

Given that Valid is set, the lowest nine bits all have their meaning defined by Intel for interpretation by the processor. The CopyOnWrite, Prototype and (the multi-processor) Write bits are how Windows uses the three bits that Intel leaves to the operating system even in a hardware PTE.

The CopyOnWrite bit acts most usefully for a page that is not enabled for Write access. The physical page that corresponds to the linear address is protected from change. Attempting to write to it causes a page fault which the operating system resolves by finding a new physical page, filling it with the contents of the protected page, and then remapping the linear address to the new page.

The Prototype bit has an important role in the various types of software PTE. In general, an MMPTE_SOFTWARE that has the Prototype bit set is interpreted next as an MMPTE_PROTOTYPE. Early versions go straight to this interpretation even without testing that Valid is clear.

Redefining the Write bit from Intel’s 0x00000002 to 0x00000800 is Microsoft’s way around a problem that multi-processor systems present to operating-system software that would clear the Dirty bit. This latter is set in the PTE by any processor that writes to any linear address whose translation ends with this PTE. That the processor does this is vital for the Memory Manager’s tracking of which pagable pages, i.e., pages of linear address space whose contents are subject to being paged in and out between physical memory and disk storage, have been modified. When the Memory Manager acts on a set Dirty bit, it clears not just this bit but also the Writable bit so that whatever it aims to do about the modified page cannot be interfered with by other processors (which temporarily see the page as having no write access). Meanwhile, the Write bit is the Memory Manager’s record of whether the page is eventually to have its write access restored. Windows Vista, which discontinues the single-processor kernels, formalises that the Dirty and Writable bits (the latter now named Dirty1) are cleared together.

64-Bit PTE

For 64-Bit Windows but also for the x86 builds that use PAE, page table entries are eight bytes and the MMPTE_HARDWARE is a structure of ULONGLONG bit fields. The low 12 bits of the 64-bit PTE, whether for PAE and x64, match closely those of the 32-bit PTE:

ULONGLONG Valid : 1;

ULONGLONG Write : 1;

ULONGLONG Writable : 1;

ULONGLONG Dirty1 : 1;

ULONGLONG Owner : 1;

ULONGLONG WriteThrough : 1;

ULONGLONG CacheDisable : 1;

ULONGLONG Accessed : 1;

ULONGLONG Dirty : 1;

ULONGLONG LargePage : 1;

ULONGLONG Global : 1;

ULONGLONG CopyOnWrite : 1;

ULONGLONG Prototype : 1;

ULONGLONG Unused : 1;

ULONGLONG reserved0 : 1;

ULONGLONG Write : 1;

Remember that “all” for PAE support begins with version 5.0. Similarly, the first x64 build is version 5.2 from Windows Server 2003 SP1.

For these low bits, the only difference from the 32-bit structure (for bits that aren’t reserved) is that version 6.1 removed Prototype from one but not the other. The remaining bits differ significantly, not just from the 32-bit PTE but between the PAE and x64 implementations:

ULONGLONG PageFrameNumber : 24;

ULONGLONG PageFrameNumber : 26;

ULONGLONG reserved1 : 28;

ULONGLONG reserved1 : 26;

ULONGLONG reserved1 : 25;

ULONGLONG NoExecute : 1;

In the first PAE kernels, the PageFrameNumber can describe 16M pages, as if for 36 address lines and 64GB of physical memory. Version 5.1 raises this to 64M pages, as if for 38 address lines. This would allow 256GB of physical memory, even though 32-bit Windows cannot possibly support so much. (It has a long-standing architectural limit of 128GB caused by needing kernel-mode address space for an array of MMPFN structures, one per page of physical memory. At 0x1C bytes per MMPFN, even 128GB of physical memory requires 896MB for the MMPFN array when at most 1GB can be available.)

It is not known why NoExecute is not defined for the PAE builds until a later Windows 10 release. The x64 builds have it from the start.

ULONGLONG PageFrameNumber : 28;

ULONGLONG PageFrameNumber : 36;

ULONGLONG reserved1 : 12;

ULONGLONG reserved1 : 4;

ULONGLONG ReservedForHardware : 4;

ULONGLONG SoftwareWsIndex : 11;

ULONGLONG ReservedForSoftware : 4;

ULONGLONG WsleAge : 4;

ULONGLONG WsleProtection : 3;

ULONGLONG NoExecute : 1;

The first 64-bit kernels provide for 256M pages, as if for 40 address lines and 1TB of physical memory. This was raised for the version 6.0 from Windows Vista SP1. The widened PageFrameNumber allows 48 address lines and thus 256TB of physical memory. Note that the PageFrameNumber in the otherwise very close HARDWARE_PTE does not get this same widening until the version 6.1 from Windows 7 SP1. It is not known whether the lag in updating the HARDWARE_PTE had real-world consequence.

In the high dword, 64-bit Windows defines SoftwareWsIndex as using all 11 bits that Intel leaves as available if the processor is not using protection keys.

MMPTE_HARDWARE_LARGEPAGE

The MMPTE_HARDWARE leaves unspecified that not all the bits of the PageFrameNumber are meaningful in a PTE for a large page. In a PTE for a 2MB page—or, if you prefer, a PDE for which LargePage is set—the lowest bit of the PageFrameNumber is Intel’s PAT and the next eight are reserved, i.e., must be zero. This applies also in 64-bit Windows to a PTE for a 1GB page, i.e., a PDPTE for which LargePage is set, except that nine more bits are reserved.

Early versions of the 64-bit kernel define an MMPTE_HARDWARE_LARGEPAGE to model this for 2MB pages:

ULONGLONG PAT : 1;

ULONGLONG reserved1 : 8;

ULONGLONG PageFrameNumber : 19;

ULONGLONG PageFrameNumber : 27;

ULONGLONG reserved2 : 24;

ULONGLONG reserved2 : 16;

This was discontinued for version 6.1, perhaps because it’s more trouble than it’s worth. It captures that the page frame number loses its low nine bits but it leaves the PageFrameNumber no longer equal to the page frame number.