Geoff Chappell - Software Analyst
Evidently, I don’t get much done these days. Among the many pointless diversions that have nothing to do with work (except for interfering with it) and the several welcome diversions for paid work, I also got diverted by a report on Twitter about Windows hanging when one process reserves all of another’s virtual address space.
There is indeed a long-standing problem of spectacularly poor performance in this particular case of resource exhaustion. Especially interesting is that what is ordinarily a useful aid, namely the VAD bitmap, becomes the problem. At any given time, some of the user-mode address space has its allocation status optimised into the VAD bitmap for ready access. On searching for clear bits but finding none, the Memory Manager expands the bitmap to account for more of the address space, and then repeats its search. In the problem case, however, all the address space that is newly optimised into the bitmap is reserved. All the new bits are set. Each repeated search still finds no clear bits and the Memory Manager falls into a cycle of expanding and re-searching, ever slower as the bitmap gets ever larger. Eventually, the bitmap is as large as it can possibly be. It has “optimised” allocation from the whole of user-mode address space. The expansions stop and allocation fails. There isn’t really a hang, but the realisation that allocation must fail takes orders of magnitude more time than any real-world user should ever have patience for.
Or so goes my rough summary. Plainly Microsoft has something to look into there and the computer industry’s security researchers might usefully reopen discussion of when misbehaviour from exhaustion of resources becomes a security issue.
But that’s all to be thought through by the many who get do this as salaried work. For this website, it’s just a little story of what it was that got me referring to my notes on the Memory Manager and to what I’ve yet published of them.
It turns out that I, and apparently everyone else who writes on the Internet about Windows, have largely left the VAD bitmap alone. Even my own notes have very little on it explicitly. At first this suprised me, since of course I know of the VAD bitmap well enough to recognise it immediately in the observed problem—and it must be as well known to others, too. Against this is that it’s not fundamental to an understanding of virtual address space in Windows. It’s just an optimisation of the allocation algorithms. This perhaps is just far enough away from the nitty gritty to remove it from routine attention.
Also looking neglected, and again not just by me, is the relatively recent but sweeping re-architecture of the Working Set List. The start of this for the 1607 edition of Windows 10 is recorded among my own published pages, but I never returned to it. Obviously there’s much here that could do with some fresh attention!
Of course, the need for fresh attention applies not just to my pages about memory management. Add that I have since mid-2020 aimed to rearrange my pages on kernel structures around what the public symbol files tell of which Microsoft’s headers define which structures, and I fear I shall easily be overwhelmed. Still, I’ll try to do what I can. Who’s to know how long this revisit will hold my attention against all those other diversions, but let’s see where it runs…
One place it runs to is memory partitions. These were new for Windows 10 and back in 2016 I thought them important enough to venture an attempt at documentation such as might be called alternative were there any from Microsoft to be an alternative to. As often, nobody seems to have agreed with me. As far as I could tell from readers’ enquiries or from anything Google showed of take-up by other writers, my writing about memory partitions did nobody any good. Then I started getting the occasional alert to “correct” or “fix” my work. It is of course nice to know the work gets read eventually and I certainly don’t want to dissuade readers from sending wish lists, but neither do I see that work dated 2016 needs correction for changes that Microsoft makes in the years since. It needs updating, sure, but not correction. The serious point, though, is not whether the work needs either. It is instead why new work on the topic is still so hard to find that anyone should want that old work cannot simply be left to lie in peace. What has gone wrong that an industry of security researchers, etc., has somehow not funded the development of better resources?