Missing Settlement Functions

As a condition of the Final Judgment in United States v. Microsoft, CA No. 98-1232, Microsoft is required to disclose “the APIs and related Documentation that are used by Microsoft Middleware to interoperate with a Windows Operating System Product”. Late in 2002 Microsoft presented a collection of Settlement Program Interfaces as satisfying this requirement. Little or no explanation was given of the criteria by which these interfaces were selected for documentation to comply with the settlement. The documentation itself is extraordinarily grudging and miserly. Rarely is any overview of the functionality provided. For many functions, all that Microsoft offers is such little information as a competent reader might discern from uncommented C-language prototypes. Moreover, Microsoft very often misrepresents how old some of these functions truly are. Many are said to have been introduced with Windows 2000, conveniently obscuring whether their presence in earlier years gave Microsoft an anti-competitive advantage while developing Internet Explorer.

Among the outcomes of the present survey of Windows shell functions are a list of API functions that three important software modules are known to make accessible for use by other modules and for each function a list of other modules that are known to use the function. These lists would seem well suited to a quick test of whether Microsoft’s API disclosure is credibly complete.

Of course, completeness must be assessed in the sense intended by the judgement. Not all previously undocumented API functions have to be documented for compliance with the settlement, just the ones “used by Microsoft Middleware to interoperate with a Windows Operating System Product”. To test the compliance, I extract from the survey’s lists each function that:

1. is supplied by a module that qualifies as a component of a Windows Operating System Product;
2. qualifies as an API;
3. has among its known callers at least one module that qualifies as a component of Microsoft Middleware;
4. and is (still) not documented by Microsoft after the settlement.

The ideal expectation is that no function in the survey should satisfy all these conditions. Note especially that the Joint Status Report on Microsoft’s Compliance with the Final Judgments, dated 3rd July 2003, reports to the court that “to identify the relevant interfaces, Microsoft developed new software tools and drew upon the expertise of more than 50 Windows software developers and program managers.” This seems intended to convey an impression of substantial work, perhaps even of thoroughness, specifically just to enumerate APIs for disclosure. It seems reasonable to expect that all this effort will have got Microsoft very close to the ideal.

Even if some few functions satisfy all the conditions, Microsoft surely has something to explain, if only that slight inaccuracies are inevitable, no matter how hard one works or how much care one takes. If hundreds of functions satisfy all the tests, which they do, then although no appeal to occasional error in good faith would be credible, much might be explained as Microsoft having applied some different interpretation, so that Microsoft enumerated APIs for disclosure by testing against very different conditions than listed above. Against that, since Microsoft doesn’t present any conditions that it tested for, nor say anything in public that might help an independent assessment of its methods, one might reasonably wonder whether the omissions amount to “knowing, willful or systematic violations”, especially from having calculated the likelihood of never being called to account.

If you tire of the text between here and a list of functions that I say Microsoft has missed, then please consider that I take the trouble to explain the interpretations that I test against: Microsoft does not.

Windows Components

To apply the first test to the survey is to ask which of the three shell modules (COMCTL32.DLL, SHELL32.DLL and SHLWAPI.DLL) that supply the surveyed functions qualify as components of a Windows Operating System Product. I contend that the only sensible answer is that all three do.

All three are distributed with all versions of Windows 2000 and Windows XP. The edition used for the survey is Windows XP Service Pack 1a, which the settlement lists explicitly as the one by which compliance should be demonstrable.

Moreover, all three of these modules are used for numerous everyday Windows activities that have little or nothing to do with Internet access—and by saying “little” I mean to allow that some might contend such things as that using a JPEG file for the background of a listing of files on the local computer is sort of an Internet activity because JPEG files are used widely for HTML pages and the rendering of images from JPEG files is naturally the work of Internet Explorer even in the absence of an Internet connection or even of an HTML page.

Not only are COMCTL32, SHELL32 and SHLWAPI distributed in the Windows package and essential for all practical purposes to using Windows without the Internet, but Microsoft has itself listed all three as core components of Windows. Admittedly, Microsoft seems to have done this only for earlier versions of Windows than specified in the judgement. See for instance the MSDN Library from January 1997 for a list of Core Components given as an appendix to the Designed for Microsoft Windows NT and Windows 95 Logo Handbook for Software Applications. Curiously, this appendix does not persist in later descriptions of the logo requirements.

Two of these modules, namely COMCTL32 and SHLWAPI, have also been listed by Microsoft as core components of Internet Explorer. That Internet Explorer, among any number of web browsers supposedly in open competition for customers, gains unfairly from having privileged access to core components of Windows is, to many people, a good brief summary of how all this trouble started.

It’s possible, of course, that Microsoft genuinely does believe that the judgement leaves to Microsoft a freedom to choose whether an executable, or perhaps even a fragment of an executable, is a component of a Windows Operating System Product or of a Microsoft Middleware Product. If so, then Microsoft’s thinking on this is specious, at best. For the relatively few SHLWAPI functions that Microsoft does document, Microsoft’s practice has always been, and still is, to place them under the heading Windows Shell. To say that they are not Windows API functions for the purpose of the settlement would be to go against a decade of Microsoft’s own practice.

APIs

Applicability of the second test is trivial. All functions in the survey are formally exported by the three shell modules in question. Each is therefore locatable and callable from any other module that knows what to locate and how to call it. That Microsoft does not document all such functions as Application Program Interfaces or document how to use the functions (at all, let alone safely) does not mean that the functions are not APIs.

Middleware Components

To apply the third test, examine the list of modules that are known to use the function. The present interest is with functions for which the list of known users includes even one module that is distributed not only with Windows but also with some other Microsoft product that qualifies as Microsoft Middleware.

If a function has no known users, it is eliminated. Most likely, the function is obsolete.

A function is also eliminated if all its known users seem to be distributed only with Windows. Specially prominent examples of such modules are COMDLG32.DLL, EXPLORER.EXE and SHELL32.DLL. Less prominent are PRINTUI.DLL, PROGMAN.EXE, SYNCUI.DLL and TASKMAN.EXE. These seven examples suffice for eliminating numerous shell functions as being in some sense internal to Windows.

An open question in my mind arises from the judgement’s definition of Microsoft Middleware as excluding any “software code described as part of, and distributed separately to update, a Microsoft Middleware Product”. Does this apply to modules that are redistributed with other products apparently only as setup tools, rather than as lasting elements of the installed product? Of particular interest among such setup tools is GRPCONV.EXE, because it uses surprisingly many undocumented shell functions. I have chosen to treat a function’s use by GRPCONV.EXE as an internal Windows matter, not as use that qualifies under the judgement.

Note that all the eliminations so far are generous to Microsoft’s compliance (in the sense that they lower the number of functions that would have to be documented).

That a user is not known for some function does not mean that the function is not used. The known users obtained for the survey are those for which Microsoft distributes symbol files as customer support for Windows, and the modules listed as users of any one function are those that the symbol files show as formally importing the function. A function may be used in some way that does not generate the corresponding import symbol. Such other ways certainly are known, and some are arguably exotic: for instance, the SHLWAPI function EventTraceHandler is used by MSHTML.DLL because the former creates a named memory mapping in which to store the function’s address and the latter knows the name of the memory mapping. For the tables below, I leave such exotica alone, so that they do not count as usage even though they are arguably the most egregious examples.

What remains after the preceding eliminations is a list of functions each of which has at least one known importer that is distributed with some Microsoft product other than Windows. Moreover, it turns out that for every remaining function, at least one of the known importers is a module that is distributed with Internet Explorer (which I take to include Outlook Express), except for some few modules that support the Microsoft Network (MSN). Indeed, almost all the remaining functions are imported by either BROWSEUI.DLL or SHDOCVW.DLL, and a few more by MSHTML.DLL.

SHDOCVW

If anything, SHDOCVW is not just part of Internet Explorer: it is the main software code for Internet Explorer. There is even a literal sense to this. Some users start Internet Explorer by running the program IEXPLORE.EXE, and some even consider that this program is the essence of the product. However, IEXPLORE has long been little more than a front end for executing SHDOCVW, and the way that this execution is arranged is that IEXPLORE calls a function named IEWinMain in SHDOCVW (exported as ordinal 101 and apparently, if unsurprisingly, not documented).

Another measure by which SHDOCVW is the essence of Internet Explorer is provided by Microsoft’s own advice on How to Determine Which Version of Internet Explorer is Installed, particularly where a “table lists the different versions of the Shdocvw.dll file and the corresponding versions of Internet Explorer”.

BROWSEUI

There may be more debate about BROWSEUI. For instance, as a rough analogue of SHDOCVW’s providing a sort of WinMain function for Internet Explorer, BROWSEUI has a function named SHParseIECommandLine (exported as ordinal 125 and apparently undocumented) that gets the job of parsing the IEXPLORE command line. Surely that marks BROWSEUI as working specifically for Internet Explorer. However, BROWSEUI parses the command line for the Windows Explorer too, through the function SHExplorerParseCmdLine (exported as ordinal 107 and also apparently undocumented). On this matter, and many others, BROWSEUI behaves in part as a component of Windows and in part as a component of Internet Explorer. Is BROWSEUI one or the other, in terms of the judgement’s definitions, or is it both?

A good case could be made for both, but if I have to choose one or the other, not that the terms of the settlement require it, then it’s as well to follow Microsoft’s own practice for how it has arranged files in the retail product. Several large executables that are supplied with Windows have most of their localised resources separated into a file whose name ends in LC. For instance, the resource file for BROWSEUI.DLL is BROWSELC.DLL. A copy is in the Windows SYSTEM32 directory, with the executable, but the resources in this BROWSELC are for what’s called the system default UI language. If you have installed multi-language support, you may have a BROWSELC for one or more other languages. Where (for Windows XP) does Microsoft install such other copies of BROWSELC? The answer is: in subdirectories of the Internet Explorer installation, e.g., beneath “C:\Program Files\Internet Explorer”. How can BROWSEUI not be regarded as a component of Internet Explorer for the settlement, when Microsoft itself places BROWSEUI’s localised resources as belonging with Internet Explorer?

Perhaps the last word comes indirectly from what Microsoft has chosen to document about BROWSEUI for compliance with the settlement. If BROWSEUI is not a component of Internet Explorer, then presumably it is a component of Windows. Its use of undocumented shell functions would then be an internal Windows matter and would not qualify for documentation under the judgement. But if this is so, then the functions that BROWSEUI exports and Internet Explorer calls, e.g., SHParseIECommandLine, ought to have been documented among the Settlement Program Interfaces—and none were. Microsoft cannot have it both ways. At least one of the following is required of BROWSEUI, and Microsoft did neither:

• the Windows functions called by BROWSEUI should all be documented;
• the BROWSEUI functions called by Internet Explorer should all be documented.

Documentation

The last test is whether the function is documented. A function’s status with respect to Microsoft’s documentation has no controversy. The judgement cites the Microsoft Developer Network (MSDN) specifically and the choice of reference for this survey is the January 2004 edition, already at least a year later than Microsoft’s supposed compliance. If a function was not documented by then, yet ought to have been, then Microsoft has a compliance problem to explain and put right—and the compliance authorities create a hostage for the future if they don’t explain the process in public.

Lack of explanation, whether by Microsoft or the compliance authorities, is especially striking where functions, including some ancient ones, have been documented since the original announcement of supposed compliance. This post-settlement documentation was not done all at once, but as a process of apparently irregular updates which Microsoft does not describe as rectifying any incompleteness of the initial disclosure.

Tracking this programme of post-settlement documentation should be the (paid) work of the compliance authorities. I cannot track it in detail. The lists below give as much accuracy as I happen to know from occasional queries at Microsoft’s website. Mostly however, I resort to the phrase “documented in 2004-2006” as meaning simply that a function wasn’t documented in the January 2004 edition of the MSDN Library on CD but is documented in the Windows Software Development Kit (SDK) for Windows Vista on a CD dated January 2007.

Function Lists

All up, I count 363 shell functions as passing all four tests, so that they ought to have been documented by Microsoft but were not. A large proportion are perhaps dismissible on the grounds that they repackage otherwise well-documented Windows API functions—but even after dismissing these, I still count 196.

The tables that follow are extracted from the detailed lists of known users of each module: COMCTL32 Usage, SHELL32 Usage and SHLWAPI Usage. They show for each function one or more modules that are known to use the function and which I regard as sufficient to qualify the function for documentation under the judgement. These modules are represented according to the following key.

Unless some argument exists that BROWSEUI.DLL and SHDOCVW.DLL (and to a lesser extent, MSHTML.DLL) are not components of Internet Explorer, or that COMCTL32.DLL and (especially) SHLWAPI.DLL are not components of Windows, one can only wonder at how Microsoft can possibly have omitted from the Settlement Program Interfaces more “shell and common controls” functions than it documented.

COMCTL32 Functions

Microsoft documented 17 COMCTL32 functions as Settlement Program Interfaces. I count another 17 functions as omitted (and for one more, both the ANSI and Unicode forms ought to qualify but Microsoft documented only the Unicode form).

These missing functions are exported from the COMCTL32.DLL that shippred with Windows XP Service Pack 1a, are each used by at least one of the following three DLLs that were all distributed with that same version of Windows and with the roughly contemporaneous Internet Explorer 6.0:

• browseui.dll
• shdocvw.dll (required only for counting the ANSI form of Str_SetPtr as missing)
• webcheck.dll (required only for counting DPA_CreateEx as missing)

yet were not documented by Microsoft as late as the January 2004 edition of the MSDN Library on CD. Note, however, that they have all been documented subsequently.

SHELL32 Functions

Microsoft listed among the Settlement Program Interfaces 110 “shell functions”, almost all of which actually are implemented in SHELL32. Hundreds more SHELL32 functions remain undocumented, but I accept that all but 2 seem either to be used only internally within Windows or to be not used at all. Again, for one more function, Microsoft has documented only the Unicode form, even though both ANSI and Unicode forms surely qualify.

The functions shown below as missing from the Settlement Program Interfaces are exported from the SHELL32.DLL that shipped with Windows XP Service Pack 1a, are each used by at least the following DLL that was distributed with that same version of Windows and with the roughly contemporaneous Internet Explorer 6.0:

• browseui.dll

yet were not documented by Microsoft in the January 2004 edition of the MSDN Library on CD.

SHLWAPI Functions

It is with SHLWAPI that Microsoft has, if only as I see it, seriously and perhaps culpably undercounted. Microsoft lists only one SHLWAPI function (namely SHGetViewStatePropertyBag) among the Settlement Program Interfaces. That this might be anywhere near complete is not even remotely credible if SHLWAPI is regarded as a Windows component used by Internet Explorer.

Against this is that Microsoft perhaps deems SHLWAPI to be a component of Internet Explorer but somehow not a component of Windows. How else, for instance, does the SHAboutInfo function, which very obviously exists only to support the About dialog in Internet Explorer, escape attention as a function in a component of Windows getting called from a component of Internet Explorer?

With this thinking, bizarre as it would seem to many, including to whoever at Microsoft documents SHLWAPI functions in the section for Windows rather than for Internet Explorer, no SHLWAPI functions used by Internet Explorer would need to be documented for the settlement. Inclusion of SHGetViewStatePropertyBag might have been necessary only because it is used by some Middleware product other than Internet Explorer. Or perhaps it is just that Microsoft was in this one case more generous than the settlement requires. Until Microsoft condescends to explain its thinking, which might never happen unless asked by a court, we will never know.

Meanwhile, if I take SHLWAPI as a component of Windows for the purposes of the settlement (even if it is also a component of Internet Explorer), then I count 344 SHLWAPI functions as omitted. Roughly half are so-called wrappers of otherwise well-known API functions, mostly to present a Unicode interface even if the underlying implementation will convert to and from ANSI. Microsoft documents a Microsoft Layer for Unicode (MSLU) that does something similar, though by a significantly different method. Whatever the advantages and disadvantages, these long-standing wrappers in SHLWAPI still find very heavy use by all sorts of modules, including many that surely qualify as components of Microsoft Middleware. Since these wrappers form a set and since many will think them a very uninteresting set, I present them separately, later.

Discounting the wrappers for now, there remain 177 functions that are exported from the SHLWAPI.DLL that shipped with Windows XP Service Pack 1a, are each used by at least one of the following DLLs that were all distributed with that same version of Windows and with the roughly contemporaneous Internet Explorer 6.0:

• browseui.dll
• iedkcs32.dll (required only for counting SHGetRestriction as missing)
• mshtml.dll
• msieftp.dll (required only for counting SHMessageBoxCheckEx as missing)
• msnmetal.dll (required only for counting SHCancelTimerQueueTimer as missing)
• msrating.dll (required only for counting SHGlobalCounterIncrement as missing)
• shdocvw.dll
• urlmon.dll (required only for counting StrCmpNC as missing)

yet were not documented by Microsoft in the January 2004 edition of the MSDN Library on CD.

Consultation of the MSDN Library on-line in late 2004 revealed that some of the functions in the table below have been documented since this study was first imagined (and the January 2004 edition chosen as the initial point of reference). More have been documented since. This still leaves more than a hundred as missing.

Unicode Wrapper Functions

The Unicode wrapper functions exported from SHLWAPI are numerous. The January 2004 edition of the MSDN Library on CD troubles to document only nine of them. Since nine is not zero, it is not as if Microsoft has always discounted these wrappers as being entirely unworthy of comment. None, however, were documented for the settlement.

The 167 wrapper functions listed below are all exported from the SHLWAPI.DLL that shipped with Windows XP Service Pack 1a, are each used by at least one of the following DLLs that were all distributed with that same Windows version and with the roughly contemporaneous Internet Explorer 6.0:

• browseui.dll
• iepeers.dll (required only to count StartDocWrapW as missing)
• inetcomm.dll (required only to count DrawTextExWrapW as missing)
• mshtml.dll
• msxml3.dll (required only to count IsCharUpperWrapW as missing)
• shdocvw.dll
• wininet.dll (required only to count SHFlushSFCacheWrap as missing)

yet were not documented by Microsoft as late as the January 2004 edition of the MSDN Library on CD.

Many, but nowhere near all, have been documented since, not by the end of 2004 but in time at least for the January 2007 edition of the Windows Vista Software Development Kit (SDK). Microsoft’s selection is at best capricious. How, for instance, does anyone with the slightest appreciation of logic and consistency decide to document IsCharUpperWrapW but leave IsCharLowerWrapW alone, except if instructed to document only those functions that meet some very strict criteria however absurd may be the outcome?