Geoff Chappell - Software Analyst
The easier answer is of what I can’t do for you. I keep rigorously to a domain of expertise in low-level Windows programming and reverse engineering. Outside of this I cannot help you and I would not want to waste your time by pretending differently.
In case it’s not already clear from whatever page got you here, I work only with Windows and only then on the i386 and amd64 processor architectures. I specialise in what happens on the one Windows computer, whether client or server, but I tend to leave the networking of Windows computers to others (there apparently being no shortage). Within this one operating system, I work only at its lower levels—not necessarily kernel mode, but definitely low-level.
A rough measure of what counts as low can be had from the language you’d expect for the programming. I do all my programming and reverse engineering in C++, C and assembly language. Anything written in a higher-level language, or even to use some of the higher-level C++ frameworks, whether from Microsoft or others, is too high-level for me.
Given this context, the easy answer to what I can do for you really is easy: any problem of low-level Windows programming.
Some programming work with Windows just cries out to be done by a specialist who already knows how Windows behaves and what Windows expects—or, where this is not yet known, has proven ways of finding out quickly and reliably.
You perhaps already have experience of a Linux programmer, even your best Linux programmer, feeling confident to do some system-level Windows programming and even of seeming to do it well—and then, over time, it becomes clear that not everything was quite right or even that some things were very wrong. At some low-enough level in the hierarchy from the hardware up through your software and beyond to the generality of other people’s software that will depend on yours, the details are highly specific to the operating system. Get that programming done by a specialist in that operating system!
If you find yourself stepping into Microsoft’s code while debugging your own, you may soon be overwhelmed by the unfamiliarity of it. Consider that I’ve been immersed in this stuff for years. I don’t have Microsoft’s source code but their binary code is my source material. Much of what you see at this site is alternative documentation of Microsoft’s code. If you’re lost in that code, there’s every chance that I’ve been in there before and I can help you out of it now.
Help from afar is especially easy when the system has itself detected a problem, as when a user-mode program crashes from a fatal exception. Just send me the fullest crash dump you were able to collect and perhaps what symbol files you have for your own code in the faulting process. For kernel-mode driver problems that have caused a bug check (blue screen), just send me a mini-dump, your driver and a matching symbol file. Either way, just this information, plus a billing address, is enough nine times out of ten to get an explanation and even a fix.
A long-standing specialty is kernel-mode programming, especially of file-system filter drivers and of device drivers for disk I/O. For years there was not much sign of that specialty here at this site, because this site is for free publication of research into Windows for the public interest, not for showing professional (and thus confidential) work that’s done in your interest. But look now at the Kernel section!
One subject that is much on display at this site is the Windows Shell (including Microsoft’s adaptation of it to support Internet Explorer). Very few Windows programs of any substance do not depend on the Windows Shell. Yet much is undocumented and a lot more is under-documented. When new versions of Windows programs from Microsoft have a new, distinctive and even usefully attractive appearance, the reason has often been that Microsoft’s programs use something new (and only partly documented) in the Windows Shell. If you want that your programs can do some of the interesting things you see in, say, List-View groups and footers in the Windows Explorer from Windows Vista, then the large amount of information I have published here for free may be enough to get you going. But the nature of writing, let alone for a freely accessible website, is that I write up only a tiny proportion of what I find. If you want that the rest of what I find is put to use working for you, then you have to ask.
Although I regard as folly the notion that closed-source software can usefully be studied only by trying to reconstitute the missing source code, I haven’t been studying software for 30-something years without having developed some translation skills.
You are likely familiar with how machine translations of one human language into another often suffice for getting a rough idea of what is meant but do not begin to help with detail and fall far short of being useful for legal proceedings. For any text written in another language, if you want to read it with as close a sense as possible to reading it as intended, then you need a human translator. Indeed, you will need a good one, and will do for many years yet. So it is with software. The gobbledegook that’s spewed out by automated decompilers is impressive in its way and may even suffice for taking in a rough view, but it is just not readable for detail. For a properly crafted translation of x86 or x64 software into source code that actually does look like a human programmer wrote it, you need a human translator. Indeed, you will need a good one, and will do for many years yet.
If you’re in the unfortunate position of having lost the source code to your product, e.g., through contractual dispute, then few people anywhere will be as able to reconstruct readable maintainable source code for you.
If you’re in the better position of knowing that something’s wrong with someone else’s software that you are evaluating or already depend on, then consider what reverse engineering might do for you. If the software’s important but its problems are recurring, then reverse engineering (strictly for your own analysis) lets you add instrumentation that the manufacturer didn’t (or has kept from you). If you’re meeting with them about the problem, then reverse engineering can go a long way to rebalancing the asymmetry by arming you with reconstructed readable source code in which someone clearly comments the defects. You don’t get that from automated decompilers.
As malware becomes more sophisticated, there is only so much you can sensibly hope to learn from observation, however systematically and carefully you set about it. Even the anti-virus companies see some merit in what they call static analysis as their way to reverse-engineer the malware without running it. If you need to know everything that some malware can possibly do (or have done), then you need to pick apart its code. Even that won’t be enough for malware that works by interacting with Windows in some novel way. For true understanding, you must pick apart both the malware and the bits of Windows that it uses, abuses or exploits. To get that done comprehensively and reliably, ask someone who has made a point of practising it for years.
Much the same applies to the reverse-engineering of proprietary file formats. If you want a competitor’s customers to become your customers, you typically have to overcome the huge obstacle that those customers have stored their work in your competitor’s file format. Your software will need at least to understand that format well enough for reliable conversion to yours.
The apparently traditional way to discover details of an undocumented file format is to create a wide variety of files in more-or-less controlled circumstances and hope to spot what data turns up where. And I must admit that this does get you much of the way, if only after lots of fumbling and who knows how many iterations. But if you need to know everything about the file format, then there is no substitute for picking apart the code of whatever program is the definitive creator or interpreter of files in this format. To get that done comprehensively and reliably, ask someone who has made a point of practising it for years.
For the little while longer that PCs have a ROM BIOS to start executing code from disk in real mode, the exotic niche of real-mode programming just doesn’t want to disappear. Yet efficient coding in real mode is a skill that many people lost years ago, if they ever had it. Even though I, the author of DOS Internals long ago, haven’t written anything for MS-DOS in years, I have kept those real-mode skills alive on commercial software that is written to run from boot sectors before the Windows loader. If you’re in that niche of still needing to run code in real mode, I can help.