12 May 2017

VMProtect and dbghelp.dll bug in export processing

If your Olly is crashing when loading executable protected by VMProtect, you most likely have outdated dbghelp.dll somewhere on your path. Grab the latest version from Microsoft and put it in the Olly folder.

Well, that might be enough to work around the issue that I had - but I still wanted to know what's causing the crash.

Cause of the problem

If you try to debug Olly with another Olly, you'll see the Access Violation happening somewhere in dbghelp.dll:

Check register values in Olly:

For some reason, value in EDX is garbage and therefore access violation happens.

Call stack doesn't tell us much:

And same piece of code in IDA doesn't help much either:

So, it's debugging time! Set breakpoint to start of LoadExportSymbols, then set hardware breakpoint on write to address [ebp+ptrAllocatedMemory].

First hit is initialization of variable with 0:

Second hit stores the address of allocated memory:

And third time is a charm:

Good folks at Microsoft have left us with a nice buffer overflow. exportFunctionName is defined as byte array of size 2048 bytes. Any exported function name longer than that will cause stack overflow and (possibly) subsequent crash.

010Editor with PETemplate confirms that the export name is indeed very long (3100 chars):

From what I can tell, it's a similar (but not the same) bug to what was described by j00ru at http://j00ru.vexillium.org/?p=405 (see "PE Image Fuzzing (environment + process)")

Stay safe!

P.S Here's an example file, if you want to test your Olly: https://forum.tuts4you.com/topic/38963-vmprotect-professional-v-309-custom-protection/
P.P.S. CFF Explorer, HIEW and IDA do not show us any exports in this example file - but that's a matter of another story..