15 May

Unity3D, Mono and invalid PE files

Some time ago, Reoto asked a very nice question on Black Storm forum:

Can someone fix the .dll (.net) pe header to MS DOS?
How can I do that?
If you know about protecting .net files for Android, please help me.
I have another question.
Can I fix dnspy to resolve .dll pe header isn't .net?

Obviously, English is not author's first language but it seemed like an interesting problem, so I decided to look into it.

Here is one of the files in question: https://mega.nz/#!0g4VHaIR!KmpQirte4_3lv8MSxyjETiufjFGb-CITpFGrXwxSgGY

TL;DR: Mono loader used by Unity3D accepts invalid PE files. It can be used to break most .NET decompilers. dnlib and tools based on dnlib (dnSpy, de4dot) were updated on 20-Apr-2018 but the rest of the tools still can't handle such files.

Quick background on Unity3D and Mono

I quickly checked file in CFF and it looked like the file doesn't have proper PE header.

I fixed that. But even then it was not recognized as a .NET file.

So, the file is clearly invalid, yet it works just fine in Android! How is that possible?

Well, Android has no clue about PE files or .NET Framework. When you build your program in Unity3D and deploy it to Android, it uses Mono to run your code. Mono is supposed to be open-source alternative of .NET Framework. And it is incredibly buggy.

I'm not Unity3D or Android wizard but the whole monstrosity works something like this:

To make matters worse, even current versions of Unity3D are using a very old Mono version. Like 3+ years old. You can easily tell it by looking at the PE loader error messages. This is how it looks in IDA:

The commit df51163 is clearly missing.

Therefore, for the rest of the article I'll be using Mono sources from commit 74be9b6, so that they would more or less match the code used by Unity3D.

Cause of the problem

A good place to start looking for bugs would be Mono implementation of PE loader:

do_mono_image_load calls function mono_verifier_verify_pe_data which should filter out any invalid PE file and stop loading it. For some reason (which I really don't care about), this function does nothing in Unity3D and returns success.

After that, buggy mono_image_load_pe_data takes over and uses PE parser to load PE structures and .NET metadata. It is followed by equally buggy processing of .NET metadata in both mono_verifier_verify_cli_data and mono_image_load_cli_data.

But first things first..

Invalid PE signature

First of the errors is inside do_load_header. It gets called indirectly from mono_image_load_pe_data:

It checks only first 2 bytes of PE signature, so you can change it to "PE\0\1" or "PEPE" and it will still work under Mono.

Invalid NumberOfRvasAndSizes value

Next bug is located get_data_dir used indirectly by mono_verifier_verify_cli_data.

and get_data_dir looks like this:

Both verify_cli_header and get_data_dir ignore NumberOfRvasAndSizes field in PE header and access "CLR Runtime Header Entry".

Identical bug is in load_cli_header called from mono_image_load_cli_data:

They do not use broken get_data_dir function but the check of NumberOfRvasAndSizes field is still missing. That's why our PE file can have NumberOfRvasAndSizes == 0xA and it still works under Mono.

Invalid metadata size

Mono does a very limited checking of .NET metadata size in load_metadata_ptrs.

However, this check is insufficient. For example, you can set .NET metadata size = 0, most of .NET reversing tools will break but Mono will happily accept the file.

Invalid number of .NET streams

.NET metadata streams are loaded by load_metadata_ptrs.

You can use an arbitrary large number of streams in .NET Metadata header, as Mono will ignore all the invalid data. It probably will spam Android log with "Unknown heap type" messages, but the file will still run.

Invalid number of rows in .NET tables

Final nail in the coffin is the incorrect processing on .NET metadata tables. Tables are loaded in load_tables method which seems to be correct on the first look:

However, the definition of .NET metadata table information (_MonoTableInfo) is wrong:

As a result of this definition, they ignore high-order byte in row count. You can set number of rows to, say, 0xCC000001, .NET metadata will be invalid but Mono happily accepts the file. WTF?

Extra protection by AndroidThaiMod

Game hacks created by AndroidThaiMod.com take the Unity3D/Mono abuse to the next level. They exploit all the bugs I already mentioned. In addition to that, they replace the original libmono.so with their own version. In their version there are few extra lines of code in the function mono_image_open_from_data_with_name:

This change allows AndroidThaiMod to distribute their DLLs encrypted with "extra-strong" XOR encryption. smile

Another fun fact - their cheats only hack ARM version of Unity3D runtime. x86 version, even if present in original APK, gets removed from the hacked APK. So, if you have an Android device with x86 CPU, you're out of luck - AndroidThaiMod cheats won't work there.

Few AndroidThaiMod files I was able to find on Google:
https://drive.google.com/file/d/1g2_43rP9LLrXSmGL-Lw36GTehSUmg3CJ/view
https://drive.google.com/file/d/0B7jRiqM-QmgUeUF2RldrRUFmcUk/view

Other possibilities of abuse

As I mentioned, Mono PE loader is extra buggy. Pretty much all fields in PE header are not validated properly. Plenty of fields in .NET structures are ignored. PE32+ header processing is broken beyond belief. File/section alignment is not enforced. You can have several streams named "#~" and Mono will happily use one of them. Or you could just search for a phrase "FIXME" in Mono sources - you'll find lots of other dirty hacks that can be exploited.

Can it be fixed?

Probably. But considering how broken and messy the entire Mono codebase is, I wouldn't bet on it.

For example, on 13-Apr-2018 Mono developers made this awful commit called "Verify all 4 bytes of PE signature.":

There was no explanation why it was done, what are the side effects and whether they plan to fix all the broken code or if this was just a one-off fix. What a surprise!

And, as I already mentioned, Unity3D is using 3+ years old version of Mono. So, you'll probably have to wait until year 2021 until Unity3D gets they necessary fixes. bigsmile

Workarounds

Even if Mono and Unity3D was fixed, it doesn't help us to analyze existing broken files.

One workaround was to update dnlib (commit c40e148) and reversing tools to support such broken files. It allows to analyze existing files using dnSpy and de4dot. But Reflector and other tools still won't work. So, I decided to make my own tool which will fix broken files and will allow you to use any .NET reversing tool you like.

I needed a PE/.NET reading library which is very simple and low-level. dnLib/Mono.Cecil is way too abstract. After a quick search, I decided to use parts of McCli - but any other library would do just fine.

The code is very straightforward. Read PE structure->validate->fix->repeat.. wink

Conclusion

Games and cheats made using Unity3D are great. It's possible to make very unique protections that will stop most Windows/.NET reversing tools but the program will still work on Android. Enjoy the fun times! smile

Unity3D fixer with source code: https://mega.nz/#!g4A3WajL!PBRF5f6OIj-0xkm09Zg2T_YydcmCW2XtYI9Di82aIJ8

02 Jun 2015

Since you asked.. How to inject byte array using dnlib

Quite often I receive random questions about dnlib from my friends. To be honest, I have no idea why they think I know the answers to life the universe and everything else. smile So, in this series of posts I'll attempt to solve their problems - and hope that the solution helps someone else too.

So, today's question is:

We're trying to add a byte array to an assembly using dnlib. We wrote some code* but dnlib throws exception when saving modified assembly:
An unhandled exception of type 'dnlib.DotNet.Writer.ModuleWriterException' occurred in dnlib.dll
Additional information: Field System.Byte[] ::2026170854 (04000000) initial value size != size of field type

I gave the friend the standard answer - make a sample app, see how it looks and then implement it with dnlib. Seriously, how hard can it be? smile

Well, array initialization in .NET is anything but simple.

How arrays are initialized in C#

Note - the following explanation is shamelessly copied from "Maximizing .NET Performance" by Nick Wienholt. It's a very nice book but getting little outdated. You can Google for "Apress.Maximizing.Dot.NET.Performance.eBook-LiB", if interested.

Value type array initialization in C# can be achieved in two distinct ways—inline with the array variable declaration, and through set operations on each individual array element, as shown in the following snippet:

For a value type array that is initialized inline and has more than three elements, the C# compiler in both .NET 1.0 and .NET 1.1 generates a type named <PrivateImplementationDetails> that is added to the assembly at the root namespace level. This type contains nested value types that reference the binary data needed to initialize the array, which is stored in a .data section of the PE file. At runtime, the System.Runtime.CompilerServices.RuntimeHelpers::InitializeArray method is called to perform a memory copy of the data referenced by the <PrivateImplementationDetails> nested structure into the array's memory location. The direct memory copy is roughly twice as fast for the initialization of a 20-by-20 element array of 64-bit integers, and array initialization syntax is generally cleaner for the inline initialization case.

Say what? You can read the text 3 times and still be no wiser. So, let's make a small sample application and disassemble it.

How array initialization looks in MSIL

Let's start with sample app that does nothing.

Compile without optimizations, and disassemble using ildasm. And even after removing all extra stuff, there's still a lot of code & metadata for such a simple thing. smile

For one byte array that we declared, compiler created .data directive, 2 static fields, one class and one nested class. And it added a global static constructor. Yikes!

Implementing it in dnlib

Now that we know all the stuff that's required for an array, we can make a tool that will add byte array to an assembly of our choice. To make things simpler, I decided not to create a holder class (named <PrivateImplementationDetails>{E21EC13E-4669-42C8-B7A5-2EE7FBD85904} in the example) and put everything in global module instead.

Note - Since I'm not a .NET/dnlib wizard, I always do it one step at a time, make sure it works and then continue. So, my workflow looks like this: write a code that does X → compile and run it → disassemble the result → verify that result X matches the expected → fix the bugs and repeat. Only after I've tested one thing, I move to the next one.

It also helps to make small test program first. Once you know that your code works as intended, you can use it in a larger project. Debugging the entire ConfuserEx project just to find a small bug in modifications made by someone - it's not fun! So, step-by-step...

First, we need to add the class with layout. It's called '__StaticArrayInitTypeSize=5' in the example above. That's quite simple to do in dnlib:

Now we need to add the static field with data, called '$$method0x6000003-1'.

Once that is done, we can add our byte array field, called bla in the example.

That's it, we have all the fields. Now we need to add code to global .cctor to initialize the array properly.

And that's it! Simples!

Further reading

Commented demo code at Pastebin
Longer explanation how array initialization works in C#


Updates

Just to clarify - this is a sample code. It works for me but if it blows up in your project, it's your problem. And there always are some things that can be improved.

• Sometimes I'm overcomplicating things.. You don't need to explicitly import System.Byte, you can use mod.CorLibTypes.Byte for that.

SZArraySig is a cleaner but less obvious way to refer to any array. If you need to reference complex arrays, this is better: