Unity3D, Mono and invalid PE files

Some time ago, Reoto asked a very nice question on Black Storm forum:

Can someone fix the .dll (.net) pe header to MS DOS?
How can I do that?
If you know about protecting .net files for Android, please help me.
I have another question.
Can I fix dnspy to resolve .dll pe header isn't .net?

Obviously, English is not author's first language but it seemed like an interesting problem, so I decided to look into it.

Here is one of the files in question: https://mega.nz/#!0g4VHaIR!KmpQirte4_3lv8MSxyjETiufjFGb-CITpFGrXwxSgGY

TL;DR: Mono loader used by Unity3D accepts invalid PE files. It can be used to break most .NET decompilers. dnlib and tools based on dnlib (dnSpy, de4dot) were updated on 20-Apr-2018 but the rest of the tools still can't handle such files.

Quick background on Unity3D and Mono

I quickly checked file in CFF and it looked like the file doesn't have proper PE header.

I fixed that. But even then it was not recognized as a .NET file.

So, the file is clearly invalid, yet it works just fine in Android! How is that possible?

Well, Android has no clue about PE files or .NET Framework. When you build your program in Unity3D and deploy it to Android, it uses Mono to run your code. Mono is supposed to be open-source alternative of .NET Framework. And it is incredibly buggy.

I'm not Unity3D or Android wizard but the whole monstrosity works something like this:

To make matters worse, even current versions of Unity3D are using a very old Mono version. Like 3+ years old. You can easily tell it by looking at the PE loader error messages. This is how it looks in IDA:

The commit df51163 is clearly missing.

 		if (table > MONO_TABLE_LAST) {
-			g_warning("bits in valid must be zero above 0x2d (II - 23.1.6)");
+			g_warning("bits in valid must be zero above 0x37 (II - 23.1.6)");
 		} else {
 			image->tables [table].rows = read32 (rows);
 		}

Therefore, for the rest of the article I'll be using Mono sources from commit 74be9b6, so that they would more or less match the code used by Unity3D.

Cause of the problem

A good place to start looking for bugs would be Mono implementation of PE loader:

static MonoImage *
do_mono_image_load (MonoImage *image, MonoImageOpenStatus *status,
		    gboolean care_about_cli, gboolean care_about_pecoff)
{
	MonoCLIImageInfo *iinfo;
	MonoDotNetHeader *header;
	GSList *errors = NULL;

	mono_profiler_module_event (image, MONO_PROFILE_START_LOAD);

	mono_image_init (image);

	iinfo = image->image_info;
	header = &iinfo->cli_header;
		
	if (status)
		*status = MONO_IMAGE_IMAGE_INVALID;

	if (care_about_pecoff == FALSE)
		goto done;

	if (!mono_verifier_verify_pe_data (image, &errors))
		goto invalid_image;

	if (!mono_image_load_pe_data (image))
		goto invalid_image;
	
	if (care_about_cli == FALSE) {
		goto done;
	}

	if (!mono_verifier_verify_cli_data (image, &errors))
		goto invalid_image;

	if (!mono_image_load_cli_data (image))
		goto invalid_image;
...

do_mono_image_load calls function mono_verifier_verify_pe_data which should filter out any invalid PE file and stop loading it. For some reason (which I really don't care about), this function does nothing in Unity3D and returns success.

After that, buggy mono_image_load_pe_data takes over and uses PE parser to load PE structures and .NET metadata. It is followed by equally buggy processing of .NET metadata in both mono_verifier_verify_cli_data and mono_image_load_cli_data.

But first things first..

Invalid PE signature

First of the errors is inside do_load_header. It gets called indirectly from mono_image_load_pe_data:

/*
 * Returns < 0 to indicate an error.
 */
static int
do_load_header (MonoImage *image, MonoDotNetHeader *header, int offset)
{
	MonoDotNetHeader64 header64;

#ifdef HOST_WIN32
	if (!image->is_module_handle)
#endif
	if (offset + sizeof (MonoDotNetHeader32) > image->raw_data_len)
		return -1;

	memcpy (header, image->raw_data + offset, sizeof (MonoDotNetHeader));

	if (header->pesig [0] != 'P' || header->pesig [1] != 'E')
		return -1;
...

It checks only first 2 bytes of PE signature, so you can change it to "PE\0\1" or "PEPE" and it will still work under Mono.

Invalid NumberOfRvasAndSizes value

Next bug is located get_data_dir used indirectly by mono_verifier_verify_cli_data.

static void
verify_cli_header (VerifyContext *ctx)
{
	DataDirectory it = get_data_dir (ctx, CLI_HEADER_IDX);
	guint32 offset;
	const char *ptr;
	int i;

	if (it.rva == 0)
		ADD_ERROR (ctx, g_strdup_printf ("CLI header missing"));

	if (it.size != 72)
		ADD_ERROR (ctx, g_strdup_printf ("Invalid cli header size in data directory %d must be 72", it.size));
...

and get_data_dir looks like this:

static DataDirectory
get_data_dir (VerifyContext *ctx, int idx)
{
	MonoCLIImageInfo *iinfo = ctx->image->image_info;
	MonoPEDirEntry *entry= &iinfo->cli_header.datadir.pe_export_table;
	DataDirectory res;

	entry += idx;
	res.rva = entry->rva;
	res.size = entry->size;
	res.translated_offset = translate_rva (ctx, res.rva);
	return res;

}

Both verify_cli_header and get_data_dir ignore NumberOfRvasAndSizes field in PE header and access "CLR Runtime Header Entry".

Identical bug is in load_cli_header called from mono_image_load_cli_data:

static gboolean
load_cli_header (MonoImage *image, MonoCLIImageInfo *iinfo)
{
	guint32 offset;
	
	offset = mono_cli_rva_image_map (image, iinfo->cli_header.datadir.pe_cli_header.rva);
	if (offset == INVALID_ADDRESS)
		return FALSE;
...

They do not use broken get_data_dir function but the check of NumberOfRvasAndSizes field is still missing. That's why our PE file can have NumberOfRvasAndSizes == 0xA and it still works under Mono.

Invalid metadata size

Mono does a very limited checking of .NET metadata size in load_metadata_ptrs.

...
	size = iinfo->cli_cli_header.ch_metadata.size;

	if (offset + size > image->raw_data_len)
		return FALSE;
...

However, this check is insufficient. For example, you can set .NET metadata size = 0, most of .NET reversing tools will break but Mono will happily accept the file.

Invalid number of .NET streams

.NET metadata streams are loaded by load_metadata_ptrs.

...
	streams = read16 (ptr);
	ptr += 2;

	for (i = 0; i < streams; i++){
		if (strncmp (ptr + 8, "#~", 3) == 0){
			image->heap_tables.data = image->raw_metadata + read32 (ptr);
			image->heap_tables.size = read32 (ptr + 4);
			ptr += 8 + 3;
		} else if (strncmp (ptr + 8, "#Strings", 9) == 0){
			image->heap_strings.data = image->raw_metadata + read32 (ptr);
			image->heap_strings.size = read32 (ptr + 4);
			ptr += 8 + 9;
		} else if (strncmp (ptr + 8, "#US", 4) == 0){
			image->heap_us.data = image->raw_metadata + read32 (ptr);
			image->heap_us.size = read32 (ptr + 4);
			ptr += 8 + 4;
		} else if (strncmp (ptr + 8, "#Blob", 6) == 0){
			image->heap_blob.data = image->raw_metadata + read32 (ptr);
			image->heap_blob.size = read32 (ptr + 4);
			ptr += 8 + 6;
		} else if (strncmp (ptr + 8, "#GUID", 6) == 0){
			image->heap_guid.data = image->raw_metadata + read32 (ptr);
			image->heap_guid.size = read32 (ptr + 4);
			ptr += 8 + 6;
		} else if (strncmp (ptr + 8, "#-", 3) == 0) {
			image->heap_tables.data = image->raw_metadata + read32 (ptr);
			image->heap_tables.size = read32 (ptr + 4);
			ptr += 8 + 3;
			image->uncompressed_metadata = TRUE;
			mono_trace (G_LOG_LEVEL_INFO, MONO_TRACE_ASSEMBLY, "Assembly '%s' has the non-standard metadata heap #-.\nRecompile it correctly (without the /incremental switch or in Release mode).\n", image->name);
		} else {
			g_message ("Unknown heap type: %s\n", ptr + 8);    <---- ignorance is bliss! :)
			ptr += 8 + strlen (ptr + 8) + 1;
		}
		pad = ptr - image->raw_metadata;
		if (pad % 4)
			ptr += 4 - (pad % 4);
	}
...

You can use an arbitrary large number of streams in .NET Metadata header, as Mono will ignore all the invalid data. It probably will spam Android log with "Unknown heap type" messages, but the file will still run.

Invalid number of rows in .NET tables

Final nail in the coffin is the incorrect processing on .NET metadata tables. Tables are loaded in load_tables method which seems to be correct on the first look:

...
	for (table = 0; table < 64; table++){
		if ((valid_mask & ((guint64) 1 << table)) == 0){
			if (table > MONO_TABLE_LAST)
				continue;
			image->tables [table].rows = 0;
			continue;
		}
		if (table > MONO_TABLE_LAST) {
			g_warning("bits in valid must be zero above 0x2d (II - 23.1.6)");
		} else {
			image->tables [table].rows = read32 (rows);
		}
		rows++;
		valid++;
	}
...

However, the definition of .NET metadata table information (_MonoTableInfo) is wrong:

struct _MonoTableInfo {
	const char *base;
	guint       rows     : 24;
	guint       row_size : 8;

	/*
	 * Tables contain up to 9 columns and the possible sizes of the
	 * fields in the documentation are 1, 2 and 4 bytes.  So we
	 * can encode in 2 bits the size.
	 *
	 * A 32 bit value can encode the resulting size
	 *
	 * The top eight bits encode the number of columns in the table.
	 * we only need 4, but 8 is aligned no shift required. 
	 */
	guint32   size_bitfield;
};

As a result of this definition, they ignore high-order byte in row count. You can set number of rows to, say, 0xCC000001, .NET metadata will be invalid but Mono happily accepts the file. WTF?

Extra protection by AndroidThaiMod

Game hacks created by AndroidThaiMod.com take the Unity3D/Mono abuse to the next level. They exploit all the bugs I already mentioned. In addition to that, they replace the original libmono.so with their own version. In their version there are few extra lines of code in the function mono_image_open_from_data_with_name:

...
  if ( datac[0] != 0x4D || datac[1] != 0x5A || datac[2] != 0x90 || datac[3] )
  {
    for ( i = 0; i < data_len; ++i )
    {
      datac[i] ^= 0x41;
      datac[i] ^= 0x30;
    }
  }
...

This change allows AndroidThaiMod to distribute their DLLs encrypted with "extra-strong" XOR encryption. 🙂

Another fun fact - their cheats only hack ARM version of Unity3D runtime. x86 version, even if present in original APK, gets removed from the hacked APK. So, if you have an Android device with x86 CPU, you're out of luck - AndroidThaiMod cheats won't work there.

Few AndroidThaiMod files I was able to find on Google:
https://drive.google.com/file/d/1g2_43rP9LLrXSmGL-Lw36GTehSUmg3CJ/view
https://drive.google.com/file/d/0B7jRiqM-QmgUeUF2RldrRUFmcUk/view

Other possibilities of abuse

As I mentioned, Mono PE loader is extra buggy. Pretty much all fields in PE header are not validated properly. Plenty of fields in .NET structures are ignored. PE32+ header processing is broken beyond belief. File/section alignment is not enforced. You can have several streams named "#~" and Mono will happily use one of them. Or you could just search for a phrase "FIXME" in Mono sources - you'll find lots of other dirty hacks that can be exploited.

Can it be fixed?

Probably. But considering how broken and messy the entire Mono codebase is, I wouldn't bet on it.

For example, on 13-Apr-2018 Mono developers made this awful commit called "Verify all 4 bytes of PE signature.":

 	memcpy (header, image->raw_data + offset, sizeof (MonoDotNetHeader));
 
-	if (header->pesig [0] != 'P' || header->pesig [1] != 'E')
+	if (header->pesig [0] != 'P' || header->pesig [1] != 'E' || header->pesig [2] || header->pesig [3])
 		return -1;
 
 	/* endian swap the fields common between PE and PE+ */

There was no explanation why it was done, what are the side effects and whether they plan to fix all the broken code or if this was just a one-off fix. What a surprise!

And, as I already mentioned, Unity3D is using 3+ years old version of Mono. So, you'll probably have to wait until year 2021 until Unity3D gets they necessary fixes. 😀

Workarounds

Even if Mono and Unity3D was fixed, it doesn't help us to analyze existing broken files.

One workaround was to update dnlib (commit c40e148) and reversing tools to support such broken files. It allows to analyze existing files using dnSpy and de4dot. But Reflector and other tools still won't work. So, I decided to make my own tool which will fix broken files and will allow you to use any .NET reversing tool you like.

I needed a PE/.NET reading library which is very simple and low-level. dnLib/Mono.Cecil is way too abstract. After a quick search, I decided to use parts of McCli - but any other library would do just fine.

The code is very straightforward. Read PE structure->validate->fix->repeat.. 😉

...
            MemoryStream stream = new MemoryStream(data, 0, data.Length, true, true);
...
            var ntHeadersSignature = stream.ReadStruct();
...
            if (ntHeadersSignature != IMAGE_NT_HEADERS.IMAGE_NT_SIGNATURE)
            {
                Console.WriteLine("[!] Fixing wrong PE signature.");
                stream.WriteStruct(savedPos, IMAGE_NT_HEADERS.IMAGE_NT_SIGNATURE);
            }
...

Conclusion

Games and cheats made using Unity3D are great. It's possible to make very unique protections that will stop most Windows/.NET reversing tools but the program will still work on Android. Enjoy the fun times! 🙂

Unity3D fixer with source code: