11 Mar 2016

About .NET, googling and lazy programmers.

Delphi fail. .NET win.

Recently, several people sent me bug reports where my EnigmaVB unpacker failed to extract files. In all cases, the problem was caused by really large files, like 3.5GB in size. So, what's causing the problem?

EnigmaVB unpacker is a 32bit application written in Delphi. And Delphi streams are retarded. They look like they've been written in 1990s and were never updated. TMemoryStream uses a continuous memory range, so it can never support anything larger than 2GB. TFileStream internally uses longint, so it doesn't properly support files larger than 2GB. WTF?

So, I have two choices. I can either make a custom stream class in Delphi, or I can pick another framework and rewrite my tool to use that.

I'm not a programmer, I'm a reverser. I don't want to spend my time developing custom stream classes. I'd very much rather use this time breaking some code. So, say hello to .NET framework - my next version of EnigmaVB unpacker will be made in C#.. smile

Am I a programmer or a googler?

While researching all the Delphi limitations and possible workarounds, I ran into this great article by Scott Hanselman. Reading both the post and the comments made me think a lot.

Does using Google to solve your programming tasks makes you less of a programmer? I don't think so.

In fact, I'm just lazy. Most people are. Why should I spend 30 minutes remembering basic algorithms for converting string to hex, if Google query can solve it in 10 seconds? Why reinvent the wheel and write CRC calculation from scratch? I'll just open StackOverflow and have a solution that's already tried and tested. It doesn't mean I can't do those boring tasks - I just don't want to.

How about you? Would you be able to write some tools without using Google and StackOverflow?

19 Jun 2015

Weirdness of C# compiler

I've been quite busy lately. I made OMF file template I promised few weeks ago, found a remote code execution vulnerability in One Big Company's product and spent some time breaking keygenme by li0nsar3c00l. I'll make a blog post about most of these findings sooner or later.

But today I want to show you something that made me go WTF..

I needed to see how the loop "while true do nothing" looks like in IL. Since C# compiler and .NET JIT compiler are quite smart and optimize code that's certainly an eternal loop, I needed to get creative:

Nothing fancy, but C# & JIT compiler don't track param values, so they both generate proper code..

Well, I thought it's a proper code, until I looked at generated MSIL:

WTF WTF WTF? Can anyone explain to me why there are 2 ceq instructions?

I could understand extra nop and stloc/ldloc instructions, but that ceq completely blows my mind.. And it's the same for .NET 2.0 and 4.5 compilers.

02 Jun 2015

Since you asked.. How to inject byte array using dnlib

Quite often I receive random questions about dnlib from my friends. To be honest, I have no idea why they think I know the answers to life the universe and everything else. smile So, in this series of posts I'll attempt to solve their problems - and hope that the solution helps someone else too.

So, today's question is:

We're trying to add a byte array to an assembly using dnlib. We wrote some code* but dnlib throws exception when saving modified assembly:
An unhandled exception of type 'dnlib.DotNet.Writer.ModuleWriterException' occurred in dnlib.dll
Additional information: Field System.Byte[] ::2026170854 (04000000) initial value size != size of field type

I gave the friend the standard answer - make a sample app, see how it looks and then implement it with dnlib. Seriously, how hard can it be? smile

Well, array initialization in .NET is anything but simple.

How arrays are initialized in C#

Note - the following explanation is shamelessly copied from "Maximizing .NET Performance" by Nick Wienholt. It's a very nice book but getting little outdated. You can Google for "Apress.Maximizing.Dot.NET.Performance.eBook-LiB", if interested.

Value type array initialization in C# can be achieved in two distinct ways—inline with the array variable declaration, and through set operations on each individual array element, as shown in the following snippet:

For a value type array that is initialized inline and has more than three elements, the C# compiler in both .NET 1.0 and .NET 1.1 generates a type named <PrivateImplementationDetails> that is added to the assembly at the root namespace level. This type contains nested value types that reference the binary data needed to initialize the array, which is stored in a .data section of the PE file. At runtime, the System.Runtime.CompilerServices.RuntimeHelpers::InitializeArray method is called to perform a memory copy of the data referenced by the <PrivateImplementationDetails> nested structure into the array's memory location. The direct memory copy is roughly twice as fast for the initialization of a 20-by-20 element array of 64-bit integers, and array initialization syntax is generally cleaner for the inline initialization case.

Say what? You can read the text 3 times and still be no wiser. So, let's make a small sample application and disassemble it.

How array initialization looks in MSIL

Let's start with sample app that does nothing.

Compile without optimizations, and disassemble using ildasm. And even after removing all extra stuff, there's still a lot of code & metadata for such a simple thing. smile

For one byte array that we declared, compiler created .data directive, 2 static fields, one class and one nested class. And it added a global static constructor. Yikes!

Implementing it in dnlib

Now that we know all the stuff that's required for an array, we can make a tool that will add byte array to an assembly of our choice. To make things simpler, I decided not to create a holder class (named <PrivateImplementationDetails>{E21EC13E-4669-42C8-B7A5-2EE7FBD85904} in the example) and put everything in global module instead.

Note - Since I'm not a .NET/dnlib wizard, I always do it one step at a time, make sure it works and then continue. So, my workflow looks like this: write a code that does X → compile and run it → disassemble the result → verify that result X matches the expected → fix the bugs and repeat. Only after I've tested one thing, I move to the next one.

It also helps to make small test program first. Once you know that your code works as intended, you can use it in a larger project. Debugging the entire ConfuserEx project just to find a small bug in modifications made by someone - it's not fun! So, step-by-step...

First, we need to add the class with layout. It's called '__StaticArrayInitTypeSize=5' in the example above. That's quite simple to do in dnlib:

Now we need to add the static field with data, called '$$method0x6000003-1'.

Once that is done, we can add our byte array field, called bla in the example.

That's it, we have all the fields. Now we need to add code to global .cctor to initialize the array properly.

And that's it! Simples!

Further reading

Commented demo code at Pastebin
Longer explanation how array initialization works in C#


Updates

Just to clarify - this is a sample code. It works for me but if it blows up in your project, it's your problem. And there always are some things that can be improved.

• Sometimes I'm overcomplicating things.. You don't need to explicitly import System.Byte, you can use mod.CorLibTypes.Byte for that.

SZArraySig is a cleaner but less obvious way to refer to any array. If you need to reference complex arrays, this is better:

08 Apr 2015

Catch me when you can

Introduction

Exception filters have been part of ECMA-335 specification since the very beginning. I'm guessing, they were added because Visual Basic used them extensively and therefore Visual Basic.NET had to support them as well. They look something like this:

Until now C# supported try/catch but did not have support for filters. That's going to change in C# 6.0/VS2015.

How does it work

In early versions of VS2015 the syntax was "catch-if", as you can see in the initial announcement. In the latest VS2015 CTP builds, they changed syntax to "catch-when", and there's a good reason for it.

So, how does it work and what does it mean for reversers?

It's a compiler-level feature

As I mentioned before, .NET Framework has supported exception filters since the very beginning. So, this feature works even in .NET 2.0 - if you decide to target .NET 2.0 Framework in VS2015 project settings. Not that you really want to do that..

It's very useful for debugging

catch-when is implemented as an IL exception filter. So, when an exception is thrown, exception filters are processed before the stack is unwound. This means that filter method has created an error report that included the current stack trace, it would show the frame in which the exception occurred. Sounds complicated? It isn't.

Let's implement exception filtering in the "old" way:

and this is how the stack looks when we get to filter(ex):
stack_trace_old
You can't see much here. All the context is gone, you must rely on exception stack trace and message. That's what we've always done, right? smile

If we write it in a "new" way, the code looks like this:

and stack trace will give us full context of exception:
stack_trace_new
Much better, isn't it? You can see which method threw the exception, on which line, you have access to local variables and everything else. Yummy! smile

Decompiler support for exception filters is crappy

They say, a picture is worth thousand words.. In a very simple example, Reflector gets the code structure right, just filter conditions are missing:
catch_when_reflector
ILSpy handles it slightly worse, filters are messed up and unreadable. Filter code is gone, too:
catch_when_ilspy
And the latest JustDecompile just throws an exception:
catch_when_justdecompile

Have fun with it!

Here is a small keygen-me for you to play with: https://www.mediafire.com/?k5b9vy0p9dfgb97

The difficulty is 2/10, you should be able to solve it in 30 minutes or so. The entire protection is designed to show you try-catch-when feature, so avoid patching - you can't learn anything by nopping-out few instructions. wink

11 Mar 2015

Miserable state of open source code

Yesterday I wanted to make a small API hook detector in C#. It has to parse PE file, find exported functions, read bytes from the beginning of function and then compare them with the bytes in process memory. Sounds simple, right?

Well, good luck finding a PE parser that actually works!

Looking for PE parser

Most of PE parsers stop at parsing DOS header, NT headers and section headers. But I needed something that would also parse export table for me. After a couple of Google searches I ended up with PEReader by DKorablin. From the first look it's decent and even has a demo application. What else could you want?

Hmmm, how about working correctly on really simple files? wink

Sorry, nope.

Results from PEReader and CFF
It sure finds exported functions but it mismatches function names & RVAs. So, if you wanted to examine, say, CreateFileW, you will end up examining DeleteFileA. Or some other random API. Great job!

But it's opensource. Just fix it and submit a patch!

Umm, no. I was looking for a PE parser that I can take, load it in VS and use it. I don't want to spend days hunting down bugs and fixing them - this stops me from doing what I really want to do.

So, dear opensourcer, if you are publishing your code, make sure it actually works. If it doesn't work, please don't publish it at all - it's not helping anyone. Don't waste other people's time..

P.S. I ended up with using DNLib and writing my own PE export parsing. At least, I know it works properly..

09 Feb 2015

.NET exception “The remote certificate is invalid”

For one project I needed to make a simple tool that fetches file from webserver via HTTPS connection. Sounds easy, right?

There are plenty of ways of doing things, but to keep it simple, lets ignore error handling, timeouts, authorization and other features:

It should work, right?

And yes, it works most of the time. For example, it fetches page from https://google.com/ without any problems.

But for some sites it mysteriously crashes with exception:

Cause of the error

It turns out that sites that are using self-signed, expired and otherwise bad certificates will cause this exception. How nice from .NET devs - they're trying to protect me! Well, no, because I still need to download that information!

Solution

I found lots of solutions of the web - and quite a few of them failed. Some of them are .NET 4.5 only, some of them are overly complicated class consisting of 20+ lines. I need something simple, that works everywhere from .NET 2.0 up and is actually readable!

This one works for me: put this line before doing any HTTPS requests:

It effectively suppresses all certificate validation errors. Problem solved!

Example site for testing your code: https://www.pcwebshop.co.uk/