An arbitrary dive into assembler and certain Windows internals, part 1

I decided yesterday, for reasons, to poke around in the internals of a certain Windows API function. I only know enough assembler to be dangerous, but I learned some things along the way. In this post, I'll take you along for the ride. Hopefully, this will be comprehensible to programmers who know very little if any assembler. I'll make sure to include lots of colorful pictures!

In which we decide our target

We're going to be examining the NetUserModalsGet function, which lets programs get information about the domain and its security settings. It takes a server name, a "level", and a pointer to a buffer. The "level" specifies which kind of information you want to get out of the function; the type of stuff that's going to be placed in the buffer depends on it. We're interested in level 2, which will yield a USER_MODALS_INFO_2 structure, containing the SAM domain's name and SID. Even more specifically, we'll only be looking at the 32-bit version, since I don't know any 64-bit assembler, and also because the tool I want to use doesn't support 64-bit stuff. Speaking of tools...

In which we acquire our tools

I like IDA Pro as a disassembler. The last version that's free is 5.0, but that will work just fine. Download a copy from the official web site, and run the installer in compatibility mode for Windows XP SP3. When the installer finishes, run IDA Pro Free. You'll get a notice about newer versions; ignore it. You'll then get a Welcome to IDA! dialog. It has a New button. Click it. You then get the option to choose what kind of thing you want to disassemble.

The MSDN documentation on our target function says at the bottom of the page that it lives in Netapi32.dll. DLLs are dynamic libraries, so select that option. Browse into \Windows\SysWOW64 and choose that file. You get a wizard that helps you load the file. All the defaults are fine, so click through until you Finish. IDA will then detect that "the input file was linked with debug information" and will allow you to have it search for symbols. Choose Yes, because that will let it fill in some names for us.

Now it actually starts disassembling. IDA does an "initial autoanalysis", which I think is what produces the fancy visual arrangements of code chunks. You'll know that pass is done when you see this:

By default, IDA shows us the function called DllEntryPoint, which is called at certain points in the DLL's lifecycle. It's not relevant to the function we're interested in.

In which we encounter an obstacle

The function we want will be exported from this DLL. To see the exports list, click the Exports tab near the top. You'll get a big report of all the exported functions and their addresses.

Find NetUserModalsGet and double-click it to move to it.

Uh oh. That's not code, that's just a bunch of function declarations! Interestingly, there's something else before the function name in one of the strings: SAMCLI. A look through a directory listing reveals that there is a DLL by that name. Choose File | Close to exit this disassembly (check Don't save the database to save on disk space). Then choose File | New and repeat the opening process from samcli.dll.

In which progress continues

A look through this DLL's Exports window shows that it does indeed have the function we want! The entry in the other DLL was a forwarder. Double-click this new entry we found, and we get real code:

Let's start from the top. At the very top (which didn't fit in the screenshot), there's the function's signature: the parameters we saw on MSDN. Then there are a bunch of things involving dword ptr. They aren't part of the assembler, IDA just shows them for our convenience as local variables. Each happens to be 32 bits (4 bytes) long, hence DWord. The last three, with names matching the arguments, indicate where those arguments can be accessed. More on this later.

Then we get into actual assembler instructions. The first is a mov. This kind of instruction always copies data from the second thing to the first thing. Those things can be registers, memory locations, or in the case of the source, a literal. Since this one is mov edi, edi, it does nothing. It copies data from one register to itself. This is to allow patching.

The next three lines are a standard prologue. They set up the stack like IDA shows above, with 0x28 bytes available for local variables. (That's how IDA knows about those variables, actually.) The ebp register now holds the address where all local variables and arguments are based. Things can be added and subtracted from it to get the address of such a variable. That's what the next line does: ebp+bufptr is the address of the bufptr argument. Since it's enclosed in brackets, the move instruction dereferences that pointer to get what's at that address. The result stored in the eax register. The next two pushes preserve the values of those registers so as to not wreck their contents. The next instruction stores the level argument in edi, just like was done for bufptr.

A bunch of moves zero out the data in each local variable. The last move writes a zero into the location specified by the address in eax. I believe the dword ptr here specifies that the written value is 32 bits long (as opposed to two or one bytes of zero). As you recall, eax received the value of bufptr, which is an address (hence the "pointer" part), so that instruction zeroes out the first four bytes of the provided buffer. I'm guessing this is to make sure the buffer is long enough.

In which choices are made

The next instruction, cmp edi, 2, compares the two values. As you recall, edi received the level, which we're assuming is 2 because that's the one we care about. Comparisons are always followed by conditional jumps, like the jnz you see at the end of this block. That instruction stands for "jump if not zero", which, since we're comparing two different values, actually means "jump if not equal." IDA provides two lines, one green and one red. The green one will be followed if the comparison is true, the red if it doesn't. edi and 2 are the same, so they're not not the same, so the red line it is. Double-click a line to follow it.

The red line we just followed is the vertical one above the the darkened left box. This sets ebx to 4. There's only one path out: the blue line to the bottom, larger chunk. That chunk stores the servername parameter in ecx. Then we meet a new instruction: lea, Load Effective Address. I don't understand why it has brackets, since it doesn't dereference anything. It just adds up ebp (the base pointer) and the offset of one of our local variables and stashes that result (which will be a valid pointer to that variable's space) in eax. esi and eax are then pushed. xor edx, edx zeros that register. Then there's a call to UaspOpenSam, a function defined in this DLL.

Double-click a function name to jump to its definition. Scrolling around in UaspOpenSam shows that it's pretty complicated, but the most interesting thing is that it calls SamConnect. If you poke around, you'll find that it's imported from samlib.dll. Let's not get bogged down in that, though. As one Stack Overflow user discovered, that other DLL's API matches the SAMR RPC interface. How convenient. Once you're done inspecting a function in IDA, you can press Escape to go back to the previous screen.

Given that eax was just pushed and that it's now being assigned to a different register and then tested, it's a pretty good guess that UaspOpenSam uses that register to return something, probably whether it succeeded. After all, it doesn't really make sense to check whether a memory location is zero if you just assigned it something that would never be zero. I don't know whether that register should be zero on success or the other way around, but let's check both. If that function returns nonzero, the jnz will jump, and we follow the green arrow.

A boatload of different chunks end up here, and a memory-freeing call follows. That sure seems like a cleanup section to me. Therefore, I conclude that that jump is made when NetUserModalsGet is bailing out, i.e. on failure of UaspOpenSam. Double-click the rightmost green arrow to go back before the jump. (Mousing over an arrow shows you the code shortly before its jump instruction.) We follow the red arrow instead.

That looks really similar. The value of the variable named var_24 is loaded into ecx, and the address of the variable named hMem (for reasons unknown) is loaded into eax, then pushed. The same thing happens for var_4. ebx (which, if you recall, holds the number 4) is copied into edx in between the previous lea and push. A literal 1 is pushed, and UaspOpenDomain is called.

The three pushes seem to match up with UaspOpenDomain's three arguments. Looking back, UaspOpenSam only has one argument, and the last push before it was the address of var_24. That variable's contents are passed to the domain opener, which suggests that it holds a handle to the SAM database. Indeed, SamrConnect produces a handle of the same type that SamrOpenDomain receives. Following that pattern, it makes sense that the var_4 variable receives the domain handle.

(If you're wondering how UaspOpenDomain got a hold of the data at var_24, it seems to have taken it via ecx, since that register's value is carefully shuffled around inside there until it's passed to SamOpenDomain.)

Anyway, another jump-if-non-zero thing follows the opening of the domain. Once again, the green path goes down to the hub of failure, while the red path goes to another branch.

We have another comparison here. Recall that edi once received the "level." A look back up the path we traveled shows that it hasn't been touched since then (assuming none of the functions called wrecked it). We're still interested in level 2, so we want those compared things to be the same. The jump-if-not-same should fail, so we follow the red path.

In which we are suspended from a cliff

We've made it a long way, but we're not there yet - we haven't arranged to return anything to the caller of NetUserModalsGet. We'll pick it up next time.

Continue: Part 2