Monday, July 24, 2017

DefinePInvokeMethod functions will unbalance the stack without PreserveSig

Today I worked on a program that adds a P/Invoke method to a dynamically generated module using DefinePInvokeMethod. Building the module worked just fine, but calling that method produced an error about a stack imbalance. I triple-checked the calling convention and parameters (the lack thereof, actually), but the error remained.

Eventually I came across an Important box in the Examples section of DefinePInvokeMethod's documentation. There is a PreserveSig flag that needs to be set for the return value to get back to managed code, but apparently the flag does other things too. After I set it, the stack imbalance went away.

dynMethod.SetImplementationFlags( _ 
 dynMethod.GetMethodImplementationFlags() Or MethodImplAttributes.PreserveSig)

No comments:

Post a Comment