Monday, November 13, 2006

Executing Dynamic Code in .Net

Dynamically executing code in .Net

By Rick Strahl

http://www.west-wind.com/

Last Update: September 8th, 2002

Code for this article:

http://www.west-wind.com/presentations/dynamicCode/dynamicCode.zip

If you find this article useful, consider making a small donation to show your support for this Web site and its content.

Dynamic code execution is a powerful feature that allows applications to be extended with code that is not compiled into the application. Users can customize applications and developers can dynamically update code easily. In this article, Rick takes a look what it takes to execute code dynamically with the .Net framework and introduces a class that simplifies the tasks by wrapping the details of the process in an easy to use interface that only requires a few lines of code.

I come from an xBase background using Visual FoxPro for many years. One of the nice features of xBase has always been the ability to dynamically execute code in applications. In the past, developers have often snubbed me and xBase in general for this capability to use 'macros' (as they would say with disdain), partially because in the early days of xBase macros where used for a number of kludges required to make the language work correctly. Its use often resulted in highly unreadable code or worse spaghetti code that was difficult to debug and was very slow. However, in recent years use of dynamic code has found a much more widespread audience as mechanisms for building user extensions and providing custom user defined interfaces and templates for representing data. I can hardly imagine an application where there's not a good use to be made of code stored externally either for presentation purpose or for extending the application by end users.

Dynamic code execution is a very powerful thing not to be underestimated! With it you can perform a number of decisions at runtime about which code to run and most importantly allow users to customize and extend existing applications via custom code. Dynamic code execution is also very important for things like templating of code into things like HTML, merge documents (mail merge) or custom document solutions. For example, a scripting engine front end like ASP or ASP.Net, actually rely on the ability to dynamically read code from disk and execute it on the fly. Dynamic code execution is also a feature that is notoriously absent from compiled languages and required specialty tools like the Microsoft Scripting ActiveX control to execute code.

The good news is that .Net provides full control over dynamic code execution natively via the .Net SDK classes. The bad news is that the process is not nearly as trivial as it was in Visual FoxPro (where Evaluate() and ExecScript() are single commands that perform most dynamic code tasks). It requires a fair amount of code to accomplish something similar and it requires dealingl with some architectural issues of the way .Net loads code into applications. But in exchange .Net provides a lot of flexibility in using dynamic code with full control over the entire process including compilation, error reporting and loading of objects and controlling the environment of dynamic code.

In this article, I'll introduce what it takes to dynamically compile and execute code as well as discussing some of the more intricate requirements that most applications will need to consider when running lots of code or constantly changing code dynamically. I'll also introduce a class that simplifies the process of executing code to a few lines and then demonstrate that class by building the beginnings of a free standing ASP like script parser that you can plug into your own desktop applications.

Compiling code on the fly

.Net provides powerful access to the IL code generation process through the System.CodeDom.Compiler and Microsoft.CSharp and Microsoft.VisualBasic namespaces. In these namespaces you find the tools that allow you to compile an assembly either to disk or into memory. You also need the Reflection namespace as it contains the tools to invoke an object and its methods once you've compiled the object.

In the following example, I demonstrate how to execute an arbitrary block of code. The code is free standing and has no dependencies. The process to execute this code dynamically involves the following steps:

  1. Create or read in the code that is to be executed as a string
  2. Wrap the code into fully functional assembly source code, which includes namespace references (using commands), a namespace and a class that is to be invoked
  3. Compile the source code into an assembly
  4. Check for errors on compilation
  5. Use the assembly reference to create an instance of the object
  6. Call the specified method on the instance reference returned using Reflection
  7. Handle any return value from the method call by casting into the proper type

Figure 1 – This sample form demonstrates how to execute code from the top text box dynamically.

The example shown in Listing 1 demonstrates the code to perform these steps. Figure 1 shows an example of the form that utilizes this code. Please note that there's only minimal error handling provided in most code snippets for brevity's sake.

Listing 1: (basicExecution.cs) – Basic code to execute code on the fly

using System.Reflection;

using System.CodeDom.Compiler;

using Microsoft.CSharp; // and/or Microsoft.VisualBasic;

private void button1_Click(object sender, System.EventArgs e)

{

// *** Example form input has code in a text box

string lcCode = this.txtCode.Text;

ICodeCompiler loCompiler = new CSharpCodeProvider().CreateCompiler();

CompilerParameters loParameters = new CompilerParameters();

// *** Start by adding any referenced assemblies

loParameters.ReferencedAssemblies.Add("System.dll");

loParameters.ReferencedAssemblies.Add("System.Windows.Forms.dll");

// *** Must create a fully functional assembly as a string

lcCode = @"using System;

using System.IO;

using System.Windows.Forms;

namespace MyNamespace {

public class MyClass {

public object DynamicCode(params object[] Parameters) {

" + lcCode +

"} } }";

// *** Load the resulting assembly into memory

loParameters.GenerateInMemory = false;

// *** Now compile the whole thing

CompilerResults loCompiled =

loCompiler.CompileAssemblyFromSource(loParameters,lcCode);

if (loCompiled.Errors.HasErrors) {

string lcErrorMsg = "";

lcErrorMsg = loCompiled.Errors.Count.ToString() + " Errors:";

for (int x=0;x

lcErrorMsg = lcErrorMsg + "\r\nLine: " +

loCompiled.Errors[x].Line.ToString() + " - " +

loCompiled.Errors[x].ErrorText;

MessageBox.Show(lcErrorMsg + "\r\n\r\n" + lcCode,

"Compiler Demo");

return;

}

Assembly loAssembly = loCompiled.CompiledAssembly;

// *** Retrieve an obj ref – generic type only

object loObject = loAssembly.CreateInstance("MyNamespace.MyClass");

if (loObject == null) {

MessageBox.Show("Couldn't load class.");

return;

}

object[] loCodeParms = new object[1];

loCodeParms[0] = "West Wind Technologies";

try {

object loResult = loObject.GetType().InvokeMember(

"DynamicCode",BindingFlags.InvokeMethod,

null,loObject,loCodeParms);

DateTime ltNow = (DateTime) loResult;

MessageBox.Show("Method Call Result:\r\n\r\n" +

loResult.ToString(),"Compiler Demo");

}

catch(Exception loError) {

MessageBox.Show(loError.Message,"Compiler Demo"); }

}

Assemblies and Namespaces

The code begins by creating various objects that will be required for compilation. It then uses the CompilerParameters object to add any assembly references that will be required during compilation. These are the physical DLLs that are required and are the equivalent of what you add in the VS.Net project References section. Note that it's very important that every reference is included or you will get compiler errors. As we'll see this is one of the more tricky parts about dynamic code compilation as this step must occur in your application code. Here the Windows Forms assemblies are included to allow using the MessageBox object to display output.

The next step is to generate the complete sourcecode for an assembly. This example makes a few assumptions about the code in that it presets the method parameter and return value signature as:

public object DynamicCode(parms object[] Parameters);

So a block of code MUST return a value of type object or null. It can also accept any number of parameters which can be referenced via the parameters collection. A simple example of a string to execute might be.

string cName = "Rick";

MessageBox.Show("Hello World" + cName);

return (object) DateTime.Now;

If you wanted to access parameters dynamically instead you might do:

string cName = (string) Parameters[0];

Note that you should cast parameters explicitly to the specific type since the object parameter is generic. You can also return any value as long as you cast it to an object type.

This code is now fixed up into an assembly by adding namespace, class and method headers. The final generated code that gets compiled looks like this:

using System.IO;

using System;

using System.Windows.Forms;

namespace MyNamespace {

public class MyClass {

public object DynamicCode(parms object[] Parameters) {

string cName = "Rick";

MessageBox.Show("Hello World" + cName);

return (object) DateTime.Now;

}

}

}

This code can now be compiled into an assembly by using the CompileAssemblyFromSource() method of the CodeCompiler. The CompilerResults object receives information about the result: Error info if the compilation failed can be retrieved via the HasErrors property and then the Error collection which contains detailed information about each error that occurred during compilation. If there were no errors you get a reference to the Assembly in CompiledAssembly property from which you can call CreateInstance() to get a live instance of the MyClass class.

This is where Reflection comes in: Because we've basically created a .Net type on the fly the object reference and all method access must occur dynamically rather than direct referencing. This is because the compiler has no idea of the type at compile time, but must delay creation and type info until runtime. So when we call CreateInstance an object of type object is returned and we have to use Reflection and InvokeMember to call a method on the object indirectly.

The actual call to the object method then proceeds and returns a reference to a generic object type (much like a variant). This type can contain data of any type and I suggest that you immediately cast the return type to an explicit type if possible. Notice also the error handling around the InvokeMember call – this is fairly crucial as it protects the calling application from any runtime errors that occur in the dynamic code.

I've demonstrated this technique by using CSharp as the dynamic code language here. You can also use VisualBasic by using the Microsoft.VisualBasic namespace and using the VBCodeProvider class instead to instantiate the loCompiler object. Of course, you'd have to change the assembly source code to VB syntax in addition to the actual dynamic code I showed here. The class I'll present later provides the ability to both execute C# and VB code by setting a language property.

As I mentioned at the start of this article - .Net provides a lot of functionality and full control over the compile and execution process but this is a lot of code to have to integrate into an application each time you want to execute dynamic code. To make life easier I've created a class that simplifies the process considerably and aids in handling errors and debugging the code should errors occur.

Understanding how .Net loads code

Before I dive into the class however I need to discuss the important subject of Application Domains and how they behave when assemblies are loaded. Application Domains are the highest level isolated instances of the .Net runtime that host application code and data. Assemblies get loaded into a specific application domain and execute and use resources in it.

What's an Application Domain?

When you normally run a .Net application, .Net simply loads each assembly on your references list into the application's primary Application Domain (see sidebar 'What's an Application Domain?'). No problem there – you want all code to load into this domain and stay loaded there. So if there's code that dynamically uses the JIT compiler to compile code the code will remain in the app domain cached and compiled so only first access is relatively slow.

So far, so good. But here's the rub in our dynamic code execution scheme: Application Domains load assemblies, but they cannot unload them! If you're only loading a handful of assemblies this won't be a problem, but often times when you run dynamic code it's quite possible that you will create a lot of snippets that need to run and compile independently then essentially throw them away. For example, I have a Desktop application that uses templates on disk to hold HTML mixed with .Net code. The application merges the content of a database record (actually an object view it) into the template. The documents are merged on the fly and only on an as needed basis as they are frequently updated. This system can have thousands of entries and pretty much each of these pages has to compile separately.

If you run the demo above in a loop for 10 – 20 times you will notice that memory usage increases with each instance of creating and releasing an assembly by a few K each time depending on the size of the assembly and its related referenced assemblies. Once loaded none of that space can be unloaded again if the assembly is loaded into the current main application's AppDomain.

So what to do? Unfortunately there's no simple answer - only a convoluted one. The answer is to create a new Application Domain and load our dynamic assemblies into that. We can either load into this AppDomain, run our code and unload it, or alternately run all of our dynamic code into the new domain and kill it later or when it reaches a certain number of executions or other metric. This process unfortunately is not trivial and requires the use of an intermediary proxy object that can be used to invoke a method in a remote app domain without referencing the object in the local application domain in any way (which again would lock the assembly into the local AppDomain). The process here is essentially the same as invoking a remote object over the network along with all the same complications.

Creating code in alternate AppDomains

Loading an assembly and creating a class instance from it in a different application domain involves the following steps:

  1. Create a new AppDomain
  2. Dynamically create the dynamic assembly and store on disk
  3. Create a separate assembly that acts as an object factory and returns an Interface rather than a physical object reference. This assembly can be generic and is reusable but must be a separate DLL from the rest of the application.
  4. Create an object reference using AppDomain::CreateInstance and then call a method to return the remote interface. Note the important point here is that an interface not an object reference is returned.
  5. Use the Interface to call into the remote object indirectly using a custom method that performs the passthrough calls to the remote object.

The whole point of this convoluted exercise is to load the object into another AppDomain and access it without using any of the object's type information. Accessing type information via Reflection forces an assembly to load into the local AppDomain and this exactly what we want to avoid. By using a proxy that only publishes an interface and thus loads only a single assembly that publishes this generic interface.

For the dynamic code execution class I'm going to create a very simple interface (shown in Listing 2) that can simply Invoke a method of the object.

Listing 2 (RemoteLoader.cs): Proxy interface used to access for AppDomain loading

/// Interface that can be run over the remote AppDomain boundary.

public interface IRemoteInterface

{ object Invoke(string lcMethod,object[] Parameters); }

This interface is then used to make pass through calls on the methods of the dynamic object. The code we now use to generate the full assembly looks like this:

using System.IO;

using System;

using System.Windows.Forms;

namespace MyNamespace {

public class MyClass:MarshalByRefObject,IRemoteInterface {

public object Invoke(string lcMethod,object[] Parameters) {

return this.GetType.InvokeMember(lcMethod,

BindingFlags.InvokeMethod,null,this,Parameters);

}

public object DynamicCode(parms object[] Parameters) {

string cName = "Rick";

MessageBox.Show("Hello World" + cName);

return (object) DateTime.Now;

} } }

By doing this we're deferring the type determination via Reflection into the class itself. Note that the class must also derive from MarshalRefObject which provides the access to data across application domain boundaries (and .Net remoting boundaries) using proxies.

In addition to the interface we'll also need a proxy loader object that acts as an Interface factory: It creates an instance reference to the remote object by returning only an interface to the client. Listing 3 shows the code for this single method class that returns an interface pointer against which we can call the Invoke method across domain boundaries without requiring to have a local reference to the type information.

Listing 3 (RemoteLoader.cs): Proxy loader class that returns an interface ref

using System;

using System.Reflection;

public class RemoteLoaderFactory : MarshalByRefObject

{

private const BindingFlags bfi =

BindingFlags.Instance | BindingFlags.Public |

BindingFlags.CreateInstance;

public IRemoteInterface Create(string assemblyFile,string typeName,

object[] constructArgs ) {

return (IRemoteInterface) Activator.CreateInstanceFrom(

assemblyFile, typeName, false, bfi, null, constructArgs,

null, null, null ).Unwrap();

} }

This class and the IRemoteInterface should be compiled into a separate, lightweight DLL so it can be accessed by the dynamic code for the interface. Both the client code and the dynamic code must link to the RemoteLoader.dll as both need access to IRemoteInterface.

To use all of this in our client code we need to do the following:

  1. Compile our DLL to disk – you can't load the assembly from memory into the other appdomain unless you run the entire compilation process in the other appdomain.
  2. Create an AppDomain.
  3. Get a reference to IRemoteInterface.
  4. Call the Invoke method to make the remote method call.

The revised code that loads an app domain, compiles the code, runs it and unloads the appdomain is shown in Listing 4. Revisions from the previous version are highlighted in blue.

Listing 4 (BasicExecution.cs): Running code dynamically in an AppDomain

using System.Reflection;

using System.CodeDom.Compiler;

using Microsoft.CSharp;

using System.Reflection;

using Westwind.RemoteLoader; // add reference too!

string lcCode = this.txtCode.Text;

// ** Create an AppDomain

AppDomainSetup loSetup = new AppDomainSetup();

loSetup.ApplicationBase = AppDomain.CurrentDomain.BaseDirectory;

AppDomain loAppDomain = AppDomain.CreateDomain("MyAppDomain",null,loSetup);

// *** Must create a fully functional assembly code

lcCode = @"using System;

using System.IO;

using System.Windows.Forms;

using System.Reflection;

using Westwind.RemoteLoader;

namespace MyNamespace {

public class MyClass : MarshalByRefObject,IRemoteInterface {

public object Invoke(string lcMethod,object[] Parameters) {

return this.GetType().InvokeMember(lcMethod,

BindingFlags.InvokeMethod,null,this,Parameters);

}

public object DynamicCode(params object[] Parameters) {

" + lcCode +

"} } }";

ICodeCompiler loCompiler = new CSharpCodeProvider().CreateCompiler();

CompilerParameters loParameters = new CompilerParameters();

// *** Start by adding any referenced assemblies

loParameters.ReferencedAssemblies.Add("System.dll");

loParameters.ReferencedAssemblies.Add("System.Windows.Forms.dll");

// *** Important that this gets loaded or the interface won't work!

loParameters.ReferencedAssemblies.Add("Remoteloader.dll");

// *** Load the resulting assembly into memory

loParameters.GenerateInMemory = false;

loParameters.OutputAssembly = "MyNamespace.dll";

// *** Now compile the whole thing

CompilerResults loCompiled =

loCompiler.CompileAssemblyFromSource(loParameters,lcCode);

if (loCompiled.Errors.HasErrors) {

...

return;

}

this.txtAssemblyCode.Text = lcCode;

// create the factory class in the secondary app-domain

RemoteLoaderFactory factory =

(RemoteLoaderFactory) loAppDomain.CreateInstance( "RemoteLoader",

"Westwind.RemoteLoader.RemoteLoaderFactory" ).Unwrap();

// with help of factory, create a real 'LiveClass' instance

object loObject = factory.Create("mynamespace.dll",

"MyNamespace.MyClass", null );

// *** Cast object to remote interface, avoid loading type info

IRemoteInterface loRemote = (IRemoteInterface) loObject;

if (loObject == null) {

MessageBox.Show("Couldn't load class.");

return;

}

object[] loCodeParms = new object[1];

loCodeParms[0] = "West Wind Technologies";

try {

// *** Indirectly call the remote interface

object loResult = loRemote.Invoke("DynamicCode",loCodeParms);

DateTime ltNow = (DateTime) loResult;

MessageBox.Show("Method Call Result:\r\n\"+loResult.ToString())

}

catch(Exception loError)

{ MessageBox.Show(loError.Message,"Compiler Demo"); }

loRemote = null;

AppDomain.Unload(loAppDomain);

loAppDomain = null;

// *** Delete the generated code DLL when done

File.Delete("mynamespace.dll");

The key differences are the loading of the AppDomain, and how the actual reference to the remote object is retrieved. The critical code that performs the difficult tasks is summarized in:

RemoteLoaderFactory factory =

(RemoteLoaderFactory) loAppDomain.CreateInstance( "RemoteLoader",

"Westwind.RemoteLoader.RemoteLoaderFactory" ).Unwrap();

// *** create interface reference from assembly

object loObject = factory.Create( "mynamespace.dll",

"MyNamespace.MyClass", null );

// *** Cast object to remote interface, to avoid loading type info

IRemoteInterface loRemote = (IRemoteInterface) loObject;

// *** Call the DynamicCode method with no parms

object loResult = loRemote.Invoke("DynamicCode",null);

This code gets a reference to a proxy. RemoteLoader actually loads the object in the remote appdomain and passes back the interface pointer. The interface then talks to the remote appdomain proxy to pass and retrieve the actual data. Because we do have the interface defined locally (through the DLL reference) we can simply call the Invoke() method published by the interface directly.

As you might expect all of this clowning around with creating an AppDomain, loading assemblies into, making remote calls and finally shutting the domain down again causes some overhead. Operation of this mechanism compared to running an assembly in process is noticeably slower.

However, you can optimize this a little by creating an application domain only once and then loading multiple assemblies into it. Alternately you can create one large assembly with many methods that are to be called and simply hang on to the application domain as long as needed. Still even without creating and deleting the domain operation is slower because of the proxy/remoting overhead.

Making life easier with wwScripting

There's a lot of power in all of that code – it shows how much flexibility there is in the .Net framework, but you certainly wouldn't want to put all of that code into your app each time you need to execute dynamically. It's reasonably easy to abstract all of this code into a class. You can find the code to such a class in the wwScript.cs source file and in the Westwind.Tools.Scripting namespace with the wwScripting class.

The class provides the following features:

  • Transparent execution of C# and CSharp code
  • Execution in the current AppDomain or via external AppDomains for shutdowns
  • Error handling
  • High level and low level methods

With the class running dynamic code gets a bit easier as shown in Listing 5.

Listing 5 (wwScriptingForm.cs): Using the wwScripting class to execute code dynamically

using Westwind.Tools.Scripting;

// *** Specify language: CSharp or VB

wwScripting loScript = new wwScripting("CSharp");

// *** get the source code from a text box

string lcCode = this.txtCode.Text;

// *** Optionally save the generated code for review

loScript.lSaveSourceCode = true;

// *** add any assemblies and namespaces required

loScript.AddAssembly("system.windows.forms.dll",

"System.Windows.Forms");

loScript.AddNamespace("System.IO");

// *** Execute the actual code with 3 parameters

lcResult = (string) loScript.ExecuteCode(lcCode,

"rick strahl",(int) x,(decimal) 10 );

if (loScript.bError)

MessageBox.Show(loScript.cErrorMsg + "\r\n\r\n" +

loScript.cSourceCode);

else

MessageBox.Show(lcResult);

loScript.Dispose() // *** force release of resources

If you want to load the code into a different AppDomain call the CreateAppDomain("Name") method before the ExecuteCode() method call.

The class also includes several methods for executing code. For example, ExecuteMethod() allows you to provide a full method including the signature defining parameters and return values. This makes it possible to create properly typed parameters and return values. For example take a code snippet like this:

public string Test(string lcName, int x) {

string cHello;

cHello = lcName;

MessageBox.Show(cHello,"Compiler Demo");

return DateTime.Now.ToString();

}

which you can then run with this code:

string lcResult = (string) loScript.ExecuteMethod(lcCode,"Test","rick strahl",x);

Notice that you can access the parameters directly by name in the dynamic code snippet. It's a little cleaner if you pass parameter and return values this way. You can also pass multiple methods as a string:

public string Test(string lcName, int x) {

string cHello;

cHello = lcName;

MessageBox.Show(cHello,"Compiler Demo");

return DateTime.Now.ToString();

}

public string Test2(string lcName, int x) {

return Test(lcName,x);

}

You can then call the two methods like this:

string lcResult = (string) loScript.ExecuteMethod(

lcCode,"Test","rick strahl",(int) x);

lcResult = (string) loScript.CallMethod(loScript.oObjRef,

"Test2","rick strahl",(int) x);

Note that making the second call is rather more efficient because the object already exists and is loaded. No recompilation or regeneration occurs on this second call.

CallMethod() is one of the lower level methods of the class. With it you can perform each step of the compile process individually. A number of other low level methods are available:

Low Level Method

Function

Parameters

CompileAssembly

Compiles an assembly and hold an internal pointer to the assembly object (only if locally loaded – app domains are handled from disk).

lcSource
Source code

CreateInstance

Creates an instance of the compiled code either in the local or a remote AppDomain. Sets the oObjRef property with the reference to the object or interface.

None
Uses internal references to the Assembly or the name of the DLL file to load into an AppDomain.

CallMethod

Executes a method by name using the oObjRef pointer. Knows about local or remote AppDomain.

lcMethod

The method to call.

Parameters()
A variable list of parameters from 0 to n.

CreateAppDomain

Creates an AppDomain and forces CreateInstance and CallMethod to use that domain to load and execute code in.

lcAppDomainName

Name of the domain

Dispose

Cleans up and releases references.

None

Property

Function

bError

Error Flag that should be checked after making calls before using any results.

cErrorMsg

Contains error information either after compiling or running code.

lSaveSourceCode

Determines whether the code that is finally compiled is saved. Full assembly source code.

cSourceCode

Set before compilation if lSaveSourceCode is true.

oObjRef

After a successful method execution (or after calling CreateInstance) this property contains an instance of the dynamic object.

cAssemblyNamespace

Name of the namespace that the code is generated into. This is used to generate the assembly and then used again when the class is instantiated to reference the type.

cClassname

Same as cAssemblyNamespace

lDefaultAssemblies

Determines if certain assemblies and namespaces are loaded by default. Loads System, System.IO, System.Reflection.

wwScripting and custom code

Why do we need a script parser?

Building an ASP like script parser

To show you how useful dynamic code execution is and how little code it takes to build powerful functionality, I've included another class wwASPScripting and a small sample app that demonstrates it with the source code. It's basically a simple ASP template parser you can use in your own non-Web applications. In .Net we already have ASP.Net which is very powerful at parsing content from templates into HTML or any other output. It works well for the Web, but unfortunately the scripting engine is completely tied to the HTTP engine and so you can't use this same kind of templating in your own non-Web code.

While ASP.Net is extremely powerful and easy use it only works with Web

However, it's not too difficult to build a basic parser that can handle this task. Take a look at Figure 2, which shows both the generated C# code and the output.


Figure 2 – The wwASPScripting class in conjunction with the wwScripting class can run C# based script code that works with basic ASP syntax.

If you look closer at Figure 2 you can see that what happens behind the scenes. The HTML template is turned into C# source code. The parser simply runs through the page finding all of the <% tags and inserts the appropriate Response.Write() or Response.oSb.Append() commands. Non tagged text is expanded into strings delimited with quotes. As a special case the <%@ %> directive handles Assembly and Import keyword to allow importing namespaces and assembly files for linking. To include assemblies and namespaces you can use directives like this:

<%@ Assembly name="System.Windows.Forms.dll"%>

<%@ Import namespace="System.Windows.Forms"%>

I put a separate class wwASPScripting together to handle the parsing of a string into C# code. it's only a demo and provides rudimentary functionality – a first stab. This parser also only handles C# code at this time as VB code would require generating code quite differently and my VB skills lack a bit in that department.

The code to accomplish parsing of a template page then looks like this:

Listing 6 (wwAspscriptingForm.cs): Executing a C#/ASP template

// *** Use the script object to parse the template

// *** into runnable code

wwASPScripting oASP = new wwASPScripting();

string lcCode = oASP.ParseScript(this.txtCode.Text);

// *** Access the built-in Script processor – will have

// *** Namespaces loaded from <%@ %> directives

wwScripting loScript = oASP.oScript;

loScript.lSaveSourceCode = true;

// loScript.CreateAppDomain("wwScriptDomain");

string lcResult = (string) loScript.ExecuteCode(lcCode);

if (loScript.bError)

MessageBox.Show(loScript.cErrorMsg + "\r\n\r\n" + loScript.cSourceCode);

else

{

MessageBox.Show(lcResult,"Script Output");

MessageBox.Show(loScript.cSourceCode,"Generated Assembly Source Code");

}

loScript.Dispose();

The key and new feature of this code is the ParseScript method which basically turns the ASP style code seen in Figure 2 into runnable C# code which is then passed to the wwScripting class to dynamically execute.

The wwASPScripting class is only a first shot and doesn't do much else than parse. But it has a private implementation of a Response object that is used to write output into the output stream which natively uses a string build. The ParseScript method is rather short and you can review the source code of how the code conversion is performed in the wwAspScripting.cs source file included with the downloadable code.

You're so dynamic!

It is interesting how .Net allows you to run dynamic code – essentially it provides you all the tools that a compiler uses to generate an executable. If you want to get even more low level you can use the System.Reflection.Emit namespace to generate IL level code directly. What is amazing is how little overall code this mechanism requires even if coming up with that code wasn’t quite so trivial digging through the .Net docs (and help from several people on the newsgroups especially!). It's also interesting to see how to apply this technology and build a custom script parser with even less code. The process is relatively easy and straightforward especially once the wrapper classes can be utilized. Well, easy may be overstated. This whole exercise requires deployment of two DLLs in your apps – the wwScripting dll that holds both the code execution and scripting classes as well as the remote loader DLL required to handle the AppDomain proxy interface. But regardless I hope these classes and this discussion have helped you understand how you can run dynamic code in .Net. I know I've learned a lot about how .Net works under the covers and I hope this article and the provided helper classes are useful to you in extending your applications with dynamic code. I know I couldn't live without this capability in my applications…

As usual if you have any questions or comments, please comment on the Code Message board at:

http://www.west-wind.com/wwthreads/default.asp?Forum=Code+Magazine. Source code for this article is available at: http://www.west-wind.com/presentations/dynamicCode/dynamicCode.zip.

Thursday, November 09, 2006

DirectShow开发环境--趴趴熊爱程序

1,使用VC向导生成一个具体项目,如Win32 Dynamic-Link;

2,包含头文件streams.h;

3,在VC的菜单中选择Project|Settings|C/C++,在弹出的对话框中的Category中选择Code generation,然后在Calling convention中选择_stdcall;

4,使用多线程语言运行时库,即在VC的菜单中选择Project|Settings|C/C++,在弹出的对话框中的Category中选择Code generation,然后在Use run-time library中,Debug版选择Debug Multithreaded,Release版选择Multithreaded。

5,配置必要的链接库文件,即在VC的菜单中选择Project|Settings|Link,在弹出的对话框中的Category中选择General,然后在Object/library modules中输入如下代码:

Debug版本 strmbasd.lib, msvcrtd.lib, winmm.lib

Release版本 strmbase.lib, msvcrt.lib, winmm.lib

并且选中Ignore all default libraries。

DirectShow SDK建议,DirectShow应用程序应该至少连接库文件strmiids.lib和quartz.lib。前者定义了DirectShow标准的 CLSID和IID,后者定义了导出函数AMGetErrorText(如果应用程序中没有使用到这个函数,也可以不连接这个库)。如果程序里包含了头文件streams.h,则一般库文件还要连接strmbasd.lib、uuid.lib、winmm.lib。

6,将DirectX SDK的Include和Lib目录配置到VC的系统目录中去,并且放在标准的VC目录之前,以保证编译器能够拿到最新版本的源文件。选择 Tools|Options|Directories,在弹出的对话框中的Show directories for中选择Include files,配置如下(假设DirectX SDK安装在D:\DXSDK目录下,VC安装在C:\Program Files下):

D:\DXSDK\Include

D:\DXSDK\SAMPLES\C++\DIRECTSHOW\BASECLASSES

D:\DXSDK\SAMPLES\C++\COMMON\INCLUDE

C:\Program Files\Microsoft Visual Studio\VC98\INCLUDE

C:\Program Files\Microsoft Visual Studio\VC98\MFC\INCLUDE

C:\Program Files\Microsoft Visual Studio\VC98\ATL\INCLUDE

再在Show directories for中选择Library files,配置如下:

D:\DXSDK\Lib

D:\DXSDK\SAMPLES\C++\DIRECTSHOW\BASECLASSES\DEBUG

D:\DXSDK\SAMPLES\C++\DIRECTSHOW\BASECLASSES\RELEASE

C:\PROGRAM FILES\MICROSOFT SDK\LIB

C:\Program Files\Microsoft Visual Studio\VC98\LIB

C:\Program Files\Microsoft Visual Studio\VC98\MFC\LIB

7,因为DirectShow应用程序是一种COM客户程序,因此在调用任何COM函数之前调用CoInitialize()(或 CoInitializeEx)函数进行COM库的初始化(一般是在应用程序启动的时候调用一次),在结束COM库使用时调用 CoUninitialize()函数进行反初始化(一般是在应用程序退出前调用一次)。

Wednesday, August 23, 2006

Code Shifting

--------------------------------------------------------------------------------

Code Shifting

Written: Saturday 11th Nov, 2003
--------------------------------
Revised: Thursday 8th July, 2004
--------------------------------
Subject: Code Shifting
----------------------
Author: spookie
---------------

CodeShifting happens simply when a DLL is dynamically loaded with LoadLibrary() by the game,
and inside this DLL is the game code that you need to edit. This is becoming more and
more common these days because of the amount of games coming out based on generic engines.
To name a few, Max Payne 2, Splinter Cell, Tron 2.0 and Counter-Strike: Condition Zero.
Out of these four chart topping games, only two working trainers were released. There was
an additional obstacle with Tron 2.0, being based on the Lithtech engine, as not only is the
game code in a dynamically loaded dll, it drops this dll in the temp files with a random
name before loading.

Now, to get around this problem we've got to find the address at which our game code dll
has been loaded, and with the help of the ToolHelp32 APIs it's not a hard task. First of
all, you need to know what dll your code is inside...

So for example lets say our cheat address is 008C3627 - We need to find a dll in the
currently running target game that's loaded at an address lower than 008C3627, but with an
image size of more than 008C3627. If you have LordPE handy (http://protools.cjb.net/) you
can get a list of loaded dlls and their base address and image size by selecting the game
process in the upper list and the dlls will then be displayed in the lower list. Or, you
could use a small tool, codeloc, I created for this, available at http://spookie.rom.cd/.
codeloc takes 2 parameters, the first is the game window text or the game window class
name, the second is your cheat location. e.g...

--------------------------------------------------------------------------------code---
codeloc "MaxPayne2" 8C3627

...will result as...

<008C3627> is inside module "X_GameObjectsMFC.dll" with base address <008B0000>
-------------------------------------------------------------------------------/code---

Now we know what dll we're working inside, we have to offset our given cheat address by
the current base address. Given the above example, my cheat address was 8C3627, and the dll
that contains it was loaded at 008B0000, so our cheat address offset would be 13627...

Cheat Address - Base Address = Cheat Offset
8C3627 - 8B0000 = 13627

So now we have our new cheat address we have to get the base address of the dll within our
trainer at runtime. To do this we'll use the ToolHelp32 APIs to cycle through all the
loaded "Modules" of the target game until we find ours.

Here is the code to do so in Delphi...

--------------------------------------------------------------------------------code---
uses TlHelp32;

function GetModuleBaseAddress(iProcId: Cardinal; DLLName: String): Cardinal;
var
hSnap: THandle; // Process snapshot handle.
xModule: ModuleEntry32; // Module information structure.
begin
Result:= 0; // If the result of the function is 0, it didn't find the base address.
// i.e.. the dll isn't loaded.
hSnap:= CreateToolHelp32Snapshot(TH32CS_SNAPMODULE, iProcId); // Creates a module
// snapshot of the
// game process.
xModule.dwSize:= SizeOf(xModule); // Needed for Module32First/Next to work.
If Module32First(hSnap, xModule) Then Begin // Gets the first module.
While Module32Next(hSnap, xModule) Do // Loops through the rest of the modules.
If xModule.szModule = DLLName Then // If this is the module we want...
Result:= Cardinal(xModule.modBaseAddr); // Save the base address it in result.
End;
CloseHandle(hSnap); // Free the handle.
end;
-------------------------------------------------------------------------------/code---

...and here is the same thing in C...

--------------------------------------------------------------------------------code---
#include

DWORD GetModuleBaseAddress(DWORD iProcId, char* DLLName)
{
HANDLE hSnap; // Process snapshot handle.
MODULEENTRY32 xModule; // Module information structure.

hSnap = CreateToolhelp32Snapshot(TH32CS_SNAPMODULE, iProcId); // Creates a module
// snapshot of the
// game process.
xModule.dwSize = sizeof(MODULEENTRY32); // Needed for Module32First/Next to work.
if (Module32First(hSnap, &xModule)) // Gets the first module.
{
while (Module32Next(hSnap, &xModule)) // Loops through the rest of the modules.
{
if (strcmp(xModule.szModule, DLLName) == 0) // If this is the module we want...
{
CloseHandle(hSnap); // Free the handle.
return (DWORD)xModule.modBaseAddr; // return the base address.
}
}
}
CloseHandle(hSnap); // Free the handle.
return 0; // If the result of the function is 0, it didn't find the base address.
// i.e.. the dll isn't loaded.
}
-------------------------------------------------------------------------------/code---

...and finally in masm...

--------------------------------------------------------------------------------code---
GetModuleBaseAddress proc iProcID:DWORD, DLLName:DWORD
LOCAL hSnap:DWORD
LOCAL xModule:MODULEENTRY32
invoke CreateToolhelp32Snapshot, TH32CS_SNAPMODULE, iProcID
mov hSnap,eax
mov xModule.dwSize, sizeof xModule
invoke Module32First, hSnap, addr xModule
test eax, eax
jnz getdll
mov eax, 0
ret
getdll:
invoke Module32Next, hSnap, addr xModule
test eax, eax
jnz checkdll
mov eax, 0
ret
checkdll:
invoke lstrcmpi, DLLName, addr xModule.szModule
test eax, eax
jnz getdll
mov eax, xModule.modBaseAddr
ret
GetModuleBaseAddress endp
-------------------------------------------------------------------------------/code---

These functions require the ProcessID of the game (GetWindowThreadProcessID()), and the
name of the dll you want the base address of. I've learnt the hard way that you should
use this function before doing any injection, not when the game has first started,
because some games like to reload the game dlls when you change level (Counter-Strike: CZ)

In the end, the lpBaseAddress parameter you will send to WriteProcessMemory will be...

--------------------------------------------------------------------------------code---
GetModuleBaseAddress(GAME_PROCESS_ID, "X_GameObjectsMFC.dll") + 0x13627
-------------------------------------------------------------------------------/code---

Any problems just email me at: spo@kie.org.uk

- spookie

Class DynamicArray

Class DynamicArray
'************** Properties **************
Private aData
'****************************************
'*********** Event Handlers *************
Private Sub Class_Initialize()
Redim aData(0)
End Sub
'****************************************
'************ Property Get **************
Public Property Get Data(iPos)
'Make sure the end developer is not requesting an
'"out of bounds" array element
If iPos <> UBound(aData) then
Exit Property 'Invalid range
End If
Data = aData(iPos)
End Property
Public Property Get DataArray()
DataArray = aData
End Property
'****************************************
'************ Property Let **************
Public Property Let Data(iPos, varValue)
'Make sure iPos >= LBound(aData)
If iPos < LBound(aData) Then Exit Property
If iPos > UBound(aData) then
'We need to resize the array
Redim Preserve aData(iPos)
aData(iPos) = varValue
Else
'We don't need to resize the array
aData(iPos) = varValue
End If
End Property
'****************************************
'************** Methods *****************
Public Function StartIndex()
StartIndex = LBound(aData)
End Function
Public Function StopIndex()
StopIndex = UBound(aData)
End Function
Public Sub Delete(iPos)
'Make sure iPos is within acceptable ranges
If iPos <> UBound(aData) then
Exit Sub 'Invalid range
End If
Dim iLoop
For iLoop = iPos to UBound(aData) - 1
aData(iLoop) = aData(iLoop + 1)
Next
Redim Preserve aData(UBound(aData) - 1)
End Sub
'****************************************
End Class

Wednesday, July 19, 2006

Visual Basic Secrets

Visual Basic Secrets

Visual Basic Secrets

Copyright� 2002 by Kevin Wilson


Introduction
Using Pointers In Visual Basic
VarPtr, StrPtr, and ObjPtr
ByRef / ByVal
AddressOf and Callbacks
Accessing "Hidden" API's


Introduction:

Visual Basic is called a "Rapid Application Development (RAD) Development Tool" because it was designed to take care of the Windows "ground work" for you, thus allowing you to concentrate on the important stuff like the program's functionality and documentation.

For example, when you open VB and add a standard "Form" to your project, there's A LOT that goes into putting that form on the screen when you hit "F5" to execute the program and simply display the form. You have to call the "CreateWindow" to actually create the Form and give it it's properties that make up it's interface. You then have to modify it's text font, forecolor, backcolor, device context, etc. by calling various Win32 API's. Lastly, you have to hook into the Windows messages that are being sent to the newly created form by subclassing it and then catching and processing each Windows messages properly via a "WindowProc" callback function. More complex interfaces require more complex object creation and handling functionality to be programmed into the form. C and C++ programmers actually have to create all that object creation, message handling, and object destruction code by hand (or have a template of it generated).

Visual Basic's ability to do the "basics" for you like this is a powerful thing to programmers who know how to correctly use VB as a development tool, but also puts a lot of power into the hands of people that don't know much about programming. Visual Basic is mocked by C/C++ programmers because of this. They say, "Anyone can develop with VB, but it takes a real programmer to develop with C/C++." I say that the SMART programmer chooses Visual Basic because VB eliminates potential bugs in your object creation, message handling, and object destruction routines, VB offers easier and quicker handling of Windows events, VB gives you a more robust interface capabilities, VB gives you easier access to COM objects and third party controls, VB is easier to read because it is very close to reading English where C/C++ is VERY cryptic, VB allows you easy access to the Win32 API (which gives the programmer the ability to tap into the power of Windows), and on top of ALL THAT... Visual Basic has the ability to hook into the power and speed of C/C++ via components, libraries, and other code written in C/C++. Heh... where's the bragging rights now? :)

Here's the thing though... even VB programmers that have been in the industry for years don't realize the real power of VB because they don't grasp (or realize) a few key concepts and functionalities that VB offers. These concepts aren't taught, or at least are not emphasized the way they should, so I call them "VB SECRETS".

^ TOP ^

Using Pointers In Visual Basic:

I was once asked in a job interview a question that I now realize was a TRICK QUESTION. The question was, "Does Visual Basic have or use 'pointers' ?" The obvious answer to anyone that uses Visual Basic is "NO". You don't see pointer declarations and macros in VB like you do in C/C++... and that's what I think the interviewer was getting at. She accepted my answer with that reasoning. However, the correct answer should have been "YES".

Visual Basic (like just about every other programming language) does use pointers... EXTENSIVELY. The difference is, Visual Basic hides them from you whenever possible, or calls them something different so as to not burden you with the formalities and protocols required when using them.

I will proceed to explain how you can use pointers to access information held in variables directly (VarPtr / StrPtr / ObjPtr), pass information to functions by pointers (ByRef / ByVal), and retrieve and pass pointers to functions (AddressOf).

^ TOP ^

VarPtr, StrPtr, and ObjPtr:

The VB Functions "VarPtr" (Variable Pointer), "StrPtr" (String Pointer), and "ObjPtr" (Object Pointer) are UNDOCUMENTED, UNSUPPORTED functions that Microsoft has made available in Visual Basic 5.0, and 6.0. These functions (along with many others) are no longer available in VB.net. These functions allow you to get the address in memory where VB variables (pointers) are, as well as the address in memory where the actual data that the variables point to are. The reason these functions are so useful is because if you know the memory address of data, you can manipulate it, copy it, or pass it around directly instead of relying on VB to do it for you. This is MUCH faster and (in some cases) gives you the ability to do things that VB on it's own can't do.

This is what Microsoft's MSDN says about "VarPtr":

This function can be used to get the address of a variable or an array element. It takes the variable name or the array element as the parameter and returns the address. However, you should be aware that unlocked Dynamic Arrays may be reallocated by Visual Basic, so you must be very careful when you use VarPtr to get the address of an array element.

The following example gets the address of a variable:

Dim lngVariableAddress As Long
Dim dblMyVariable As Double
lngVariableAddress = VarPtr(dblMyVariable)

This example gets the address of the fourth element of an array:

Dim lngElementAddress As Long
Dim lngArrayOfLongs(9) As Long
' The following will get the address of the 4th element in the array
lngElementAddress = VarPtr(lngArrayOfLongs(3))

Limitations: The VarPtr function cannot be used to get the address of an array...

This is what Microsoft's MSDN says about "StrPtr":

Strings in Visual Basic are stored as BSTR's. If you use the VarPtr on a variable of type String, you will get the address of the BSTR, which is a pointer to a pointer of the string. To get the address of the string buffer itself, you need to use the StrPtr function. This function returns the address of the first character of the string. Take into account that Strings are stored as UNICODE in Visual Basic.

To get the address of the first character of a String, pass the String variable to the StrPtr function.

Example:

Dim lngCharAddress As Long
Dim strMyVariable As String
strMyVariable = "Some String"
lngCharAddress = StrPtr(strMyVariable)

You can use this function when you need to pass a pointer to a UNIOCODE string to an API call.

This is what Microsoft's MSDN says about "ObjPtr":

ObjPtr takes an object variable name as a parameter and obtains the address of the interface referenced by this object variable.

One scenario of using this function is when you need to do a collection of objects. By indexing the object using its address as the key, you can get faster access to the object than walking the collection and using the Is operator. In many cases, the address of an object is the only reliable thing to use as a key.

Example:

objCollection.Add MyObj1, CStr(ObjPtr(MyObj1))
...
objCollection.Remove CStr(ObjPtr(MyObj1))

Note that the "Limitations" at the end of the information about "VarPtr", it said that you can't use VarPtr to get the address of an array. That's true... to a point. You can't pass the variable "MyArray" to it (because VB stores arrays in an OLE object called a "SafeArray"), but if you get the address of the first element of the array "MyArray(0)", you have the address of the whole array because arrays elements are stored in memory contiguously (in numeric order from the first to the last). So if a Win32 API, or a C/C++ function asks for a pointer to a byte array, like this:

Option Explicit
Private Type POINTAPI
X As Long
Y As Long
End Type


'BOOL Polyline(
' HDC hDC, // handle of device context
' CONST POINT *lpPT, // address of array containing endpoints
' int cPoints // number of points in the array
');

Private Declare Function Polyline Lib "GDI32.DLL" (ByVal hDC As Long, _

ByRef
lpPT As Any
, ByVal cPoints As Long) As Long

You could call it like this:

Private Sub Form_Load()
Dim ThePoints() As POINTAPI
Me
.AutoRedraw = True
Me
.Visible = True
Me
.Move 0, 0, Me.Width, Me.Height
ReDim ThePoints(1 To 5) As POINTAPI
ThePoints
(1).X = 0: ThePoints(1).Y = 0
ThePoints
(2).X = 100: ThePoints(2).Y = 0
ThePoints
(3).X = 100: ThePoints(3).Y = 100
ThePoints
(4).X = 0: ThePoints(4).Y = 100
ThePoints
(5).X = 0: ThePoints(5).Y = 0
If Polyline(Me.hDC, ByVal VarPtr(ThePoints(1)), 5) = 0 Then Debug.Print "FAILED!"
Me
.Refresh
Erase ThePoints
End Sub

NOTE: Be careful about storing pointers to dynamic arrays because when arrays are reallocated, resized, or redim'ed... it is very possible that you'll have a completely new memory address for the actual data.

For more information on VarPtr, StrPtr, and ObjPtr, see the following:

http://support.microsoft.com/default.aspx?scid=kb;en-us;Q199824
http://msdn.microsoft.com/library/en-us/dnw32dev/html/ora_apiprog6_topic1.asp
http://msdn.microsoft.com/library/en-us/dnovba00/html/LightningStrings.asp
http://msdn.microsoft.com/library/en-us/dnovba01/html/Lightweight.asp

^ TOP ^

ByRef / ByVal:

By far the biggest problem VB programmers run into while working with Win32 API's (or any exported C/C++ function for that matter) is correctly passing the required parameters to the function. Inserting a "ByRef" where a "ByVal" should've been (or visa versa), or passing a value or variable when the function was expecting a pointer can be the one difference between the function being called working perfectly or causing Windows to crash and burn. Understanding how to pass parameters correctly takes an understanding of how Windows programs work with "calling stacks" and memory allocation between the calling program and the function being called.

First of all, lets discuss what a "call stack" is and how it relates to memory allocation when passing parameters to a function. The "call stack" is simply a spot in memory where the variables and values being passed to and from a function are stored. It's called a "stack" because parameter values follow one after the other in memory and are accessed with that assumption. Because of this, parameters are in a way "stacked on top of eachother" to make up all the information being given to the function. When a parameter is added to a function's call stack, it is said to be "pushed" onto the call stack. When a parameter is removed (or cleaned up) from a function's call stack, it is said to be "popped" off the call stack. The terms "stack", "push", and "pop" are assembler terms (yes, we are that low-level at this point) and if you were to decompile a program or DLL into assembly, you'd see lines with the words "push", "pop", etc.

When Visual Basic calls a Win32 API (or any exported C/C++ function), it expects the function to use the "Standard Calling Convention" ( __stdcall ) as apposed to the default C/C++ calling convention ( __cdecl ). This means that when the function is called, parameters are passed into memory (or pushed onto the stack) from right to left, and the function being called is responsible for cleaning up the memory passed to it (or popping the memory passed off of the stack). If you try to call an exported function that is declared with any other calling convention but __stdcall, Visual Basic will not know how to handle the stack and parameters being passed back and forth so you will get a message from VB saying "Bad DLL Calling Convention".

For a more in-depth and advanced explanation of how memory is allocated and deallocated when calling parameters, and what call stacks are and how they work in Windows, I strongly recommend reading an EXCELLENT book by Dan Appleman (Desaware) called "Dan Appleman's Win32 API Puzzle Book and Tutorial for Visual Basic Programmers".

Now lets back up a little and get out of the inner workings of Windows memory and get back to working with VB. When you call a function, you can pass parameters to it in one of two ways. You can pass it an explicit value that you want the function to take and work with, or you can pass it a pointer to information already present in memory. When you're passing simple information like numbers, sizes, flags, etc. you want to pass the information in ByVal (meaning By Value) because you want it to take the value of what you are passing, not the memory address of where that value is currently being held. Now when you want to pass more complex data to a function like a data type, an array of values, or an object reference, you need to pass a reference (or pointer) to the function telling it where in memory the data is. This is done by specifying the ByRef (meaning By Reference) keyword. The function will then go to that point in memory and read the data that applies. The exception to this is when you pass strings to Win32 API calls (or any C/C++ exported function). Always pass strings ByVal to API's (unless you're passing a string array... in which case you'd use ByRef, or just pass the first element of the array ByVal).

So at this point you're saying, "I already know about passing parameters ByRef/ByVal". Yes, but did you realize that what you're doing when you pass "ByRef" is passing a pointer? If you take that concept a step further, you can do things like make function interfaces more generic (thus opening them up for more broad application) by altering the "ByRef" to "ByVal" and passing an explicit pointer to the data you want to pass. So instead of declaring your function like this:

Option Explicit

Private Type RECT
Left As Long
Top As Long
Right As Long
Bottom As Long
End Type

'int FillRect(
' HDC hDC, // handle to device context
' CONST RECT *lpRC, // pointer to structure with rectangle
' HBRUSH hBR // handle to brush
');

Private Declare Function FillRect Lib "USER32.DLL" (ByVal hDC As Long, _
ByVal lpRC As Long, ByVal hBR As Long) As Long
Private Declare Function CreateSolidBrush Lib "GDI32.DLL" (ByVal crColor As Long) As Long
Private Declare Function DeleteObject Lib "GDI32.DLL" (ByVal hObject As Long) As Long

Private Sub Form_Load()
Dim hBrush As Long
Dim MyRect As RECT

Me.AutoRedraw = True
Me.Visible = True
Me.Move 0, 0, Me.Width, Me.Height
With MyRect
.Top = 0: .Left = 0: .Right = 100: .Bottom = 100
End With

hBrush = CreateSolidBrush(vbRed)
If hBrush = 0 Then Exit Sub
If FillRect(Me.hDC, VarPtr(MyRect), hBrush) = 0 Then Debug.Print "FAILED!"
Me.Refresh
DeleteObject hBrush
End Sub

If you think about it, this gives you all kinds of options when declaring functions and parameters. You're not restricted anymore to the exact variable type. You could make EVERYTHING "Long" variable types and pass pointers around to everything (as long as you were careful about how you did it). So you're having trouble passing that monster custom type around, FORGET ABOUT IT... pass it with a pointer. So you're having problems passing that object around, FORGET ABOUT IT... pass it with a pointer. VB 5.0 won't let you return variable arrays (or funky types and objects) as the return type for a function, FORGET ABOUT IT... pass back a long value representing where the array is in memory and use the CopyMemory API to copy it down into a local array. See where I'm going with this? :)

If you just said, "NO"... using VarPtr, StrPtr, and ObjPtr in conjunction with ByRef and Byval allows you to pass around data in ANY format if you know what you're doing.

^ TOP ^

AddressOf and Callbacks:

The "AddressOf" operator is all about callbacks. "But what is a call back?" you ask. A callback is the Windows equivalent of a VB "Event". In fact, VB events (on a very low level) are just about always triggered by callback functions that catch the original event in the form of a Windows Message. Callbacks are most often seen within the Win32 API (and other C/C++ code) where notification of user and/or Windows activity is required or desired within your application. You don't see callbacks within VB much because VB handles messaging and notification via "Events", which are much easier and safer to deal with compared to callbacks and all that goes into them.

Lets say that you wanted to receive notification of EVERY message that Windows is sending to a Form within your project (even ones that you'd never use), along with a few custom messages that may be sent to your Form via some other API call(s). What you would do is setup a callback function that is recognized by Windows ("WindowProc") and then tell Windows (via the "SetWindowLong" API) to send all of it's messages meant for your Form to your callback function instead, so you can inspect them and/or react to them... and then send them on their way (via the "CallWindowProc" API). Doing this is called "Sub-Classing" and is a very powerful (but at the same time very dangerous) technique that you can use to redraw your Form, it's menus, and/or it's contents in a custom manner (or whatever you want to do with your Form).

There two draw-backs to using "AddressOf":

1) You can only retrieve the address to a function or sub (public or private) contained within a Standard VB Module. There's no way around this.

2) It can only be called as part of a parameter list to a function or sub. The way to get around it is like this:

Option Explicit

Public Sub Main()
Dim lngProcAddress As Long
lngProcAddress = GetProcAddress(AddressOf WindowProc)
End Sub

Public Function GetProcAddress(ByVal lngAddressOf As Long) As Long
GetProcAddress = lngAddressOf
End Function

Public Function WindowProc(ByVal hWnd As Long, ByVal uMsg As Long, _
ByVal wParam As Long, ByVal lParam As Long) As Long
' <>
End Function

You'll notice that we pass the "AddressOf" with the name of the function we want to get the address (memory pointer) of to the function "GetProcAddress" which simply returns back that value. Simple concept and is very effective. The addresses of functions and subs doesn't change so you could store the address of the functions and subs you want to reference this way so you don't have to repeatedly call AddressOf, etc.

"So lets see it in action!" you say... OK!

Here's an example of "sub-classing" as mentioned above:

Option Explicit

Private Const GWL_WNDPROC = (-4)
Private lngPrevProc As Long
Private lngHWND As Long

Private Declare Function SetWindowLong Lib "USER32.DLL" Alias "SetWindowLongA" _

(ByVal hWnd As Long, ByVal nIndex As Long, ByVal dwNewLong As Long) As Long
Private Declare Function CallWindowProc Lib "USER32.DLL" Alias "CallWindowProcA" _

(ByVal lpPrevWndFunc As Long, ByVal hWnd As Long, ByVal uMsg As Long, _
ByVal wParam As Long, ByVal lParam As Long) As Long

' This is the CALLBACK function that receives the messages for the specified hWnd
Private Function WindowProc(ByVal hWnd As Long, _

ByVal uMsg As Long, _
ByVal wParam As Long, _
ByVal lParam As Long) As Long
' Display the messages and their information in the IMEDIATE window
' NOTE: You can find out what messages are being passed by comparing the value of

' "uMsg" to Windows Messages (WM_*) constant values defined in the WINUSER.H file
Debug.Print _

"hWnd=" & hWnd & ", uMsg=" & uMsg & ", wParam=" & wParam & ", lParam=" & lParam

' Forward on the messages to where they were originally supposed to go. This MUST
' here or the window will become unresponsive because it will stop recieving messages
WindowProc = CallWindowProc(lngPrevProc, hWnd, uMsg, wParam, lParam)
End Function

' This function starts a new sub-classing
Public Function Subclass_Start(ByVal hWnd As Long) As Boolean
' Stop any previous sub-class
If Subclass_Stop = False Then Exit Function
' Attempt to start a new sub-class
lngPrevProc = SetWindowLong(hWnd, GWL_WNDPROC,
AddressOf WindowProc)
If lngPrevProc <> 0 Then
lngHWND = hWnd
Subclass_Start = True
End If
End Function

' This function stops any existing sub-classing
Public Function Subclass_Stop() As Boolean
' If no previous sub-class was started, just exit
If lngPrevProc = 0 Or lngHWND = 0 Then
Subclass_Stop = True
Else
' Set the message handling procedure back to what it originally was
If SetWindowLong(lngHWND, GWL_WNDPROC, lngPrevProc) <> 0 Then
Subclass_Stop = True
End If
End If
' Clear the variables used
lngPrevProc = 0
lngHWND = 0
End Function

Option Explicit

Private Sub Form_Load()
Subclass_Start Me.hWnd
End Sub

Private Sub Form_QueryUnload(Cancel As Integer, UnloadMode As Integer)
Subclass_Stop
End Sub

Here's an example of "enumeration", which is a VERY popular way for Windows to give you back information about "information lists" (like a list of all windows, a list of all the objects on a window, a list of all the installed fonts, etc):

Option Explicit

Private lngWinHandle() As Long
Private lngWinHandleCount As Long

Private Declare Function EnumWindows Lib "USER32.DLL" (ByVal lpEnumFunc As Long, _

ByVal lParam As Long) As Long

' This is the CALLBACK that enumerates through all windows in the current desktop
Private Function EnumWindowsProc(ByVal hWnd As Long, ByVal lParam As Long) As Long
' Incrament the array of window handles
lngWinHandleCount = lngWinHandleCount + 1
ReDim Preserve lngWinHandle(1 To lngWinHandleCount) As Long
' Add the information to the array
lngWinHandle(lngWinHandleCount) = hWnd
' Tell the function to keep going
EnumWindowsProc = 1
End Function

Public Function GetAllWindows(ByRef Return_Handles() As Long, _

Optional ByRef Return_WinCount As Long) As Boolean
' Clear any previous information
Erase lngWinHandle
lngWinHandleCount = 0
' Start enumerating through the windows
If EnumWindows(
AddressOf EnumWindowsProc, 0) <> 0 Then
Return_Handles = lngWinHandle
Return_WinCount = lngWinHandleCount
GetAllWindows = True
End If
Erase lngWinHandle
lngWinHandleCount = 0
End Function

Option Explicit
Private Sub Form_Load()
Dim lngWindows() As Long
Dim lngWindowsCount As Long
Dim lngCounter As Long
Dim strWindows As String
If GetAllWindows(lngWindows, lngWindowsCount) = True Then
If lngWindowsCount > 0 Then
For lngCounter = 1 To lngWindowsCount
strWindows = strWindows & " " & lngWindows(lngCounter) & Chr(13)
Next
End If
Me.AutoRedraw = True
Me.Print strWindows
End If
Erase lngWindows
End Sub

^ TOP ^

Accessing "Hidden" API's:

This part is definately the most "secret" of all the secrets decribed on this page. There are indeed MANY hidden Win32 API calls in Windows... the trick is to find them and find out how to call them because Microsoft sure won't tell you.

"But why hide them?" you ask? Because Microsoft adds extra functionality to the API that only they can gain access to. This gives their products an advantage when (running under Windows) over everyone else's because only they have access to more powerfull functionality, faster functionality, or extra functionality via these hidden API calls when everyone else has to make do with the regular, documented functionality exposed by Windows and the documented in the MSDN. "Unfair advantage" you say? DEFINATELY! But who ever said that Microsoft does business fairly... or ethically for that matter! These kinds of business practices are what are constantly keeping Microsoft in court and on newspaper headlines.

"What kind of hidden API's are out there, and how do I find out what they are and how to use them?" you ask? EXCELLENT question, and that's why I've included this here. There are many web pages out on the internet dedicated to finding these hidden API's and exposing their functionality to "level the playing field" and give the more cool functionality to developers like you and me! Here's a few good web pages on this:

http://www.geocities.com/SiliconValley/4942/index.html
http://www.users.qwest.net/~eballen1/nt.sekrits.html
http://www.mvps.org/vbnet/code/shell/undocshelldlgs.htm
http://www.mvps.org/vbnet/code/shell/undocformatdlg.htm
http://www.mvps.org/vbnet/code/shell/undocchangeicondlg.htm
http://www.mvps.org/vbnet/code/shell/undocshpaths.htm
http://www.ercb.com/ddj/1992/ddj.9211.html

You can find a few of these "hidden API's" in the modCOMDLG32.bas module under the "VB Standard Modules". They look like this:

Public Declare Function DLG_FindFile Lib "shell32.dll" Alias "#90" _
(ByVal pidlRoot As Long, ByVal pidlSavedSearch As Long) As Long
Public Declare Function DLG_FindComputer Lib "shell32.dll" Alias "#91" _
(ByVal pidlRoot As Long, ByVal pidlSavedSearch As Long) As Long

You'll notice that they are aliased by a number "#90", "#91", etc. These are called "Ordinal Numbers" and they are a way for you to expose API's through a DLL without exposing it's name. So if you wrote a function, and you wanted to use it but didn't want anyone else to, you could expose it by a number. This doesn't give anything away as to what it does or why it's there, but at the same time gives you access to it.

Sneaky, huh?! :)

WELL! That's all folks. If I think of any other "secrets" or neat "hidden" or "obscure" functionality within VB (or remember any that I meant to put here), I'll add them. Happy coding! =)