tag:blogger.com,1999:blog-7672108918948268442024-03-14T02:13:15.145+06:00Underground - Maruf Maniruzzaman's programming blogMAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.comBlogger16125tag:blogger.com,1999:blog-767210891894826844.post-27315109231557669802010-08-08T23:15:00.000+06:002010-08-08T23:15:15.590+06:00Just In Time Compiler for Managed Platform: Problem with exception handlingSo far we have generated code that is 100% equivallent to the methods byte code. But in case of exception this is a little problematic. Let me explain.<br />
<br />
For each generated method we have Prolog and Epilog that must be executed. If we skip the Epilog the native stack that us being by VM itself gets unstable- <br />
<br />
Again since we have compiled the entire method we should not do a separate compilation for the exception handler.<br />
<br />
Now that we wnat to execute the Epilog and also want to use the already compiled handler code. <br />
<br />
This part does not seems to be hard- but we dont know in advance which method will catch the exception.<br />
<br />
To solve the problem we can add an exception blog at the end of each method (a few bytes overhead for each method) and return a value from the native code to signal exception. The return value we used so far is always zero. So for exception we can use any non-zero value and also do the cleanup. <br />
<br />
When we find exception block we execute that block and continue. The way the compiler generates methods byte code actually do the nice job of managing code flow properly so our compiler already converted those code to native code and we dont need anything extra.MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com2tag:blogger.com,1999:blog-767210891894826844.post-35520222285256165842010-08-08T08:44:00.001+06:002010-08-10T04:32:04.178+06:00Just In Time Compiler for Managed Platform- Part 7: Native MethodsExecution of native methods are easiest of all methods :D. Since we dont need to generate anything for them - they are already native.<br />
<br />
We can not accept any native method for execution- Those should understand the stack. For native methods we fix the stack top to point to actual data- since there is nothing we need to generate- we dont need the clever stack pointer-<br />
<br />
The signature of each native method must match the following type signature:<br />
<br />
<pre>typedef Variable (*NativeMethod)(Context* pContext);
</pre><br />
Lets define our helper for native method. And here also we do not deal with machine instruction since there is nothing to parse- (well except the return type of the method)-<br />
<pre>u4 ExecuteNativeMethod(Context* pContext, CString strClassName, CString strMethod, CString strDesc)
{
JavaClass *pClass = pContext->pClass;
LOG(_T("Execute NativeMethod %s.%s%s \n"),strClassName , strMethod, strDesc);
CString strSignature= GetNativeMethodSignature(strClassName, strMethod, strDesc);
NativeMethod nativeMethod=pContext->pVMEnv->pNativeMethodProvider->GetNativeMethod(strSignature);
if(nativeMethod == NULL)
{
ASSERT(FALSE);
return -1;
}
else
{
Variable retVal = nativeMethod(pContext);
//if returns then get on stack
if(strDesc.Find(_T(")V")) < 0)
{
if(strDesc.Find(_T(")J")) < 0 && (strDesc.Find(_T(")D")) < 0))
{
//todo validate
pContext->stack[0]=retVal;
}
else
{
pContext->stack[0].intValue=0;
pContext->stack[1]=retVal;
}
}
}
return 0;
}
</pre><br />
OK, lets now define a simple native method that adds two numbers:<br />
<br />
<pre>Variable Add(Context* pContext)
{
Variable returnVal;
//The stack top is right in native methods-
returnVal.intValue
= pContext->stack[pContext->stackTop].intValue + pContext->stack[pContext->stackTop-1].intValue;
return returnVal;
}
</pre><br />
Thats all for native methods for now. We'll add some native methods to dynamically load native method-MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-767210891894826844.post-32348169146424234972010-08-06T07:52:00.016+06:002010-08-08T01:35:33.966+06:00Just In Time Compiler for Managed Platform- Part 6: Put Field and Get FieldNow we have our object on the heap. We need mechanism to <code>putfield</code> and <code>getfield</code>. The operation is very simple indeed. <br />
<br />
First we need a helper to get a field index (variable) in the object memory in the heap:<br />
<br />
<pre>
//And also we put the method names in HelperMethods structure
int GetFieldIndex(JavaClass *pTargetClass, Context *pContext, u2 index)
{
CONSTANT_Fieldref_info *pFieldInfo
= (CONSTANT_Fieldref_info *)pContext->pClass->constant_pool[index];
ASSERT(pFieldInfo->tag == CONSTANT_Fieldref);
u2 classIndex = getu2(&((u1 *)pFieldInfo)[1]);
u2 nameAndTypeIndex = getu2(&((u1 *)pFieldInfo)[3]);
CONSTANT_NameAndType_info *pNameTypeInfo
= (CONSTANT_NameAndType_info *) pContext->pClass->constant_pool[nameAndTypeIndex];
ASSERT(pNameTypeInfo->tag == CONSTANT_NameAndType);
u2 nameIndex = getu2(&((u1 *)pNameTypeInfo)[1]);
u2 descIndex = getu2(&((u1 *)pNameTypeInfo)[3]);
CString strFieldName, strFieldDesc;
if(!pContext->pClass->GetStringFromConstPool(nameIndex, strFieldName))
{
ASSERT(FALSE);
return -1;
}
if(!pContext->pClass->GetStringFromConstPool(descIndex, strFieldDesc))
{
ASSERT(FALSE);
return -2;
}
int superClassSize = 0;
JavaClass *pCurClass = pTargetClass;
int fieldIndex = -1;
while(true)
{
fieldIndex = pCurClass->GetFieldIndex(strFieldName, strFieldDesc);
pCurClass = pTargetClass->GetSuperClass();
if(fieldIndex >= 0)
{
fieldIndex += pCurClass ? pCurClass->GetObjectFieldCount() : 0;
break;
}
else
{
if(!pCurClass)
{
break;
}
}
}
ASSERT(fieldIndex>=0);
return fieldIndex;
}
</pre><br />
Now we define the method to execute the two instructions. We do not generate code foe this operaions and callback from the generated code since the operation is static and no parsing or instruction execution required-<br />
<br />
<pre>
#define PUT_FIELD_HELPER_INDEX 4
#define GET_FIELD_HELPER_INDEX 5
void PutField(Context *pContext, u2 index)
{
Variable obj=pContext->stack[pContext->stackTop-2];
Variable value=pContext->stack[pContext->stackTop-1];
Variable *pVarList = pContext->pVMEnv->pObjectHeap->GetObjectPointer(obj.object);
JavaClass *pTargetClass = (JavaClass *)pVarList[0].ptrValue;
ASSERT(pTargetClass && pTargetClass->magic == 0xCAFEBABE);
int fieldIndex = GetFieldIndex(pTargetClass, pContext, index);
pVarList[fieldIndex+1]=value;
pContext->stackTop-=2;
}
void GetField(Context *pContext, u2 index)
{
Variable obj=pContext->stack[pContext->stackTop-1];
Variable *pVarList=pContext->pVMEnv->pObjectHeap->GetObjectPointer(obj.object);
JavaClass *pTargetClass = (JavaClass *)pVarList[0].ptrValue;
ASSERT(pTargetClass && pTargetClass->magic == 0xCAFEBABE);
int fieldIndex = GetFieldIndex(pTargetClass, pContext, index);
pContext->stack[pContext->stackTop-1]=pVarList[fieldIndex+1];
}
</pre><br />
Now we generate instruction for them. Here is how we do it for <code>putfield</code>:<br />
<br />
<pre>void EmitCallPutField(u1* code, int &ip, u4 index)
{
u1 c[]={
//((void (*)(Context *pContext, u2 index))pContext->pVMEnv->ppHelperMethods[PUT_FIELD_HELPER_INDEX])(pContext, 0x1234);
0x8B, 0xF4, // mov esi,esp
0x68, 0x00, 0x00, 0x00, 0x00, // push index
0x8B, 0x45, 0x08, // mov eax,dword ptr [pContext]
0x50, // push eax
0x8B, 0x4D, 0x08, // mov ecx,dword ptr [pContext]
0x8B, 0x51, 0x10, // mov edx,dword ptr [ecx+10h]
0x8B, 0x42, 0x08, // mov eax,dword ptr [edx+8]
0x8B, 0x48, 0x10, // mov ecx,dword ptr [eax+10h]
0xFF, 0xD1, // call ecx
0x83, 0xC4, 0x08, // add esp,8
};
memcpy(c+3, &index, sizeof(index));
memcpy(&code[ip], c, sizeof(c));
ip+=sizeof(c);
}
void ExecutePutField(u1* code, int& ip, u2 index)
{
EmitCallPutField(code, ip, index);
}
</pre><br />
For <code>getfield</code> we just need to change the method pointer (add 4)<br />
<br />
<pre>mov ecx,dword ptr [eax+14h]
</pre><br />
So we can assign value to a class field and get that value when we need it. Next we need array handling. Comes next day.MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-767210891894826844.post-25621669947744948112010-08-05T06:44:00.001+06:002010-08-10T04:32:17.895+06:00Just In Time Compiler for Managed Platform- Part 5: Creating new object on heapLets create object on heap today.<br />
<br />
Since we have the object creation code in <code>JavaClass</code> it is very easy to create an object on the heap. We just call the <code>CreateObject</code> method of the current class that is already pushed on the stack by previous instructions:<br />
<br />
<pre>int CreateNewObject(Context *pContext, u2 index)
{
if(!pContext->pClass->CreateObject(index, pContext->pVMEnv->pObjectHeap, pContext->stack[pContext->stackTop].object))
return -1;
pContext->stackTop++;
return 0;
}
</pre><br />
Now let us create the helper methods for the <code>new</code> instruction:<br />
<br />
<pre>void EmitExecuteNew(u1* code, int &ip, u4 index)
{
u1 c[] = {
//((int (*)(Context *pContext, u2 index))pContext->pVMEnv->ppHelperMethods[EXECUTE_NEW_HELPER_INDEX])(pContext, index);
0x8B, 0xF4, // mov esi,esp
0x68, 0x00, 0x00, 0x00, 0x00, // push index
0x8B, 0x45, 0x08, // mov eax,dword ptr [pContext]
0x50, // push eax
0x8B, 0x4D, 0x08, // mov ecx,dword ptr [pContext]
0x8B, 0x51, 0x10, // mov edx,dword ptr [ecx+10h]
0x8B, 0x42, 0x08, // mov eax,dword ptr [edx+8]
0x8B, 0x48, 0x04, // mov ecx,dword ptr [eax+4]
0xFF, 0xD1, // call ecx
0x83, 0xC4, 0x08, // add esp,8
};
memcpy(c+3, &index, sizeof(index));
memcpy(&code[ip], c, sizeof(c));
ip+=sizeof(c);
}
void ExecuteNew(u1* code, int& ip, u2 index)
{
EmitExecuteNew(code, ip, index);
}
</pre><br />
With the call mechanism that wer built to call method is used to callback the object creation method here.<br />
<br />
And from the <code>Compile</code> method we just add a call for the <code>new</code> instruction:<br />
<br />
<pre>case _new:// 187(0xbb) -'new' is a keyword in C++ so we use '_new' :)
ExecuteNew(codes, ip, getu2(&bc[pc+1]));
pc+=3;
break;
</pre><br />
We actually have all the basic mechanism built. We just need to add things like native method support, garbase collector and helpers for remaining instruction. Those will be managed mostly in C++ since there is nothing to parse for them the optimizing C++ compiler does all the good thing for us.MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-767210891894826844.post-66494180124897225752010-08-03T02:12:00.000+06:002010-08-03T02:12:43.042+06:00Just In Time Compiler for Managed Platform- : Why stack is implemented wrong?Well, its not really wrong- its just efficient-<br />
<br />
The stack top is is always topvalue + 1. This is not right behaviour for stack usually- but what we do in our opetation is decrement the value first. It makes the offset zero which is efficient in terms of machine cycles. <br />
<br />
But why do that in the first place? <br />
<br />
This is because if we do not decrement the stack first before execution of the instruction it becomes complex in case of jmp (branch) instructions. Since we must decrement the stack no matter what we do this first. <br />
<br />
Now if we use a negative offset it'll take some extra cycles to decode the instruction with offset than witout the offset.<br />
<br />
This post is just to make sure we remember the stack top value points to invalid data. We must decrement it by one to get the top value.MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-767210891894826844.post-38064372197433414532010-08-02T00:19:00.001+06:002010-08-10T04:33:49.099+06:00Just In Time Compiler for Managed Platform- Part 4A: Conditional Branch CorrectionThe Jxx instructions I used for branching was not working right- It treated values unsigned. The "IA-32 Intel Architecture Software Developers Manual- Vol 2" describes this (page- 3-355): The terms "less" and "greater" are used for comparisons of <i>signed</i> integers and the terms "above" and "below" are used for <i>unsigned</i> integers. So, for signed comparison, we use JL, JG, JLE and JGE instead of JB, GA, JBE and JAE.<br />
<br />
With that change all branching instruction seems working now. We need two helper method for all of them. <br />
<br />
First one is comparison with zero: <br />
<br />
<pre>void IfXX(u1* code, int& ip, int targetpc, CMapPtrToPtr *pJmpTargetMap, u1 XX)
{
u1 c[] = {
//pContext->stackTop--;
0x8B,0x45,0x08, // mov eax,dword ptr [pContext]
0x8B,0x48,0x04, // mov ecx,dword ptr [eax+4]
0x83,0xE9,0x01, // sub ecx,1
0x8B,0x55,0x08, // mov edx,dword ptr [pContext]
0x89,0x4A,0x04, // mov dword ptr [edx+4],ecx
//if(pContext->stack[pContext->stackTop].intValue [XXoperator] 0)
0x8B, 0x45, 0x08, // mov eax,dword ptr [pContext]
0x8B, 0x48, 0x04, // mov ecx,dword ptr [eax+4]
0x8B, 0x55, 0x08, // mov edx,dword ptr [pContext]
0x8B, 0x02, // mov eax,dword ptr [edx]
0x83, 0x3C, 0xC8, 0x00, // cmp dword ptr [eax+ecx*8],0
0x0F, 0x00, 0x00, 0x00, 0x00, 0x00, // JXX <target>
};
memcpy(&code[ip], c, sizeof(c));
ip+=sizeof(c);
code[ip-5] = XX;
CreateJmpLink(&code[ip-5], targetpc, pJmpTargetMap);
}
void Ifle(u1* code, int& ip, int targetpc, CMapPtrToPtr *pJmpTargetMap)
{
IfXX(code, ip, targetpc, pJmpTargetMap, JLE);
}
void Ifeq(u1* code, int& ip, int targetpc, CMapPtrToPtr *pJmpTargetMap)
{
IfXX(code, ip, targetpc, pJmpTargetMap, JE);
}
void Ifne(u1* code, int& ip, int targetpc, CMapPtrToPtr *pJmpTargetMap)
{
IfXX(code, ip, targetpc, pJmpTargetMap, JNE);
}
void Iflt(u1* code, int& ip, int targetpc, CMapPtrToPtr *pJmpTargetMap)
{
IfXX(code, ip, targetpc, pJmpTargetMap, JL);
}
void Ifge(u1* code, int& ip, int targetpc, CMapPtrToPtr *pJmpTargetMap)
{
IfXX(code, ip, targetpc, pJmpTargetMap, JGE);
}
void Ifgt(u1* code, int& ip, int targetpc, CMapPtrToPtr *pJmpTargetMap)
{
IfXX(code, ip, targetpc, pJmpTargetMap, JG);
}
</pre><br />
<br />
Second one is comparison of any two numbers:<br />
<br />
<pre>void IfICmpXX(u1* code, int& ip, int targetpc, CMapPtrToPtr *pJmpTargetMap, u1 XX)
{
u1 c[]={
//pContext->stackTop -= 2;
0x8B, 0x45, 0x08, // mov eax,dword ptr [pContext]
0x8B, 0x48, 0x04, // mov ecx,dword ptr [eax+4]
0x83, 0xE9, 0x02, // sub ecx,2
0x8B, 0x55, 0x08, // mov edx,dword ptr [pContext]
0x89, 0x4A, 0x04, // mov dword ptr [edx+4],ecx
//if(!(pContext->stack[pContext->stackTop -2+2].intValue [XXOperator] pContext->stack[pContext->stackTop-1+2].intValue))
0x8B, 0x45, 0x08, // mov eax,dword ptr [pContext]
0x8B, 0x48, 0x04, // mov ecx,dword ptr [eax+4]
0x8B, 0x55, 0x08, // mov edx,dword ptr [pContext]
0x8B, 0x02, // mov eax,dword ptr [edx]
0x8B, 0x55, 0x08, // mov edx,dword ptr [pContext]
0x8B, 0x52, 0x04, // mov edx,dword ptr [edx+4]
0x8B, 0x75, 0x08, // mov esi,dword ptr [pContext]
0x8B, 0x36, // mov esi,dword ptr [esi]
0x8B, 0x04, 0xC8, // mov eax,dword ptr [eax+ecx*8]
0x3B, 0x44, 0xD6, 0x08, // cmp eax,dword ptr [esi+edx*8+8]
0x0F, 0x00, 0x00, 0x00, 0x00, 0x00, // JXX <target>
};
memcpy(&code[ip], c, sizeof(c));
ip+=sizeof(c);
code[ip-5] = XX;
CreateJmpLink(&code[ip-5], targetpc, pJmpTargetMap);
}
void IfIcmple(u1* code, int& ip, int targetpc, CMapPtrToPtr *pJmpTargetMap)
{
IfICmpXX(code, ip, targetpc, pJmpTargetMap, JLE);
}
void IfIcmpne(u1* code, int& ip, int targetpc, CMapPtrToPtr *pJmpTargetMap)
{
IfICmpXX(code, ip, targetpc, pJmpTargetMap, JNE);
}
void IfIcmpge(u1* code, int& ip, int targetpc, CMapPtrToPtr *pJmpTargetMap)
{
IfICmpXX(code, ip, targetpc, pJmpTargetMap, JGE);
}
void IfIcmplt(u1* code, int& ip, int targetpc, CMapPtrToPtr *pJmpTargetMap)
{
IfICmpXX(code, ip, targetpc, pJmpTargetMap, JL);
}
void IfIcmpgt(u1* code, int& ip, int targetpc, CMapPtrToPtr *pJmpTargetMap)
{
IfICmpXX(code, ip, targetpc, pJmpTargetMap, JG);
}
void IfIcmpeq(u1* code, int& ip, int targetpc, CMapPtrToPtr *pJmpTargetMap)
{
IfICmpXX(code, ip, targetpc, pJmpTargetMap, JE);
}
</pre><br />
<br />
Thats it. We have all the branching instructions working correctly now.MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-767210891894826844.post-79714305652640087032010-07-30T01:19:00.001+06:002010-08-10T04:33:37.501+06:00Just In Time Compiler for Managed Platform- Part 4: Basic Conditional BranchToday I'll generate code that can handle conditional branch. This one is really gian step- since once we have conditional branching enabled we are ready to execute almost all instructions. This is relatively complex to handle than earlier things.<br />
<br />
First let us define the java class with a condition:<br />
<br />
<pre>public class Math
{
public static int add(int x, int y)
{
int r;
r= x+y;
return r;
}
public static int SimpleCall()
{
return 17;
}
public static int SimpleCall2()
{
return add(17, 29);
}
public static int SimpleCall4(int p)
{
if(p>100) return 100;
return add(p, 29);
}
public static int SimpleCall5(int p)
{
return SimpleCall4(101);
}
public static int SimpleCall3()
{
return SimpleCall4(17);
}
public static int Main()
{
return SimpleCall3();
}
}
</pre><br />
And the generated byte code:<br />
<br />
<pre>public class Math extends java.lang.Object{
public Math();
Signature: ()V
Code:
0: aload_0
1: invokespecial #1; //Method java/lang/Object."init":()V
4: return
public static int add(int, int);
Signature: (II)I
Code:
0: iload_0
1: iload_1
2: iadd
3: istore_2
4: iload_2
5: ireturn
public static int SimpleCall();
Signature: ()I
Code:
0: bipush 17
2: ireturn
public static int SimpleCall2();
Signature: ()I
Code:
0: bipush 17
2: bipush 29
4: invokestatic #2; //Method add:(II)I
7: ireturn
public static int SimpleCall4(int);
Signature: (I)I
Code:
0: iload_0
1: bipush 100
3: if_icmple 9
6: bipush 100
8: ireturn
9: iload_0
10: bipush 29
12: invokestatic #2; //Method add:(II)I
15: ireturn
public static int SimpleCall5(int);
Signature: (I)I
Code:
0: bipush 101
2: invokestatic #3; //Method SimpleCall4:(I)I
5: ireturn
public static int SimpleCall3();
Signature: ()I
Code:
0: bipush 17
2: invokestatic #3; //Method SimpleCall4:(I)I
5: ireturn
public static int Main();
Signature: ()I
Code:
0: invokestatic #4; //Method SimpleCall3:()I
3: ireturn
}
</pre><br />
The <code>SimpleCall4</code> method has a conditional branch instruction <code>if_icmple</code>. Note that we deal with static methiods so far- so dont care the <code>init</code> method yet.<br />
<br />
Let us first define the helper method for the instruction to generate machine code:<br />
<br />
<pre>void IfIcmple(u1* code, int& ip, int targetpc, CMapPtrToPtr *pJmpTargetMap)
{
u1 c[] = {
//if((pRE->stack[pRE->stackTop-2].intValue > pRE->stack[pRE->stackTop-1].intValue))
0x8B, 0x45, 0x08, // mov eax,dword ptr [pRE]
0x8B, 0x48, 0x04, // mov ecx,dword ptr [eax+4]
0x8B, 0x55, 0x08, // mov edx,dword ptr [pRE]
0x8B, 0x02, // mov eax,dword ptr [edx]
0x8B, 0x55, 0x08, // mov edx,dword ptr [pRE]
0x8B, 0x52, 0x04, // mov edx,dword ptr [edx+4]
0x8B, 0x75, 0x08, // mov esi,dword ptr [pRE]
0x8B, 0x36, // mov esi,dword ptr [esi]
0x8B, 0x44, 0xC8, 0xF0, // mov eax,dword ptr [eax+ecx*8-10h]
0x3B, 0x44, 0xD6, 0xF8, // cmp eax,dword ptr [esi+edx*8-8]
0x76, 0x00, 0x90, 0x90, 0x90, // JBE <targetbyte> nop nop nop
//pRE->stackTop -= 2;
0x8B, 0x45, 0x08, // mov eax,dword ptr [pRE]
0x8B, 0x48, 0x04, // mov ecx,dword ptr [eax+4]
0x83, 0xE9, 0x02, // sub ecx,2
0x8B, 0x55, 0x08, // mov edx,dword ptr [pRE]
0x89, 0x4A, 0x04, // mov dword ptr [edx+4],ecx
};
memcpy(&code[ip], c, sizeof(c));
ip+=sizeof(c);
LinkedListNode *pNode = new LinkedListNode(&code[ip-20], NULL);
LOG(_T("PC = 0x%X Jmp Inst Offset = 0x%X Inst = 0x%X\n"), targetpc, (int)(&code[ip-20])-(int)(code), code[ip-20]);
JmpTarget *pJmpTarget = NULL;
if(!pJmpTargetMap->Lookup((void *) targetpc, (void *&) pJmpTarget))
{
pJmpTarget = new JmpTarget();
pJmpTarget->pTargetList = pNode;
pJmpTarget->pc = targetpc;
pJmpTargetMap->SetAt((void *)targetpc, pJmpTarget);
}
else
{
pNode->pNext = pJmpTarget->pTargetList;
}
pJmpTarget->pTargetList = pNode;
}
</pre><br />
The instruction <code>if_icmple</code> checks value on top of stack and brances if first argument is greater than second argument. We do same in the machine code. Since we can not calculate address of target instruction without first generating code that is behind the target- we keep the target address empty and fix all jump address after we generate machine code all the instructions. To do this we maintain a map of jmp instruction locations and also a map of native vs managed code locations. <br />
<br />
Since one instruction can be target of multiple jmp instructions we keep a linked list of jmp instructions we need to fix for a specific managed target instruction-<br />
<br />
<pre>struct LinkedListNode
{
void *pData;
LinkedListNode *pNext;
LinkedListNode(void *pData, LinkedListNode *pNext)
{
this->pData = pData;
this->pNext = pNext;
}
LinkedListNode()
{
this->pData = NULL;
this->pNext = NULL;
}
};
</pre><br />
Here we need to fix out <code>ireturn</code> helper- since after return from method we must not execute any instruction we must return but before that we must fix the native stack (epilog) and then return. Since we add epilog at the end of each native function we just jmp to that location for the cleanup if there is multiple retutn from managed code. To track thios we insert an unconditional jmp instruction to the epilog-<br />
<br />
<pre>// ireturn instruction takes the value from stack top and push
// to stack[0] position.
void IReturn(u1* code, int& ip, CMapPtrToPtr *pJmpTargetMap)
{
u1 c[] = {
//pRE->stack[0].intValue=pRE->stack[pRE->stackTop-1].intValue;
0x8B, 0x45, 0x08, // mov eax,dword ptr [pRE]
0x8B, 0x48, 0x04, // mov ecx,dword ptr [eax+4]
0x8B, 0x55, 0x08, // mov edx,dword ptr [pRE]
0x8B, 0x02, // mov eax,dword ptr [edx]
0x8B, 0x55, 0x08, // mov edx,dword ptr [pRE]
0x8B, 0x12, // mov edx,dword ptr [edx]
0x8B, 0x44, 0xC8, 0xF8, // mov eax,dword ptr [eax+ecx*8-8]
0x89, 0x02, // mov dword ptr [edx],eax
};
memcpy(&code[ip], c, sizeof(c));
ip+=sizeof(c);
u1 c1[] = {
0xE9, 0x00, 0x00, 0x00, 0x00 //JMP <target>, nop ,nop
};
memcpy(&code[ip], c1, sizeof(c1));
ip+=sizeof(c1);
LinkedListNode *pNode = new LinkedListNode(&code[ip-5], NULL);
LOG(_T("PC = RETURN Jmp Inst Offset = 0x%X Inst = 0x%X\n"), (int)(&code[ip-5])-(int)(code), code[ip-5]);
JmpTarget *pJmpTarget = NULL;
if(!pJmpTargetMap->Lookup((void *) 0, (void *&) pJmpTarget))
{
pJmpTarget = new JmpTarget();
pJmpTarget->pTargetList = pNode;
pJmpTarget->pc = 0;
pJmpTargetMap->SetAt((void *)0, pJmpTarget);
}
else
{
pNode->pNext = pJmpTarget->pTargetList;
}
pJmpTarget->pTargetList = pNode;
}
</pre><br />
Lets now see how we fix the address- <br />
<br />
<pre>void FixJmpLocations(u1 *codes, CMapPtrToPtr *pJmpTargetMap, CMapPtrToPtr *pManagedtoNativeMap, u1* retAddress)
{
LOG(_T("Fixing Jmp Target\n"));
//Iterate through the entire map,
for (POSITION pos = pJmpTargetMap->GetStartPosition(); pos != NULL;)
{
JmpTarget *pJmpTarget;
int pc;
pJmpTargetMap->GetNextAssoc(pos, (void *&)pc, (void *&)pJmpTarget);
ASSERT(pc == pJmpTarget->pc);
int target;
if(0 == pJmpTarget->pc)
{
target = (int)retAddress;
}
else
{
target = (int) pManagedtoNativeMap->GetValueAt((void *&)pJmpTarget->pc);
}
LinkedListNode *pTargetList = pJmpTarget->pTargetList;
do{
int offset=0;
if(0xE9 == codes[(int)((int)pTargetList->pData-(int)codes)])
{
offset = target - (int)pTargetList->pData-5; //1 for inst 4 for 4 byte offset = -5
memcpy(&codes[(int)((int)pTargetList->pData-(int)codes)+1], &offset, sizeof(offset));
}
else
{
offset = target - (int)pTargetList->pData - 2; //1 for inst 1 for 1 byte offset = -2
codes[(int)((int)pTargetList->pData-(int)codes)+1]=offset;
}
LOG(_T("Fixed 0x%X with Native Address Offset 0x%X\n"), (int)pTargetList->pData - (int)codes, offset);
pTargetList = pTargetList->pNext;
}while(NULL != pTargetList);
}
}
</pre><br />
Please note that the map <code>pManagedtoNativeMap</code> is polulated the <code>Compile</code> function like this in the giant for loop for each instruction-<br />
<br />
<pre>pManagedtoNativeMap->SetAt((void *)pc, &codes[ip]);
</pre><br />
Thats it. We are now ready to test the vodes we generate- <br />
<br />
<pre>int main()
{
Context *pRE = new Context();;
pRE->stack = new Variable[STACK_SIZE];
pRE->stackTop = 0;
memset(pRE->stack, 0, sizeof(Variable)*STACK_SIZE);
pRE->pVMEnv = new VMEnvironment();
pRE->pVMEnv->pClassHeap = new ClassHeap();
pRE->pVMEnv->pObjectHeap = new ObjectHeap();
pRE->pVMEnv->ppHelperMethods = HelperMethods;
ClassHeap* pClsHeap = pRE->pVMEnv->pClassHeap;
JavaClass jc;
pClsHeap->LoadClass("Math", &jc);
JavaClass *pVirtualClass =&jc, *pClass1 = &jc;
int mindex=pClass1->GetMethodIndex(_T("Main"),_T("()I"),pVirtualClass);
method_info_ex *pMethod = &pVirtualClass->methods[mindex];
MethodLink *pMethodLink = new MethodLink();
pMethodLink->pClass = pVirtualClass;
pMethodLink->pMethod = pMethod;
((void (*)(MethodLink *pMethodLink, Context *pRE))pRE->pVMEnv->ppHelperMethods[CALL_METHOD_HELPER_INDEX])(pMethodLink, pRE);
LOG(_T("Ret = %d"), pRE->stack[0].intValue);
return 0;
}
</pre><br />
Thats it. Our generated native code can handle branching!MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-767210891894826844.post-34244930101841271822010-07-27T06:14:00.001+06:002010-08-10T04:33:27.336+06:00Just In Time Compiler for Managed Platform- Part 3: Call a methodToday I'll try to extend the simple JIT compiler to the point where we can call a method from another method.<br />
<br />
First lets create a simple java class:<br />
<br />
<pre>public class Math
{
public static int add(int x, int y)
{
int r;
r= x+y;
return r;
}
public static int SimpleCall2()
{
return add(17, 29);
}
}
</pre><br />
Here we call <code>add</code> method from <code>SimpleCall2</code> method. Our JIT compiler will supply mechanism to handle this. When we compile using java compiler we get following class and method byte code:<br />
<br />
<pre>public class Math extends java.lang.Object{
public Math();
Signature: ()V
Code:
0: aload_0
1: invokespecial #1;
4: return
public static int add(int, int);
Signature: (II)I
Code:
0: iload_0
1: iload_1
2: iadd
3: istore_2
4: iload_2
5: ireturn
public static int SimpleCall2();
Signature: ()I
Code:
0: bipush 17
2: bipush 29
4: invokestatic #2; //Method add:(II)I
7: ireturn
}
</pre><br />
[Note: I do not describe the Java Virtual Machine basics here again- You may look at my article for basic understanding: <a href="http://www.codeproject.com/KB/cpp/jvm.aspx">Home Made Java Virtual Machine</a>]<br />
<br />
First we need to extend our <code>Context</code> structure to hold some more values. We should add members at the bottom- otherwise the code we generated so far will be invalid.<br />
<br />
<pre>struct VMEnvironment
{
ObjectHeap *pObjectHeap;
ClassHeap *pClassHeap;
void **ppHelperMethods;
};
struct Context
{
Variable *stack;
int stackTop;
JavaClass *pClass;
Context *pCallerContext;
VMEnvironment *pVMEnv;
};
</pre><br />
Also we want to keep track of native codes we generate. So we need another structure.<br />
<br />
<pre>struct MethodLink
{
JavaClass *pClass;
method_info_ex *pMethod;
void *pNativeBlock;
};
</pre><br />
Now we need a helper class to keep track of the methods we work with. We use a simple string to pointer map. The key string is generated from classname, method name and method desc. So key looks like "Math::add(II)I".<br />
<br />
<pre>MethodLink* GetMethod(JavaClass *pClass, method_info_ex *pMethod, u4 pc)
{
static CMapStringToPtr methodsMap;
u2 mi=getu2(&pMethod->pCode_attr->code[pc+1]);
char *pConstPool = (char *)pClass->constant_pool[mi];
u2 classIndex = getu2(&pConstPool[1]);
u2 nameAndTypeIndex = getu2(&pConstPool[3]);
//get class at pool index
pConstPool = (char *)pClass->constant_pool[classIndex];
ASSERT(pConstPool[0] == CONSTANT_Class);
u2 ni=getu2(&pConstPool[1]);
CString strClassName;
pClass->GetStringFromConstPool(ni, strClassName);
ClassHeap *pClassHeap = new ClassHeap();
JavaClass *pClassCallee=pClassHeap->GetClass(strClassName);
pConstPool = (char *)pClassCallee->constant_pool[nameAndTypeIndex];
ASSERT(pConstPool[0] == CONSTANT_NameAndType);
u2 name_index = getu2(&pConstPool[1]);
u2 descriptor_index = getu2(&pConstPool[3]);
CString strMethodName, strMethodDesc;
pClassCallee->GetStringFromConstPool(name_index, strMethodName);
pClassCallee->GetStringFromConstPool(descriptor_index, strMethodDesc);
JavaClass *pVirtualClass=pClassCallee;
int nIndex=pClassCallee->GetMethodIndex(strMethodName, strMethodDesc, pVirtualClass);
method_info_ex *pCalleeMethod = &pClassCallee->methods[nIndex];
/*
if( ACC_SUPER & pCalleeMethod->access_flags)
{
pCalleeMethod = pClassCallee->GetSuperClass();
}
*/
CString sign(strClassName+"::"+strMethodName+strMethodDesc);
MethodLink *pLink=NULL;
if(!methodsMap.Lookup(sign, (void *&)pLink))
{
pLink = new MethodLink();
pLink->pClass = pClassCallee;
pLink->pMethod = pCalleeMethod;
pLink->pNativeBlock = NULL;
methodsMap.SetAt(sign, pLink);
}
return pLink;
}
</pre><br />
To call a method we do not generate the statck preparation code using machine code for now to keep the things simple. We'll do that after we finish all type of code generation. So from native code we call back to a C++ method that again calls into generated codes-<br />
<br />
<pre>void CallMethod(MethodLink *pMethodLink, Context *pRE)
{
LOG(_T("CallMethod\n"));
int codeBlockSize = pMethodLink->pMethod->pCode_attr->code_length*2; //todo guess better
int (*NativeBlock)(Context *)=(int (*)(Context *)) VirtualAlloc(NULL, codeBlockSize, MEM_COMMIT, PAGE_EXECUTE_READWRITE);
u1* codes = (u1*) NativeBlock;
int ip =0;
JavaClass *pClass = pMethodLink->pClass;
if(NULL == pMethodLink->pNativeBlock)
{
Compile(pMethodLink->pClass, pMethodLink->pMethod, codes, ip);
pMethodLink->pNativeBlock = codes;
}
CString strName, strDesc;
pMethodLink->pClass->GetStringFromConstPool(pMethodLink->pMethod->name_index, strName);
pMethodLink->pClass->GetStringFromConstPool(pMethodLink->pMethod->descriptor_index, strDesc);
int params=GetMethodParametersStackCount(strDesc)+1;
//invokestatic: we are only dealing with static methods so far
int nDiscardStack =params;
if(pMethodLink->pMethod->access_flags & ACC_NATIVE)
{
}
else
{
nDiscardStack+= pMethodLink->pMethod->pCode_attr->max_locals;
}
pRE->stackTop+=(nDiscardStack-1);
LOG(_T("Invoking method %s%s, \n"), strName, strDesc);
(*NativeBlock)(pRE);
//if returns then get on stack
if(strDesc.Find(_T(")V")) < 0)
{
nDiscardStack--;
if(strDesc.Find(_T(")J")) < 0)
{
}
else
{
nDiscardStack--;
}
}
pRE->stackTop-=nDiscardStack;
LOG(_T("~CallMethod\n"));
}
</pre><br />
OK, thats the callbacks we need for now. Now we generate the actual machine code that will use the MethodLink* value to call back to the <code>CallMethod</code> function. To do this we use a function pointer list and store it in the context environment-<br />
<br />
<pre>#define CALL_METHOD_HELPER_INDEX 0
void* HelperMethods[] = {
CallMethod,
};
</pre><br />
Let us now define the <code>InvokeStatic</code> helper method.<br />
<br />
<pre>void InvokeStatic(JavaClass *pClass, method_info_ex *pMethod, u4 pc, u1* codes, int &ip)
{
MethodLink* pLink = GetMethod(pClass, pMethod, pc);
EmitCallMethod(codes, ip, pLink);
}
void EmitCallMethod(u1* code, int &ip, void* pLinkAddress)
{
//((void (*)(MethodLink *pMethodLink))pRE->pVMEnv->ppHelperMethods[CALL_METHOD_HELPER_INDEX])(pLinkAddress, pRE);
u1 c[] = {
0x8B, 0x45, 0x08, // mov eax,dword ptr [pRE]
0x50, // push eax
0x68, 0x00, 0x00, 0x00, 0x00, // push pLinkAddress
0x8B, 0x4D, 0x08, // mov ecx,dword ptr [pRE]
0x8B, 0x51, 0x10, // mov edx,dword ptr [ecx+10h]
0x8B, 0x42, 0x08, // mov eax,dword ptr [edx+8]
0x8B, 0x08, // mov ecx,dword ptr [eax]
0xFF, 0xD1, // call ecx
0x83, 0xC4, 0x08, // add esp,8
};
memcpy(c+5, &pLinkAddress, 4);
memcpy(&code[ip], c, sizeof(c));
ip+=sizeof(c);
}
</pre><br />
To compile the methods we define a function that generates machine code for java byte codes. This function does not handle branch instructions right now. To handle branch we probably need two pass- since we would not know the exact address during first pass. So, here is a large while loop to do basic things:<br />
<br />
<pre>u4 Compile(JavaClass *pClass, method_info_ex *pMethod, u1 *codes, int &ip)
{
if(pMethod->access_flags & ACC_NATIVE)
{
return 1;
}
Prolog(codes, ip);
u4 pc=0;
u1 *bc=pMethod->pCode_attr->code;
i4 error=0;
CString strMethod;
pClass->GetStringFromConstPool(pMethod->name_index, strMethod);
i4 index=0;
while(pMethod->pCode_attr->code_length>pc)
{
LOG(_T("Opcode = %s\n"),OpcodeDesc[(u1)bc[pc]]);
switch(bc[pc])
{
case nop:
pc++;
break;
case bipush:// 16 /*(0x10)*/
BiPush(codes, ip, (u1)bc[pc+1]);
pc+=2;
break;
case iload_0: //26 Load int from local variable 0
ILoad_0(codes, ip);
pc++;
break;
case iload_1: //27 Load int from local variable 1
ILoad_1(codes, ip);
pc++;
break;
case iload_2: //28 Load int from local variable 2
ILoad_2(codes, ip);
pc++;
break;
case iload_3: //29 Load int from local variable 3
ILoad_3(codes, ip);
pc++;
break;
case istore_2: // 61 /*(0x3d) */
IStore_2(codes, ip);
pc++;
break;
case iadd: //96
IAdd(codes, ip);
pc++;
break;
case invokestatic:// 184
InvokeStatic(pClass, pMethod, pc, codes, ip);
pc+=3;
break;
case ireturn: //172 (0xac)
IReturn(codes, ip);
pc++;
break;
default:
error=1;
break;
}
if(error) break;
}
Return0(codes, ip);
Epilog(codes, ip);
return error;
}
</pre><br />
OK, we are now ready to test out code:<br />
<br />
<pre>int main()
{
Context *pRE = new Context();;
pRE->stack = new Variable[STACK_SIZE];
pRE->stackTop = 0;
memset(pRE->stack, 0, sizeof(Variable)*STACK_SIZE);
pRE->pVMEnv = new VMEnvironment();
pRE->pVMEnv->pClassHeap = new ClassHeap();
pRE->pVMEnv->pObjectHeap = new ObjectHeap();
pRE->pVMEnv->ppHelperMethods = HelperMethods;
ClassHeap* pClsHeap = pRE->pVMEnv->pClassHeap;
JavaClass jc;
pClsHeap->LoadClass("Math", &jc);
JavaClass *pVirtualClass =&jc, *pClass1 = &jc;
int mindex=pClass1->GetMethodIndex(_T("SimpleCall2"),_T("()I"),pVirtualClass);
method_info_ex *pMethod = &pVirtualClass->methods[mindex];
MethodLink *pMethodLink = new MethodLink();
pMethodLink->pClass = pVirtualClass;
pMethodLink->pMethod = pMethod;
((void (*)(MethodLink *pMethodLink, Context *pRE))pRE->pVMEnv->ppHelperMethods[CALL_METHOD_HELPER_INDEX])(pMethodLink, pRE);
LOG(_T("Return Value = %d"), pRE->stack[0].intValue);
return 0;
}
</pre><br />
Do you see value 46 on the stack as return value? Cool!MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com1tag:blogger.com,1999:blog-767210891894826844.post-56010163956549037462010-07-15T21:50:00.001+06:002010-08-10T04:33:15.567+06:00Just In Time Compiler for Managed Platform- Part 2: Generate native methodToday we'll design a small block of code that is equivallent to a corresponding java method.<br />
<br />
Since the generated native executable code will be used only by our VM we are free to define out own structure and calling convention for it. We generate one native function for each Java method. Each generated native function will have only parameter that is required for operation- a pointer to a structure <code>RuntimeEnvironment</code>:<br />
<br />
<pre>union Variable
{
u1 charValue;
u2 shortValue;
u4 intValue;
f4 floatValue;
u4* ptrValue;
Object object;
};
struct RuntimeEnvironment
{
Variable *stack;
int stackTop;
//We'll add more as we require later
};
</pre><br />
The return type will be int:<br />
<br />
<pre>int ExecuteMethod(RuntimeEnvironment *pRE);
</pre><br />
To generate the final method we need a lot of helper function. We define those as: <br />
<pre>void HelperFunction(u1* code, int& ip);
</pre>These functions will take a code block and insert code in the block and fix the code pointer (ip).<br />
<br />
First let us define the Prolog and Epilog and return 0 helper functions for this function prototype:<br />
<br />
<pre>void Prolog(u1* code, int& ip)
{
u1 c[]= {
0x55,// push ebp
0x8B, 0xEC,// mov ebp,esp
0x81, 0xEC, 0xC0, 0x00, 0x00, 0x00,// sub esp,0C0h
0x53,// push ebx
0x56,// push esi
0x57, // push edi
};
memcpy(&code[ip], c, sizeof(c));
ip+=sizeof(c);
}
void Epilog(u1* code, int& ip)
{
u1 c[]= {
0x5F,// pop edi
0x5E,// pop esi
0x5B,// pop ebx
0x8B, 0xE5,// mov esp,ebp
0x5D,// pop ebp
0xC3,// ret
};
memcpy(&code[ip], c, sizeof(c));
ip+=sizeof(c);
}
void Return0(u1* code, int& ip)
{
//33 C0 xor eax,eax
code[ip++]=0x33;
code[ip++]=0xC0;
}
</pre><br />
Now, we want to generate machine code for the following simple function-<br />
<br />
<pre>public static int SimpleCall()
{
return 17;
}
</pre><br />
Here is the generated java byte code:<br />
<pre>Signature: ()I
Code:
0: bipush 17
2: ireturn
</pre><br />
Here we start to generate helper function for Java Virtual Machine instruction. <br />
<br />
For bipush [value] we need to push the [value] on the VM stack:<br />
<br />
<pre>void BiPush(u1* code, int& ip, short value)
{
// C++ equivallent
// pRE->stack[pRE->stackTop++].shortValue = value;
u1 c[] = {
0x8B, 0x45, 0x08, // mov eax,dword ptr [pRE]
0x8B, 0x48, 0x04, // mov ecx,dword ptr [eax+4]
0x8B, 0x55, 0x08, // mov edx,dword ptr [pRE]
0x8B, 0x02, // mov eax,dword ptr [edx]
0xBA, 0x00, 0x00, 0x00, 0x00, // mov edx,value
0x66, 0x89, 0x14, 0xC8, // mov word ptr [eax+ecx*8],dx
0x8B, 0x45, 0x08, // mov eax,dword ptr [pRE]
0x8B, 0x48, 0x04, // mov ecx,dword ptr [eax+4]
0x83, 0xC1, 0x01, // add ecx,1
0x8B, 0x55, 0x08, // mov edx,dword ptr [pRE]
0x89, 0x4A, 0x04, // mov dword ptr [edx+4],ecx
};
//We need to encode value and set it to the 00 00 00 00 position
u1 encVal[4];
EncodeByte4((int)value, encVal);
memcpy(c + 12, encVal, 4);
memcpy(&code[ip], c, sizeof(c));
ip+=sizeof(c);
}
</pre><br />
Thats it for the simple java function. We can now test this:<br />
<br />
<pre>int main()
{
int codeBlockSize = 4096;
int (*SimpleCall)(RuntimeEnvironment *)=(int (*)(RuntimeEnvironment *)) VirtualAlloc(NULL, codeBlockSize, MEM_COMMIT, PAGE_EXECUTE_READWRITE);
u1* codes = (u1*) SimpleCall;
int ip=0;
memset(codes, 0, codeBlockSize);
Prolog(codes, ip);
BiPush(codes, ip, 17);
Return0(codes, ip);
Epilog(codes, ip);
//No lets test if it is really pushing value 17 on the VM stack
RuntimeEnvironment *pRE = new RuntimeEnvironment();;
pRE->stack = new Variable[20];
memset(pRE->stack, 0, sizeof(Variable)*20);
int retVal = (*SimpleCall)(pRE);
printf("pRE->stack[0].intValue = %d", pRE->stack[0].intValue);
return 0;
}
</pre><br />
Thats cool- we have generated our first native function that actually does some byte code execution.MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-767210891894826844.post-26679685957389908782010-07-10T23:08:00.003+06:002010-08-10T04:33:04.807+06:00Just In Time Compiler for Managed Platform- Part 1: Code GenerationFirst we need to generate executable code block.<br />
<br />
So let us write some code in C++.<br />
<pre>int add(int x, int y)
{
int r;
r= x+y;
return r;
}
int main()
{
int r1 = add(13, 23);
printf("Returned value = %d", r1);
}
</pre><br />
Great! It returns the right value. OK, but that is very basic. We want to generate the function from data in a simple buffer-<br />
<br />
First we need a machine equivallent code for the function above:<br />
<br />
<pre>unsigned char addcode[] = {
0x55, //push ebp
0x8B, 0xEC, //mov ebp,esp
0x81, 0xEC, 0xC0, 0x00, 0x00, 0x00, //sub esp,0C0h
0x53, //push ebx
0x56, //push esi
0x57, //push edi
//r=x+y;
0x8B, 0x45, 0x08, //mov eax,dword ptr [x]
0x03, 0x45, 0x0C, //add eax,dword ptr [y]
0x89, 0x45, 0xF8, //mov dword ptr [r],eax
//return r;
0x8B, 0x45, 0xF8, //mov eax,dword ptr [r]
0x5F, //pop edi
0x5E, //pop esi
0x5B, //pop ebx
0x8B, 0xE5, //mov esp,ebp
0x5D, //pop ebp
0xC3 //ret
};
</pre><br />
OK, this code is generated by Visual Studio compiler. We use it to generate our own code block in memory.<br />
<br />
To get a memory block we can use to generate executable code we use the following Windows API:<br />
<br />
<pre>LPVOID WINAPI VirtualAlloc(
__in_opt LPVOID lpAddress,
__in SIZE_T dwSize,
__in DWORD flAllocationType,
__in DWORD flProtect
);
</pre><br />
First we allocate a 4096 byte executable code block and put it in a function pointer:<br />
<br />
<pre>int (*addfn)(int, int) = (int (*)(int, int)) VirtualAlloc(NULL, 4096, MEM_COMMIT, PAGE_EXECUTE_READWRITE);</pre><br />
Then we copy our executable code to this memory block:<br />
<br />
<pre>memcpy(addfn, addcode, sizeof(addcode));
</pre><br />
Now the majic - we call the function and print the return value:<br />
<br />
<pre>int r1 = (*addfn)(13,23);
printf("Returned value = %d", r1);
</pre><br />
Thats easy- right?<br />
<br />
Now we release the memory since we are gentle citizen-<br />
<br />
<pre>VirtualFree(addfn, NULL, MEM_RELEASE);</pre><br />
Thats it. here is the full code:<br />
<pre>#include [windows.h][stdio.h]...
unsigned char addcode[] = {
0x55, //push ebp
0x8B, 0xEC, //mov ebp,esp
0x81, 0xEC, 0xC0, 0x00, 0x00, 0x00, //sub esp,0C0h
0x53, //push ebx
0x56, //push esi
0x57, //push edi
//r=x+y;
0x8B, 0x45, 0x08, //mov eax,dword ptr [x]
0x03, 0x45, 0x0C, //add eax,dword ptr [y]
0x89, 0x45, 0xF8, //mov dword ptr [r],eax
//return r;
0x8B, 0x45, 0xF8, //mov eax,dword ptr [r]
0x5F, //pop edi
0x5E, //pop esi
0x5B, //pop ebx
0x8B, 0xE5, //mov esp,ebp
0x5D, //pop ebp
0xC3 //ret
};
int main()
{
int (*addfn)(int, int) = (int (*)(int, int)) VirtualAlloc(NULL, 4096, MEM_COMMIT, PAGE_EXECUTE_READWRITE);
memcpy(addfn, addcode, sizeof(addcode));
int r1 = (*addfn)(13,23);
printf("Returned value = %d", r1);
VirtualFree(addfn, NULL, MEM_RELEASE);
return 0;
}
</pre><br />
Thats all for now. We can now generate code in memory and execute it. First step to design a JIT compiler.MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-767210891894826844.post-91886541861531965152008-04-30T21:08:00.001+06:002008-08-05T20:35:08.757+06:00Compiler in action- C/C++ to Machine<h2>Introduction</h2>What happens when I give my C/C++ code to a compiler? It generates machine code. <br />But I want to know what machine code it generates really. I use the compiler that comes Visual C++ 2008. <br />Other versions should be similar if not same.<br /><br /><h2>Producing Assembly Output</h2><br /><p>With Visual Studio we can produce assembly language output with following settings: </p><br />Project Property Pages > Configuration Properties > C++ > Output Files<br/><br />Assembler Output: <span style="font-weight: bold;">Assembly With Source Code (/FAs)</span> <br /><p>The compiler generates assembly code and output with corresponding C/C++ source code. Its very <br />useful to understand how the compiler works. <br /></p><br /><h2>Function</h2><br /><p>A function when compiled has its prolog, epilog and ret instructions along with its body. <br />It maintains the stack and local variables.<br /></p><br /><h3>Prolog and Epilog</h3><br /><p><br />Prolog is a set of instructions that compiler generates at the beginning of a function and epilog is <br />generated at the end of a function. This two maintains stack, local variables, registers and unwind <br />information.<br /></p><br /><p>Every function that allocates stack space, calls other functions, saves nonvolatile registers, or <br />uses exception handling must have a prolog whose address limits are described in the unwind data <br />associated with the respective function table entry. The prolog saves argument registers in their home <br />addresses if required, pushes nonvolatile registers on the stack, allocates the fixed part of the stack <br />for locals and temporaries, and optionally establishes a frame pointer. The associated unwind data must <br />describe the action of the prolog and must provide the information necessary to undo the effect of the <br />prolog code [MSDN].<br /></p><br /><p><br />Let us see what is generated as prolog and epilog. We have a function named <code>add</code> like this:<br /><pre>int add(int x, int y)<br />{<br /> int p=x+y;<br /> return p;<br />}<br /></pre><br />And the generated assembly listing:<br /><br /><pre>_p$ = -4 ; size = 4<br />_x$ = 8 ; size = 4<br />_y$ = 12 ; size = 4<br />?add@@YAHHH@Z PROC ; add, COMDAT<br />; 12 : {<br />;Prolog<br />push ebp<br />mov ebp, esp<br />push ecx<br /><br />; 13 : int p=x+y;<br />mov eax, DWORD PTR _x$[ebp]<br />add eax, DWORD PTR _y$[ebp]<br />mov DWORD PTR _p$[ebp], eax<br />; 14 : return p;<br />mov eax, DWORD PTR _p$[ebp]<br />; 15 : }<br /><br />;Epilog<br />mov esp, ebp<br />pop ebp<br /><br />ret 0 ;disposition of stack- 0 disp as returning through register<br />?add@@YAHHH@Z ENDP ; add<br /><br /></pre><br /><br />Not much work. The compiler just saves EBP register copies the ESP register in EBP register and use <br />EBP as stack pointer at prolog and at epilog stage it restores the EBP register. Sometimes there is a <br />subtraction to handle local variables. There are two instruction ENTER and LEAVE that can be used <br />in place of push pop things.<br /></p><br /><br /><h3>Function Parameters/Local Variables</h3><br /><p><br />The function parameters are placed at positive offset from the stack pointer and local variables are <br />located at negative offset at the time of calling the function.<br />Function parameters are pushed on the stack before calling and the function may initialize the <br />local variable. From previous assembly listing we find parameters x and y is at offset 8 and 12 <br />and the local variable p is at offset -4 from the stack top.<br /></p><br /><br /><h2>Function call</h2><br /><p><br />The CALL instruction is used to invoke a function. Before doing so the caller function pushes <br />parameter values or set register (this pointer) and issue CALL instruction. After returning the <br />caller function may need to set stack pointer depending on calling convention it used. We discus <br />this in next subsection.<br /></p><br /><br /><h3>Calling conventions</h3><br /><p>There are several calling conventions. Calling convention tells compiler how the parameters <br />are passed, how stack is maintained and how to decorate the function names in object files. <br />Following table shows basic things at a glance:</p><br /><table border="1"><tr><td valign="top"><b>Calling Convention</b></td><td valign="top"><b>Argument Passing </b></td><td valign="top"><b>Stack Maintenance</b> </td><td valign="top"><b>Name Decoration (C only) </b></td><td valign="top"><b>Notes</b> </td></tr><tr><td valign="top">__cdecl </td><td valign="top">Right to left. </td><td valign="top">Calling function pops arguments from the stack. </td><td valign="top">Underscore prefixed to function names. Ex: _Foo. </td><td valign="top"><br /></td></tr><tr><td valign="top">__stdcall </td><td valign="top">Right to left. </td><td valign="top">Called function pops its own arguments from the stack. </td><td valign="top">Underscore prefixed to function name, @ appended followed by the number of decimal bytes in the argument list. Ex: _Foo@10. </td><td valign="top"><br /></td></tr><tr><td valign="top">__fastcall </td><td valign="top">First two DWORD arguments are passed in ECX and EDX, the rest are passed right to left. </td> <td valign="top">Called function pops its own arguments from the stack. </td><td valign="top">A @ is prefixed to the name, @ appended followed by the number of decimal bytes in the argument list. Ex: @Foo@10. </td> <td valign="top">Only applies to Intel CPUs. This is the default calling convention for Borland compilers. </td> </tr> <tr><td valign="top">thiscall </td><td valign="top">this pointer put in ECX, arguments passed right to left. </td> <td valign="top">Calling function pops arguments from the stack. </td> <td valign="top">None. </td> <td valign="top">Used automatically by C++ code. </td> </tr> <tr> <td valign="top">naked </td> <td valign="top">Right to left. </td> <td valign="top">Calling function pops arguments from the stack. </td> <td valign="top">None. </td> <td valign="top">Only used by VxDs. </td> </tr></table><br /><br />Source: Debugging Applications by John Robbins<br /></p><br /><h4>StdCall</h4><br /><pre><br />int _stdcall StdCallFunction(int x, int y)<br />{<br /> return x;<br />}<br /></pre><br />The generated code is like this:<br /><pre><br />_x$ = 8 ; size = 4<br />_y$ = 12 ; size = 4<br />?StdCallFunction@@YGHHH@Z PROC ; StdCallFunction, COMDAT<br />; 7 : {<br />push ebp<br />mov ebp, esp<br />; 8 : return x;<br />mov eax, DWORD PTR _x$[ebp]<br />; 9 : }<br />pop ebp<br />ret 8<br />?StdCallFunction@@YGHHH@Z ENDP ; StdCallFunction<br /><br /></pre><br />To call the compiler generates code like this:<br /><br /><pre><br />; 26 : r=StdCallFunction(p, q);<br /><br />mov eax, DWORD PTR _q$[ebp]<br />push eax<br />mov ecx, DWORD PTR _p$[ebp]<br />push ecx<br />call ?StdCallFunction@@YGHHH@Z ; StdCallFunction<br />mov DWORD PTR _r$[ebp], eax<br /></pre><br /><br /><h4>Cdecl</h4><br />The function declaration uses <code>_cdecl</code> keyword.<br /><pre><br />int _cdecl CDeclCallFunction(int x, int y)<br />{<br /> return x;<br />}<br /></pre><br /><br />Compiler generates following assembly listing:<br /><br /><pre><br />_x$ = 8 ; size = 4<br />_y$ = 12 ; size = 4<br />?CDeclCallFunction@@YAHHH@Z PROC ; CDeclCallFunction, COMDAT<br /><br />; 12 : {<br /><br />push ebp<br />mov ebp, esp<br /><br />; 13 : return x;<br /><br />mov eax, DWORD PTR _x$[ebp]<br /><br />; 14 : }<br /><br />pop ebp<br />ret 0<br />?CDeclCallFunction@@YAHHH@Z ENDP ; CDeclCallFunction<br /></pre><br /><br />To call the function compiler generates following code:<br /><pre><br />; 27 : r=CDeclCallFunction(p, q);<br /><br />mov edx, DWORD PTR _q$[ebp]<br />push edx<br />mov eax, DWORD PTR _p$[ebp]<br />push eax<br />call ?CDeclCallFunction@@YAHHH@Z ; CDeclCallFunction<br />add esp, 8<br />mov DWORD PTR _r$[ebp], eax<br /></pre><br /><h4>Fastcall</h4><br /><pre><br />int _fastcall FastCallFunction(int x, int y)<br />{<br />return x;<br />}<br /></pre><br />The generated code:<br /><pre><br />_y$ = -8 ; size = 4<br />_x$ = -4 ; size = 4<br />?FastCallFunction@@YIHHH@Z PROC ; FastCallFunction, COMDAT<br />; _x$ = ecx<br />; _y$ = edx<br />; 17 : {<br />push ebp<br />mov ebp, esp<br />sub esp, 8<br />mov DWORD PTR _y$[ebp], edx<br />mov DWORD PTR _x$[ebp], ecx<br />; 18 : return x;<br />mov eax, DWORD PTR _x$[ebp]<br />; 19 : }<br />mov esp, ebp<br />pop ebp<br />ret 0<br />?FastCallFunction@@YIHHH@Z ENDP ; FastCallFunction<br /></pre><br /><br />And to call the function:<br /><br /><pre><br />; 28 : r=FastCallFunction(p, q);<br /><br />mov edx, DWORD PTR _q$[ebp]<br />mov ecx, DWORD PTR _p$[ebp]<br />call ?FastCallFunction@@YIHHH@Z ; FastCallFunction<br />mov DWORD PTR _r$[ebp], eax<br /></pre><br /><h4>Thiscall</h4><br /><p><br />Used for class member functions. We discuss it later in detail.<br /></p><br /><h4>Nacked</h4><br /><p>This calling conven is used for VxD drivers. </p><br /><h2>Representation of a class</h2><br /><p>A class is just a structure of varuables with functions. While creating an object compiler reserves <br />space on heap and call the constructure of the class. A class can have a table of functions (the vtable) <br />as the first member. It is used to call virtual functions. Class member functions are treated similar <br />as normal C functions with the exception that it receives this pointer as one parameter in the <br />ECX register. </p><br /><br /><h2>Class Member Functions</h2><br />Here is a simple class for demonstration of member function. <br /><pre><br />class Number<br />{<br /> int m_nMember;<br /> public:<br /> void SetNumber(int num, int base)<br /> {<br /> m_nMember = num;<br /> }<br />};<br /></pre><br /><br />The <code>SetNumber</code> in class Number generates following listing:<br /><br /><pre><br />_this$ = -4 ; size = 4<br />_num$ = 8 ; size = 4<br />_base$ = 12 ; size = 4<br />?SetNumber@Number@@QAEXHH@Z PROC ; Number::SetNumber, COMDAT<br />; _this$ = ecx<br /><br />; 30 : {<br /><br />push ebp<br />mov ebp, esp<br />push ecx<br />mov DWORD PTR _this$[ebp], ecx<br /><br />; 31 : m_nMember = num;<br /><br />mov eax, DWORD PTR _this$[ebp]<br />mov ecx, DWORD PTR _num$[ebp]<br />mov DWORD PTR [eax], ecx<br /><br />; 32 : }<br /><br />mov esp, ebp<br />pop ebp<br />ret 8<br />?SetNumber@Number@@QAEXHH@Z ENDP ; Number::SetNumber<br /></pre><br /><br />Call member function <code>SetNumber</code> of the class. The thiscall convension is used- this <br />parameter is passed in ECX register:<br /><pre><br />; 42 : Number nObject;<br />; 43 : nObject.SetNumber(r, p);<br /><br />mov ecx, DWORD PTR _p$[ebp]<br />push ecx<br />mov edx, DWORD PTR _r$[ebp]<br />push edx<br />lea ecx, DWORD PTR _nObject$[ebp]<br />call ?SetNumber@Number@@QAEXHH@Z ; Number::SetNumber<br /></pre><br /><br /><h2>Virtual Functions</h2><br /><p><br />In case of virtual functions the compiler does not call a function of a classe directly. It rather <br />maintains table (called vtable) of function pointer for each class and while creating object of <br />a class assigns the corresponding classes vtable as the first member of the class. The function <br />call is indirect through this tables entry. <br /><br /><br />Let us create two classes with virtual functions here. <br /><pre><br />//A class with 2 virtual functions<br />class VirtualClass<br />{<br />public:<br /> VirtualClass()<br /> {<br /> }<br /> virtual int TheVirtualFunction()<br /> {<br /> return 1;<br /> }<br /> virtual int TheVirtualFunction2()<br /> {<br /> return 2;<br /> }<br />};<br /><br /><br />//Subclass<br />class SubVirtualClass: public VirtualClass<br />{<br />public:<br /> SubVirtualClass()<br /> { <br /> }<br /><br /> virtual int TheVirtualFunction()<br /> {<br /> return 3;<br /> }<br />};<br /></pre><br /><br />Here is vtable of class <code>VirtualClass</code>.<br /><br /><pre><br />CONST SEGMENT<br />??_7VirtualClass@@6B@ DD FLAT:??_R4VirtualClass@@6B@ ; VirtualClass::`vftable'<br /> DD FLAT:?TheVirtualFunction@VirtualClass@@UAEHXZ<br /> DD FLAT:?TheVirtualFunction2@VirtualClass@@UAEHXZ<br /> DD FLAT:__purecall<br />CONST ENDS<br /></pre><br />Please note that we have a table of three entry with one entry set to NULL (__purecall). This will <br />be assigned in subclass. Without this pure virtual function in source class we could create an object <br />and call the two virtual functions that would be base classes.<br /><br /><br />And the <code>SubNumber</code> classe's vtable is like this:<br /><pre><br />CONST SEGMENT<br />??_7SubVirtualClass@@6B@ DD FLAT:??_R4SubVirtualClass@@6B@ ; SubVirtualClass::`vftable'<br /> DD FLAT:?TheVirtualFunction@SubVirtualClass@@UAEHXZ<br /> DD FLAT:?TheVirtualFunction2@VirtualClass@@UAEHXZ<br /> DD FLAT:?PureVirtualFunction@SubVirtualClass@@UAEHXZ<br />CONST ENDS<br /></pre><br /><br />We get all three virtual functions assigned here. As we did not override the <br /><code>TheVirtualFunction2</code> function we have the base classes pointer in the subclasses <br />vtable- expected. <br /><br /><br />OK, but we must set the table as first member of a class object, right? Its done in the constructor. <br />Here is the constructor of subclass:<br /><br /><pre><br />; 64 : SubVirtualClass()<br /> push ebp<br /> mov ebp, esp<br /> push ecx<br /> mov DWORD PTR _this$[ebp], ecx<br /> mov ecx, DWORD PTR _this$[ebp] ;we get this pointer<br /> ;lets call base classes constructor here<br /> call ??0VirtualClass@@QAE@XZ ; VirtualClass::VirtualClass<br /> mov eax, DWORD PTR _this$[ebp]<br /> mov DWORD PTR [eax],OFFSET ??_7SubVirtualClass@@6B@ ;the vtavle set now<br /></pre><br /></p><br /><br /><h2>Conclusion</h2><br /><p><br />Thats all for now. I want to add Inheritance, Polymorphism, Operator Overloading, Event mechanism, <br />Template, COM Programming and exception handling in future.<br /></p><br /><!--<br /><h2>Inheritance</h2><br /><h2>Polymorphism</h2><br /><h2>Operator Overloading</h2><br /><h2>Event mechanism</h2><br /><h2>Template</h2><br /><h2>COM Programming</h2><br /><h2>Exception Handling</h2><br />-->MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com1tag:blogger.com,1999:blog-767210891894826844.post-84467647470935429942008-04-28T15:29:00.000+06:002008-04-28T15:37:53.312+06:00Longest month of my lifeApril 2008. What really makes time shorter or longer? I do not think its velocity. Its waiting for something you really want but do not know if you are going to get that.MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com2tag:blogger.com,1999:blog-767210891894826844.post-7636571000787997582007-09-06T11:15:00.000+06:002007-09-06T11:39:55.319+06:00Switching on Silverlight“Web is approaching the desktop” – Wahid Bhai said while demonstrating the new Flex project at KAZ. Flex is Flash but on steroids… well to be truthful Flex is Flash with libraries and an easy programming model and the nicest of all - a good eclipse based IDE.<br /><br />Microsoft has of course an answer to Flex - The Silverlight which used to be known as WPF/e for a while. In Microsoft’s wording:<br /><br />“Silverlight is Microsoft’s latest technology to design interactive web client with truly object oriented language C# with full support of VB .NET and other .NET citizens. Microsoft® Silverlight™ is a cross-browser, cross-platform plug-in for delivering the next generation of .NET based media experiences and rich interactive applications for the Web … Silverlight offers a flexible programming model that supports AJAX, VB, C#, Python, and Ruby, and integrates with existing Web applications. Silverlight supports video to all major browsers running on the Mac OS or Windows…. ” (the babble continues promising cure for cancer, food for Sudan and even the impossible - patriotism for the Bangladeshi middle class!)<br /><br />Back on Earth, we the “Psycho group” at KAZ had a week of free time in between projects. So in the time honored way at KAZ we decided to do a spacer project to try out silverlight in all its Alpha glory and MS’s super hype! We decided to make a webtop with silverlight that would show an abstracted backend filesystem exposed by WCF.<br /><br />This post is about our pains and joys during the project. Not gifted with great writing skills (apart from the coding kind) I will try putting down disperate items that we learnt or I felt like telling about our project.<br /><br /><span style="font-size:130%;">First principle </span><br /><br /><br />Starting from Silverlight 1.1 the managed code is supported at client side. We decided to be very strict in this project to use only managed code for programming- no JavaScript. Like all principles this soon turned out to be a pain in a not so polite place<br /><br /><span style="font-size:130%;">Installing Silverlight </span><br /><br />To develop with Silverlight 1.1 you need Visual Studio 2008 (code name Orcas). While writing this downloadable beta version is available from Microsoft site. You also need Silverlight 1.1 plug-in distribution – alpha is available while writing this. Though not necessary the Silverlight 1.1 SDK is highly recommended. The SDK includes controls, samples, documents which may be useful to start. Please note that the Silverlight plug-in is required on any client from where the site is viewed.<br /><br /><span style="font-size:130%;">Controls that comes from Boss<br /></span><br />None!<br /><br />Well not exactly true, but close enough. Silverlight 1.1 alpha distribution comes with a very limited set of controls. It has controls like rectangle, ellipse and label, canvas. It does not include button, edit box, scroll bar or any other advanced control. The SDK includes some controls like scrollbars.<br /><br /><br /><span style="font-size:130%;">Our Architecture</span><br /><br />With no major controls available and no infrastructure for a shell, we soon realized that we need a petzold like approach to the project. We actually need to create a windowing system and the bare basics of message management.<br /><br />Our final designe came out with 3 layers. Top layer is the application layer where user applications run. Middle layer is kernel which controls the events and communication between applications. It also provides a set of API for application developers. The lowest layer is an abstraction layer to communicate with the web server.<br /><br /><img id="BLOGGER_PHOTO_ID_5106955919490422386" style="DISPLAY: block; MARGIN: 0px auto 10px; CURSOR: hand; TEXT-ALIGN: center" alt="" src="http://1.bp.blogspot.com/_E3TPmqtfN0E/Rt-NWdjOFnI/AAAAAAAAAAM/SOdg2xyYgHw/s320/image002.jpg" border="0" /><br /><br /><p></p><p>The system provides API’s for application developers for the platform while hiding the http calls – making the application development similar to desktop application development. Web call abstraction layer does that for user application.<br /></p><br /><p><span style="font-size:130%;">The Kernel<br /></span><br />The controller behind the scene controls form events, focus of controls, controls and windows to common file operations. The kernel has four main parts:</p><ul><li>• Messaging System and Focus Manager<br />• Window Manager<br />• Process Manager<br />• File system Driver<br />• Resource Manager</li></ul><img id="BLOGGER_PHOTO_ID_5106957289584989826" style="DISPLAY: block; MARGIN: 0px auto 10px; CURSOR: hand; TEXT-ALIGN: center" alt="" src="http://4.bp.blogspot.com/_E3TPmqtfN0E/Rt-OmNjOFoI/AAAAAAAAAAU/HU_VYpXm2Xk/s320/image002.jpg" border="0" /><br /><br /><br /><p><span style="font-size:130%;">Messaging System and Focus Manager</span><br /><br />An event first comes to this manager and depending on the type routed to controls. If the user clicks on a control the focus manager updates the focus of controls. It keeps track of current on focus control. If it finds a change in focus it send OnLostFocus call to old control and OnSetFocus call to new focused control. With this exception most messages are routed to the focused control on arrival. </p><br /><br /><p><br /><span style="font-size:130%;">Window Manager</span><br /><br />The window manager keeps track of each window that is created on the system. At any time window manager provides the list of windows currently available in the system. A utility application like TaskManager or TaskBar can use the list for display and control the windows. In our case the TaskBar buttons are created from the list and user can control minimize or restore operation from the TaskBar application. To keep track of the windows the window manager uses an internal generic Window list. When a new window is created the window is registered with window manager and when a window is disposed the window is unregistered. </p><br /><br /><p><br /><span style="font-size:130%;">Process Manager<br /></span><br />The SilverlightDesktop applications that are created by implementing the IApplication interface in the main class are managed by the process manager. The process manager exposes a service to create a new process. It accepts class name as its first parameter and other parameters are passed through a Parameter object that can contain strings. On success the process manager returns a Instance object that can identify the process. Simplified version of Instance class is like this –<br /><br />public class Instance{ public int Handle; public int ParentHandle; public string Name; }<br /><br />The returned interface object can uniquely identify the process and it is required to handle the process.</p><br /><p><img id="BLOGGER_PHOTO_ID_5106958328967075474" style="DISPLAY: block; MARGIN: 0px auto 10px; CURSOR: hand; TEXT-ALIGN: center" alt="" src="http://2.bp.blogspot.com/_E3TPmqtfN0E/Rt-PitjOFpI/AAAAAAAAAAc/ubIcuJrRTR4/s320/image002.jpg" border="0" /><span style="font-size:130%;">Designing a new control<br /><br /></span>ControlBase class can be extended to design a custom control. The control may have a xaml also. The ControlBase getter ResourceName of type string should be overirden to specify the name of the xaml resource in the assembly. It is also possible to set the assembly of the control. The constructor of ControlBase class iterates through each resource and look for the specified resource to get the right xaml resource. Same technique is used in the Silverlight SDK. For example we want to have a icon control that has a image and text. Our xampl file would be similar to this:<br /><br /><blockquote><p><Canvas xmlns=”<a href="http://schemas.microsoft.com/client/2007">http://schemas.microsoft.com/client/2007</a>”<br /> xmlns:x=”<a href="http://schemas.microsoft.com/winfx/2006/xaml">http://schemas.microsoft.com/winfx/2006/xaml</a>”<br /> Width=”32″ Height=”50″ x:Name=”_container" ><br /><Canvas x:Name=”_icon” Width=”32″ Height=”32″ Canvas.Left=”0″ Canvas.Top=”0″/><br /><br /><TextBlock x:Name=”_name” Width=”80″ Height=”15″ Canvas.Left=”-24″ Canvas.Top=”32″ /><br /></Canvas></p></blockquote><br /><p>We used a Canvas for icon, a TextBlock for label and a container canvas to hold them both.<br /><br />And our code back for the control would be:</p><p><br />We used a Canvas for icon, a TextBlock for label and a container canvas to hold them both.And our code back for the control would be:<br /><br />public class Icon : ControlBase{<br />Canvas icon;<br />TextBlock text;<br />Canvas container;<br />protected override void OnInitialized() {<br />icon = actualControl.FindName(”_icon”) as Canvas;<br />text = actualControl.FindName(”_name”) as TextBlock;<br />container = actualControl.FindName(”_container”) as Canvas;<br />}<br />protected override string ResourceName {<br />get {<br />return “Icon.xaml”;<br />}<br />}<br />}</p><p>Now if we add some getter/setter we get the icon control ready to be used.</p><p><span style="font-size:130%;">The Window<br /><br /></span>Window is a customized control- but is different from others. It uses mouse events to implement feature like drag/ drop, has a title bar and sizing buttons for minimize to system taskbar, maximize to cover full user desktop area or close button to close and dispose the control from the system.</p><img id="BLOGGER_PHOTO_ID_5106959707651577506" style="DISPLAY: block; MARGIN: 0px auto 10px; CURSOR: hand; TEXT-ALIGN: center" alt="" src="http://3.bp.blogspot.com/_E3TPmqtfN0E/Rt-Qy9jOFqI/AAAAAAAAAAk/vQPmW-oh1no/s320/image002.jpg" border="0" />It was first posted at <a href="http://www.kaz.com.bd/blog/index.php/?p=10">http://www.kaz.com.bd/blog/index.php/?p=10</a><br /><p></p>MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com1tag:blogger.com,1999:blog-767210891894826844.post-35083189295593091472007-06-06T15:11:00.000+06:002007-06-06T16:23:44.950+06:00MISL to C# (Sharp) -> Loops - the simple casesI wish, as a decompiler writer, there would be no loops. Programmers use thousands of goto statements with if statements. But as it is not the case I must understand how to parse MSIL instructions that were generated from the loops. <br /><br />Of the three types of most common loops (<em>for</em>, <em>while</em>, <em>do-while</em>) the <em>while</em> loop is the basic one. The block that is generated from any type of these loops usually has a conditional jump (usually a brtrue.s <label>) as the last instruction of the block. The difference from <em>if</em> structure is- the instruction jumps to an offset less than current instruction offset. The <em>for</em> and <em>while</em> loop has a unconditional branch (br.s <label>) to an offset that is between start and end of the block. The jump target usually at the beginning of the condition checking instructions. The <em>do-while</em> loop lacks this branch for the reason - it does not test the condition before it is at the end of the block. <br /><br />So, we get instruction block like following MSIL block:<br /><br />-------------------------------------------------------------------------------------<br />IL_0010: br.s IL_005a ;do-while loop does not have this line<br />IL_0012: nop<br /><br />[any type and number of instructions]<br /><br />IL_0059: nop<br /><br />[condition check instruction- results boolean value on stack]<br /><br />IL_0060: brtrue.s IL_0012<br />-------------------------------------------------------------------------------------<br /><br />On my previous post on <em>if</em> structure I showed how to create a boolean condition for <em>if</em> structure. Things are similar here for the loops. Follow the instructions - get the top stack element when conditional jump found - reverse it (add just an !) for brtrue.s jump and put it as the loop statements condition statement. Please note that the conditional jump targets the instruction just at or after the starting instruction of the block.<br /><br />Here we find we can not have a single passing decompiler. We must identify the code blocks in an iteration before final iteration. Till now we can identify blocks of <em>if</em>, <em>for</em>, <em>while</em>, <em>do-while</em> structures by using conditional jump instructions and their destination. If has destination offset after the current offset and others have destination before the current offset. The <em>for</em> and <em>while</em> can not be distinguished very clearly but the <em>do-while</em> does not have a jump at the beginning. And of course there can be nested blocks that are generates from nested loops.<br /><br />There is some complex variation of the loops - like infinite loops, <em>forcach</em> loop etc. They are not much different. But I want to discuss the control structure later in more detail. This post is little bit more theoretical- see you again very soon with more interesting things.MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-767210891894826844.post-16106046493853379072007-06-04T16:32:00.000+06:002007-06-04T18:51:01.611+06:00MISL to C# (Sharp) -> Iff (if and only if)Today I’ll talk about the most basic and useful if-then structure. The compiler generates a conditional branch. We create a block of instructions for the structure. The block is separated by the branch instruction (like brtrue) and the branch label (like IL_0019 - where the code jumps). And our condition is on the stack. If we find a true condition branch we negate it and put it as a if statement condition. The block is initially a block of MSIL that we will convert to C# code later (<br />may need recursion here??). Please note that the labels are not stored in MSIL. It is just the byte offset of the MSIL in a method.<br /><br />If you have not already read you are requested to read my previous post.<br /><br /><br />We take a simple method to test our theory.<br /><br />public int IfStructure(int a, int b)<br />{<br />bool CS_4_0001;<br />if (a < b)<br />{<br />System.Console.Write("Condition is true");<br />}<br />return b;<br />}<br /><br />Here is the MSIL code generated by Visual Studio 2005 compiler.<br /><br />.method public hidebysig instance int32 IfStructure(int32 a, int32 b) cil managed<br />{<br /> // Code size 31 (0x1f)<br /> .maxstack 2<br /> .locals init ([0] int32 CS$1$0000,<br /> [1] bool CS$4$0001)<br /> IL_0000: nop<br /> IL_0001: ldarg.1<br /> IL_0002: ldarg.2<br /> IL_0003: clt<br /> IL_0005: ldc.i4.0<br /> IL_0006: ceq<br /> IL_0008: stloc.1<br /> IL_0009: ldloc.1<br /> IL_000a: brtrue.s IL_0019<br /><br /> IL_000c: nop<br /> IL_000d: ldstr "Condition is true"<br /> IL_0012: call void [mscorlib]System.Console::Write(string)<br /> IL_0017: nop<br /> IL_0018: nop<br /><br /> IL_0019: ldarg.2<br /> IL_001a: stloc.0<br /> IL_001b: br.s IL_001d<br /> IL_001d: ldloc.0<br /> IL_001e: ret<br />} // end of method ControlStructures::IfStructure<br /><br />We now start parsing:<br />-------------------------------------------------------------------------------------<br />.method public hidebysig instance int32 IfStructure(int32 a, int32 b) cil managed<br />{<br /> // Code size 31 (0x1f)<br /> .maxstack 2<br /> .locals init ([0] int32 CS$1$0000,<br /> [1] bool CS$4$0001)<br />--<br /><br />These lines generate output that we do without any processing of MSIL. There is method definition and local variables. We change the local variable names to C# current names without conflict. For simplicity here we just replace '$' with '_'. One thing to evaluate the MSIL instructions we must keep a map of local variables with variable number. In this example CS$1$0000 is local variable 0 of type int. For clarity we do not show the map here. A simple STL map should work.<br /><br />Output:<br />public int IfStructure(int a, int b)<br />{<br />int32 CS_1_0000;<br />bool CS_4_0001;<br /><br />Stack: [Empty]<br /><br />-------------------------------------------------------------------------------------<br />IL_0000: nop<br />IL_0001: ldarg.1<br />IL_0002: ldarg.2<br />--<br />So we push method argument 1 and 2 to stack:<br /><br />Output: [None]<br /><br />Stack:<br />a,b<br /><br />-------------------------------------------------------------------------------------<br />IL_0003: clt<br />--<br /><br />This instructs us if stack top-1 is less than stack top. The two elements are popped from stack and result goes to stack. No output of course.<br /><br />Output: [None]<br /><br />Stack:<br />a<b<br /><br />-------------------------------------------------------------------------------------<br />IL_0005: ldc.i4.0<br />--<br /><br />Load (aka push) constant integer of value 0 on stack.<br /><br />Output: [None]<br />Stack: a<b,0<br />-------------------------------------------------------------------------------------<br />IL_0006: ceq<br />--<br /><br />Check if stack top-1 equals stack top. Result goes to stack.<br />Output: [None]<br />Stack: a < b == 0<br /><br />-------------------------------------------------------------------------------------<br />IL_0008: stloc.1<br />IL_0009: ldloc.1<br />--<br /><br />What else? Store stack top in local variable 1 and load that on tack again. We decided previously when we store some value in a local variable we use assignment to that variable and output that code. For clarity I added parentheses:<br /><br />Output:CS_4_0001 = (a < b == 0)<br />Stack:CS_4_0001<br /><br />-------------------------------------------------------------------------------------<br />IL_000a: brtrue.s IL_0019<br />--<br /><br />We have got a conditional branch. We create a block starting from here to IL_0019. And put them in curly braces. And our condition is on the stack. We find a true condition branch so we negate it and put it as if structure as I said at the beginning.<br /><br />Output:<br />if(!CS_4_0001)<br />{<br />IL_000c: nop<br />IL_000d: ldstr "Condition is true"<br />IL_0012: call void [mscorlib]System.Console::Write(string)<br />IL_0017: nop<br />IL_0018: nop<br />}<br /><br />Stack: [Empty]<br /><br />-------------------------------------------------------------------------------------<br />IL_0019: ldarg.2<br />IL_001a: stloc.0<br />IL_001b: br.s IL_001d<br />IL_001d: ldloc.0<br />IL_001e: ret<br /><br />We do not parse them here. They are very simple to understand.<br />===========================================================<br /><br />Ok we now can work on little more complex code3s than easiest. This will also produce codes that was generated by "for structure" but in a funny way. If we add a little more intelligence to produce "goto" output code for special branching that we cannot handle with if, we get following result.<br /><br />The code like this:<br /><br /><br />for(int i=0;i<10;i++)<br />{<br />...<br />}<br />...<br /><br />will be converted to:<br /><br />int i;<br />i=0;<br />label_1:<br /><br />if(i<10)<br />{<br />---<br />i++;<br />goto label_1;<br />}<br />...<br /><br />For is not our today’s material. We'll look at loops next time.<br /><br />=====================================================================================<br />Now you may find that our theory generates a funny code block like:<br /><br />CS$4$0001=((a<b)==0);<br /><br />if(!CS$4$0001)<br />{<br /><br />---<br /><br />}<br /><br /><br />Here the optimization comes to scene. But we skip them for future.MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com4tag:blogger.com,1999:blog-767210891894826844.post-57258779987992970692007-05-28T14:43:00.000+06:002007-05-28T17:23:51.712+06:00MSIL to C# -> The theory development beginsWelcome to my journey of writting a .NET assembly decompiler.<br /><br />First of all I am trying to develop a theory to decompile MSIL. I just do whatever a MSIL instruction ask me to do. But I do it keeping in mind that I am decompiling MSIL. So when it asks to push me a value of a variable I push the name of that varuiable on stack.<br /><br />To understand the code it is required that you know or have a reference of what each isntruction<br />of MSIL actually does.<br /><br />Here is sample program to test if our concept works:<br /><br />namespace DisasmIL<br />{<br />class Math<br />{<br />public int add(int x, int y)<br />{<br />return x + y;<br />}<br />}<br /><br />class Program<br />{<br />static void Main(string[] args)<br />{<br />Math m;<br />int a, b;<br />m = new Math();<br />a = 20;<br />b = 50;<br />int p = m.add(a, b);<br />}<br />}<br />}<br /><br />We only check one method. The Main method.When the Main method is<br />compiled it takes following form.<br />.method private hidebysig static void Main(string[] args) cil managed<br />{<br />.entrypoint<br />// Code size 23 (0x17)<br />.maxstack 3<br />.locals init (<br />[0] class DisasmIL.Math m,<br />[1] int32 a,<br />[2] int32 b,<br />[3] int32 p<br />)<br />IL_0000: nop<br />IL_0001: newobj instance void DisasmIL.Math::.ctor()<br />IL_0006: stloc.0<br />IL_0007: ldc.i4.s 20<br />IL_0009: stloc.1<br />IL_000a: ldc.i4.s 50<br />IL_000c: stloc.2<br />IL_000d: ldloc.0<br />IL_000e: ldloc.1<br />IL_000f: ldloc.2<br />IL_0010: callvirt instance int32 DisasmIL.Math::'add'(int32,int32)<br />IL_0015: stloc.3<br />IL_0016: ret<br />} // end of method Program::Main<br />=============================================================<br /><br />We parse line by line:<br />------------------------------------------------------------------------<br />.method private hidebysig static void Main(string[] args) cil managed<br /><br />It is method declaration with default starting curly brace.<br />Output code:<br />static void Main(string[] args)<br />{<br />Stack: [empty]<empty><br />------------------------------------------------------------------------<br />.entrypoint<br />// Code size 23 (0x17)<br />.maxstack 3<br />.locals init (<br />[0] class DisasmIL.Math m,<br />[1] int32 a,<br />[2] int32 b,<br />[3] int32 p<br />)<br /><br />Need not to be explained. They are self explanatory. Declare variables.<br />Output code:<br />DisasmIL.Math m;<br />int a;<br />int b;<br />int p;<br />Stack: <empty><br />------------------------------------------------------------------------<br />IL_0000: nop:<br /><br />Does nothing (nop).<br />Output code: [none]<none><br />Stack: [empty]<empty><br />------------------------------------------------------------------------<br />IL_0001: newobj instance void DisasmIL.Math::.ctor()<br /><br />Create new instance of DisasmIL.Math using default constructor<br />so we push "new DisasmIL.Math()" on our stack.<br />Output code: <none><br />Stack: new DisasmIL.Math()<br />------------------------------------------------------------------------<br />IL_0006: stloc.0<br /><br />So we pop top of stack and assign it to local variable 0.<br />Output code:<br />m = new DisasmIL.Math();<br />Stack: [empty]<empty><br />------------------------------------------------------------------------<br />IL_0007: ldc.i4.s 20<br /><br />What we do is push constant 20 on stack.<br />Output code: [none]<none><br />Stack: 20<br />------------------------------------------------------------------------<br />IL_0009: stloc.1<br /><br />So we pop top value and assign it to local variable 1.<br />Output code:<br />a = 20;<br />Stack: [empty]<empty><br />------------------------------------------------------------------------<br />IL_000a: ldc.i4.s 50<br /><br />We push constant 50 on stack.<br />Output code: [none]<none><br />Stack: 50<br />------------------------------------------------------------------------<br />IL_000c: stloc.2<br /><br />We pop top value and assign it to local variable 2.<br />Output code:<br />b = 50;<br />Stack: [empty]<empty><br />------------------------------------------------------------------------<br />IL_000d: ldloc.0<br />IL_000e: ldloc.1<br />IL_000f: ldloc.2<br /><br />Push local variable 0, 1 and 2 on stack.<br />Output code: <none><br />Stack: m, a, b<br />-------------------------------------------------------------------------<br />IL_0010: callvirt instance int32 DisasmIL.Math::'add'(int32,int32)<br />IL_0015: stloc.3<br /><br />We call add method with values top-1, top of stack for instance of<br />top-2. For any method call if it returns value it is returned on<br />stack. So check next instruction. If it is a stloc then we assign the<br />return value. We assign return value to local variable 3.<br /><br />Output code: p=m.add(a,b);<br />Stack: <empty><br />--------------------------------------------------------------------------<br />IL_0016: ret<br /><br />Return void. So no code except closing curly brace.<br />Output code:<br />}<br /><br />Stack: [empty]<br /><br />Now if you add the output codes together you'll find the original C# code is generated. This works for simple cases. Need to test if it works for complex situations. Miles to go....<empty>MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com2