x64dbg plugin SDK

Contents

Overview

The x64dbg plugin software development kit (SDK) is used to create plugins for the x64dbg debugger. This article aims to do a number of things:

Please note that there is a lot of code listings in this article, which may make for rather dry reading, and more like a technical manual overall.

Common questions

A number of questions might arise straight away, so lets answer them before we continue.

Wait, what? there are two plugin SDKs?

Yes, the integrated one is already provided for by x64dbg, and comes included with the latest snapshots. The second one was created by myself, converting the existing plugin SDK to an assembler friendly one for use with Masm32 (x86) or JWasm / HJWasm (x64)

Which plugin SDK should I use?

Whichever you feel more comfortable with. If you prefer to code in C or C++ then the integrated plugin SDK is probably more suited to you.

Why create a plugin SDK in assembler?

In practice it comes down to what language you are most familiar coding with, and in my case it happens to be assembler, and by providing an SDK for assembler, it opens up the door to the possibility of other assembly coders using it. And another obvious answer would be to do with the nature of the product, which being a debugger shows its output (the disassembly) in raw assembly language - what better environment for assembly coders to learn and further their own development?

What assembler should I use, if I’m to use the plugin SDK for assembler?

As the plugin SDK for x86 was written using Masm32, for x86 plugin development I recommend you use the MASM32 SDK along with the x64dbg Plugin SDK For x86 Assembler. The plugin SDK for x64 was written using JWasm (64bit) and for x64 plugin development both JWasm (the last/latest release) and HJWasm (fork and continuation of JWasm) can be used along with x64dbg Plugin SDK For x64 Assembler.

JWasm / HJWasm are considered masm compatible, so they could also be used for the x86 development if you so wish. Other assemblers can be used, but you might have to make a few changes here and there with the examples shown in this post.

Why write a plugin?

Those answers are more general and apply to plugin development in general, but with open source projects which can be more fluid in their development, there are other considerations and caveats.

Ongoing development

Fixing issues (bug fixes) and enhancing functionality are an ongoing part of open source projects. So the additions of new features (via feature requests) is not uncommon. Also features or functionality that a plugin provide can become obsolete when the feature they provide is directly including in the open source project.

Feature request alignment

Sometimes features requests and the overall direction and goal of the open source project may not align - even if they are directly or indirectly related to the overall project - sometimes it is more expedient for the feature requests to be moved out to a plugin.

Other times, features that where planned for inclusion in the open source project are passed by as different milestones are set, which may mean directing the feature request out to plugin development.

Certain features may only used by a small amount of users, and again it is more efficient to outsource the feature to a plugin for the small amount of users who make use of the functionality.

Understanding the x64dbg plugin architecture

The plugin files for x64dbg, are files that end with the .dp32 or .dp64 extension. These correspond to the processor architecture used in each version of x64dbg - 32bit and 64bit. In reality these plugin files (.dp32 for 32bit x32dbg and .dp64 for 64bit x64dbg) are just simple dynamic link libraries (.dll files).

Each plugin file has to export a number of functions for it to be recognised as a valid and usable plugin. These are:

and optionally:

Technically only DllMain and pluginit are required at a minimum, but it is considered a good practise to include plugstop to allow for cleanup of your plugins code, should it be required, and plugsetup for obtaining menu handles if your plugin will be creating its own menu items.

The plugin load sequence

These are the steps that occur when plugins are being loaded by the x64dbg debugger. The functions and structures mentioned are covered in more detail in other sections below.

In a loop of all matching files the following takes place:

Understanding the plugin loading sequence will hopefully help you understand where best to place your code for your plugin and the obvious impact of having cpu intensive code in the initialization and setup functions. Other plugins will be delayed in loading and the x64dbg debugger itself will be waiting for your code to finish before it can continue on to do its main job of debugging.

DllMain

DllMain is the entry point into a dynamic link library, and is optional for each dll file to have one. The plugins, being dll files, can make use of this function by storing the HINSTANCE hInst value for later use in other api calls. The code required for creating a DllMain function is relatively straightforward.

Ive included an example of defining the DllMain using C++ (for both 32bit and 64bit), Masm32 for x86 assembly and JWasm / HJWasm for the x64 assembly.

C++:

extern "C" DLL_EXPORT BOOL APIENTRY DllMain(HINSTANCE hInst, DWORD fdwReason, LPVOID lpvReserved)
{
    if(fdwReason == DLL_PROCESS_ATTACH)
    {
        hInstance = hInst; // save instance handle for future use
    }
    return TRUE;
}

Assembler x86:

DllMain PROC hInst:HINSTANCE, fdwReason:DWORD, lpvReserved:DWORD
    .IF fdwReason == DLL_PROCESS_ATTACH
        mov eax, hInst
        mov hInstance, eax ; save instance handle for future use
    .ENDIF
    mov eax,TRUE
    ret
DllMain ENDP

Assembler x64:

DllMain PROC hInst:HINSTANCE, fdwReason:DWORD, lpvReserved:LPVOID
    .IF fdwReason == DLL_PROCESS_ATTACH
        mov rax, hInst
        mov hInstance, rax ; save instance handle for future use
    .ENDIF
    mov rax,TRUE
    ret
DllMain ENDP

Fairly simple for each example, and minimal changes for the 64bit assembler version vs the 32bit, mainly the use of the rax register instead of eax.

Apart from DllMain all the plugin functions (except from your own internal functions) are required to be exported for the plugin to work.

An exported function is one that has been declared as accessible externally for other external callers to make use of. DllMain is ‘seen’ and automatically handled by the operating system when loading a dynamic link library, so we aren’t required to explicitly export it, but the pluginit, plugstop, plugsetup and any other CB* callback functions are required to be exported.

The pluginit exported function

pluginit is the first exported function that x64dbg calls after loading the dynamic link libraries (.dp32 or .dp64), if pluginit isn’t present then the loading of the plugin will fail at this point.

pluginit passes a pointer to a PLUG_INITSTRUCT structure as the only parameter in the function: initStruct. This structure is used to register the plugin with x64dbg and for obtaining a valid plugin handle which is used in future api calls.

The definition for this structure is:

C++:

typedef struct
{
    //provided by the debugger
    int pluginHandle;
    //provided by the pluginit function
    int sdkVersion;
    int pluginVersion;
    char pluginName[256];
} PLUG_INITSTRUCT;

Assembler x86 and x64:

PLUG_INITSTRUCT   STRUCT
    pluginHandle  DWORD ? ; Plugin handle. Data provided by the debugger to the plugin.
    sdkVersion    DWORD ? ; Plugin SDK version, Data provided by the plugin to the debugger (required).
    pluginVersion DWORD ? ; Plugin version. Data provided by the plugin to the debugger (required).
    pluginName    DB 256 DUP (?) ; Plugin name. Data provided by the plugin to the debugger (required).
PLUG_INITSTRUCT   ENDS

So we must place valid information back into this structure for x64dbg to validate and recognise our plugin. A number of defines can be used to provide the information required by the structure’s fields: pluginVersion (PLUGIN_VERSION - defined by user) and sdkVersion (PLUG_SDKVERSION - predefined in the SDK - version of the SDK that x64dbg expects). The pluginName is a string buffer that contains the name of your plugin. The pluginHandle field is provided to us as a handle that we can save and re-use in future api calls.

Of course we need to declare some of the variables and strings used to pass this information back via the PLUG_INITSTRUCT structure, and to store information returned to us (pluginHandle in pluginit and other handles in the plugsetup call later on). Here’s how to define them:

C++:

// Plugin SDK required variables
#define plugin_name "x64dbg_plugin" // rename to your plugins name 
#define plugin_version 1

// GLOBAL Plugin SDK variables
int pluginHandle;
HWND hwndDlg;
int hMenu;
int hMenuDisasm;
int hMenuDump;
int hMenuStack;

Assembler x86:

.CONST
PLUGIN_VERSION      EQU 1

.DATA
PLUGIN_NAME         DB "x64dbg_plugin",0 ; rename to your plugins name

.DATA?
; Plugin SDK required variables
PUBLIC              pluginHandle
PUBLIC              hwndDlg
PUBLIC              hMenu
PUBLIC              hMenuDisasm
PUBLIC              hMenuDump
PUBLIC              hMenuStack

pluginHandle        DD ?
hwndDlg             DD ?
hMenu               DD ?
hMenuDisasm         DD ?
hMenuDump           DD ?
hMenuStack          DD ?

Assembler x64:

.CONST
PLUGIN_VERSION      EQU 1

.DATA
PLUGIN_NAME         DB "x64dbg_plugin",0 ; rename to your plugins name

.DATA?
; Plugin SDK required variables
PUBLIC              pluginHandle
PUBLIC              hwndDlg
PUBLIC              hMenu
PUBLIC              hMenuDisasm
PUBLIC              hMenuDump
PUBLIC              hMenuStack

pluginHandle        DD ?
hwndDlg             DQ ?
hMenu               DD ?
hMenuDisasm         DD ?
hMenuDump           DD ?
hMenuStack          DD ?

Now that we have setup our variables and strings we are ready to construct our pluginit function. Below is a couple of examples of pluginit in C++ (for both 32bit and 64bit), Masm32 for x86 assembly and JWasm / HJWasm for the x64 assembly.

C++:

DLL_EXPORT bool pluginit(PLUG_INITSTRUCT* initStruct)
{
    initStruct->pluginVersion = plugin_version;
    initStruct->sdkVersion = PLUG_SDKVERSION;
    strcpy(initStruct->pluginName, plugin_name);
    pluginHandle = initStruct->pluginHandle;

    // place any additional initialization code here
    
    return true;
}

Assembler x86:

pluginit PROC C PUBLIC USES EBX initStruct:DWORD
    mov ebx, initStruct

    ; Fill in required information of initStruct, which is a pointer to a PLUG_INITSTRUCT structure
    mov eax, PLUGIN_VERSION
    mov [ebx].PLUG_INITSTRUCT.pluginVersion, eax
    mov eax, PLUG_SDKVERSION
    mov [ebx].PLUG_INITSTRUCT.sdkVersion, eax
    Invoke lstrcpy, Addr [ebx].PLUG_INITSTRUCT.pluginName, Addr PLUGIN_NAME
    
    mov ebx, initStruct
    mov eax, [ebx].PLUG_INITSTRUCT.pluginHandle
    mov pluginHandle, eax
    
    ; Do any other initialization here
    
    mov eax, TRUE
    ret
pluginit ENDP

Assembler x64:

pluginit PROC FRAME USES RBX initStruct:QWORD
    mov rbx, initStruct

    ; Fill in required information of initStruct, which is a pointer to a PLUG_INITSTRUCT structure
    mov eax, PLUGIN_VERSION
    mov [rbx].PLUG_INITSTRUCT.pluginVersion, eax
    mov eax, PLUG_SDKVERSION
    mov [rbx].PLUG_INITSTRUCT.sdkVersion, eax
    Invoke lstrcpy, Addr [rbx].PLUG_INITSTRUCT.pluginName, Addr PLUGIN_NAME
    
    mov rbx, initStruct
    mov eax, [rbx].PLUG_INITSTRUCT.pluginHandle
    mov pluginHandle, eax
    
    ; Do any other initialization here
    
    mov rax, TRUE
    ret
pluginit ENDP

The coding for each example is simple and again we see minimal changes for the 64bit assembler version vs the 32bit.

In assembler (x86 and x64) you must also add your exported functions to a .def file for them to be visible to external callers when compiling and linking:

LIBRARY MyPlugin
EXPORTS pluginit
        plugstop
        plugsetup

The plugsetup exported function

plugsetup is the second export function that x64dbg calls - assuming the first call to pluginit was successful and the information passed back was correct.

It passes a pointer to a PLUG_SETUPSTRUCT structure as the only parameter in the function: setupStruct. This structure is used to pass back various menu handles to the plugin. These handles can be used in future api calls to create menus and menu items for your plugin.

The definition for this structure is:

C++:

typedef struct
{
    //provided by the debugger
    HWND hwndDlg;       // gui window handle
    int hMenu;          // plugin menu handle
    int hMenuDisasm;    // plugin disasm menu handle
    int hMenuDump;      // plugin dump menu handle
    int hMenuStack;     // plugin stack menu handle
} PLUG_SETUPSTRUCT;

Assembler x86 and x64:

; plugsetup - This structure is used by the function that allows the creation of plugin menu entries
PLUG_SETUPSTRUCT STRUCT 8
    hwndDlg      HWND ?  ; GUI window handle. Data provided by the debugger to the plugin.
    hMenu        DWORD ? ; Plugin menu handle. Data provided by the debugger to the plugin.
    hMenuDisasm  DWORD ? ; Plugin disasm menu handle. Data provided by the debugger to the plugin.
    hMenuDump    DWORD ? ; Plugin dump menu handle. Data provided by the debugger to the plugin.
    hMenuStack   DWORD ? ; Plugin stack menu handle. Data provided by the debugger to the plugin.
PLUG_SETUPSTRUCT ENDS

And below is a couple of examples in C++ and assembler of what the plugsetup function will look like (a bare bones version):

C++:

DLL_EXPORT void plugsetup(PLUG_SETUPSTRUCT* setupStruct)
{
    hwndDlg = setupStruct->hwndDlg;
    hMenu = setupStruct->hMenu;
    hMenuDisasm = setupStruct->hMenuDisasm;
    hMenuDump = setupStruct->hMenuDump;
    hMenuStack = setupStruct->hMenuStack;
    
    // place any additional setup code here
    
    return true;
}

Assembler x86:

plugsetup PROC C PUBLIC USES EBX setupStruct:DWORD
    mov ebx, setupStruct

    ; Extract handles from setupStruct which is a pointer to a PLUG_SETUPSTRUCT structure  
    mov eax, [ebx].PLUG_SETUPSTRUCT.hwndDlg
    mov hwndDlg, eax
    mov eax, [ebx].PLUG_SETUPSTRUCT.hMenu
    mov hMenu, eax
    mov eax, [ebx].PLUG_SETUPSTRUCT.hMenuDisasm
    mov hMenuDisasm, eax
    mov eax, [ebx].PLUG_SETUPSTRUCT.hMenuDump
    mov hMenuDump, eax
    mov eax, [ebx].PLUG_SETUPSTRUCT.hMenuStack
    mov hMenuStack, eax

    ; Do any setup here: add menus, menu items, callback and commands etc

    mov eax, TRUE
    ret
plugsetup endp

Assembler x64:

plugsetup PROC FRAME USES RBX setupStruct:QWORD
    mov rbx, setupStruct

    ; Extract handles from setupStruct which is a pointer to a PLUG_SETUPSTRUCT structure  
    mov rax, [rbx].PLUG_SETUPSTRUCT.hwndDlg
    mov hwndDlg, rax
    mov eax, [rbx].PLUG_SETUPSTRUCT.hMenu
    mov hMenu, eax
    mov eax, [rbx].PLUG_SETUPSTRUCT.hMenuDisasm
    mov hMenuDisasm, eax
    mov eax, [rbx].PLUG_SETUPSTRUCT.hMenuDump
    mov hMenuDump, eax
    mov eax, [rbx].PLUG_SETUPSTRUCT.hMenuStack
    mov hMenuStack, eax
    
    ; Do any setup here: add menus, menu items, callback and commands etc

    mov rax, TRUE
    ret
plugsetup endp

In assembler (x86 and x64) you must also add your exported functions to a .def file for them to be visible to external callers when compiling and linking:

LIBRARY MyPlugin
EXPORTS pluginit
        plugstop
        plugsetup

The callback exported functions and structures

The callback functions are provided to help you use your plugin. When a specific event occurs in the x64dbg debugger, it passes this event call to each plugin that has ‘registered’ to receive this event. Typically events that would be of interest to a plugin developer are when debugging of a target process has begun (CB_INITDEBUG), or when a breakpoint has been reached (CB_BREAKPOINT or CB_SYSTEMBREAKPOINT) or other events of interest, like handling a menu interaction for your plugin (CB_MENUENTRY).

These events provide your plugin with a way of interacting with the debugger at these points in time, when an event occurs.

Registering a callback is done in one of two ways:

_plugin_registercallback

This function registers an event callback for a plugin. The definition for this function (defined in the plugin SDK) is:

C++:

void _plugin_registercallback(
    int pluginHandle, //plugin handle
    CBTYPE cbType, //event type
    CBPLUGIN cbPlugin //callback function
);

Assembler x86:

_plugin_registercallback PROTO :DWORD, :DWORD, :DWORD ; (int pluginHandle, CBTYPE cbType, CBPLUGIN cbPlugin)

Assembler x64:

_plugin_registercallback PROTO :QWORD, :QWORD, :QWORD ; (int pluginHandle, CBTYPE cbType, CBPLUGIN cbPlugin)

pluginHandle is the handle of your plugin. In assembler this is passed as a DWORD (for x86) or a QWORD (for x64) value.

cbType is the specific callback type being registered, which is one of the following values (defined in the plugin SDK):

In assembler this is passed as a DWORD (for x86) or a QWORD (for x64) value.

cbPlugin is the address of your exported callback function that will be registered. In assembler this callback functions address is passed as a DWORD (for x86) or a QWORD (for x64) value.

The registered event callback function for _plugin_registercallback

The callback function CBPLUGIN must be an exported library function. CBPLUGIN is replaced with the name of your plugin’s callback function. The definition for your CBPLUGIN function in your plugin looks like this:

C++:

void CBPLUGIN(
    CBTYPE bType // event type, one of the `cbType` CB_* values listed above
    void* callbackInfo // pointer to a structure of information
);

Assembler x86:

CBPLUGIN PROTO :DWORD, :DWORD ; (CBTYPE bType, void* callbackInfo)

Assembler x64:

CBPLUGIN PROTO :QWORD, :QWORD ; (CBTYPE bType, void* callbackInfo)

The bType parameter contains the event type that is occurring. Same as one of the CB_* values listed above for _plugin_registercallback cbType parameter

The callbackInfo parameter contains a pointer to a specific PLUG_CB_* structure. Each callback event type uses a different callbackInfo structure that contains various information related to that particular event. These PLUG_CB_* structures (that are defined in the plugin SDK) are:

We will cover more in depth usage of the specific callback structures later on.

The CDECL export callback function

The second way of using callbacks, which may be easier to implement, is to have specifically named functions exported from your plugin. x64dbg will look for these exported functions when loading your plugin and if found will automatically register these as callback events. The exported functions that will be recognised are:

These function are defined in the plugin SDK for ease of use. And like the CBPLUGIN definition above, they have two parameters: bType and callbackInfo

You can create an export which has the name of the callback like so, taking CBINITDEBUG as our example.

The callbackInfo parameter for CBINITDEBUG is a pointer to PLUG_CB_INITDEBUG structure, defined as:

C++:

typedef struct
{
    const char* szFileName;
} PLUG_CB_INITDEBUG;

Assembler x86:

PLUG_CB_INITDEBUG STRUCT
    szFileName    DWORD ?
PLUG_CB_INITDEBUG ENDS

Assembler x64:

PLUG_CB_INITDEBUG STRUCT 8
    szFileName    QWORD ?
PLUG_CB_INITDEBUG ENDS

This is our C++ example. In C++ you can easily create a CDECL export for the CBINITDEBUG function.

C++:

extern "C" __declspec(dllexport) void CBINITDEBUG(CBTYPE bType, PLUG_CB_INITDEBUG* callbackInfo)
{
    // callbackInfo for CBINITDEBUG is a pointer to PLUG_CB_INITDEBUG 
    // structure containing one field called szFileName, which is a 
    // pointer to a string containing the name of the debuggee file 
    // about to be debugged.
    
    // do something with the filename: (const char*)callbackInfo->szFileName
    // do any other processing 
}

The prototype definitions for the exported CBINITDEBUG function in assembler are:

Assembler x86:

CBINITDEBUG PROTO :DWORD, :DWORD ; (CBTYPE bType, void* callbackInfo)

Assembler x64:

CBINITDEBUG PROTO :QWORD, :QWORD ; (CBTYPE bType, void* callbackInfo)

And the example code for each CBINITDEBUG function in assembler is:

Assembler x86:

CBINITDEBUG PROC C PUBLIC USES EBX cbType:DWORD, callbackInfo:DWORD

    ; callbackInfo for CBINITDEBUG is a pointer to PLUG_CB_INITDEBUG 
    ; structure containing one field called szFileName, which is a 
    ; pointer to a string containing the name of the debuggee file 
    ; about to be debugged.
    
    mov ebx, callbackInfo
    mov eax, [ebx].PLUG_CB_INITDEBUG.szFileName
    
    ; eax now contains the pointer to the filename string.
    ; do any other processing 
    
    ret
CBINITDEBUG ENDP

Assembler x64:

CBINITDEBUG PROC FRAME USES RBX cbType:QWORD, callbackInfo:QWORD

    ; callbackInfo for CBINITDEBUG is a pointer to PLUG_CB_INITDEBUG 
    ; structure containing one field called szFileName, which is a 
    ; pointer to a string containing the name of the debuggee file 
    ; about to be debugged.
    
    mov rbx, callbackInfo
    mov rax, [rbx].PLUG_CB_INITDEBUG.szFileName
    
    ; rax now contains the pointer to the filename string.
    ; do any other processing 
    
    ret
CBINITDEBUG ENDP

In assembler (x86 and x64) you must add your exported functions to a .def file for them to be exported when compiling and linking:

LIBRARY MyPlugin
EXPORTS pluginit
        plugstop
        plugsetup
        CBINITDEBUG

In all the examples above, the CBINITDEBUG function is registered as our callback and called when debugging begins. The callbackInfo parameter contains the pointer to a PLUG_CB_INITDEBUG, from which we can retrieve the name of the file that is about to be debugged. We could display this information in the log window or do something else instead.

Summary

Hopefully if you have read all this article it has covered the following:

Afterword

Lots of code listing and technical information, which might not necessarily be to everyone’s taste, but hopefully in a future article I will cover a specific working plugin example and walk through the development of it - most likely in assembler x86.

I’ve included a list below of additional resources that are related to x64dbg and plugin development which you may find useful.

Thank you for taking the time to read this article.

fearless

Additional resources of interest

x64dbg

x64dbg Plugin SDK For Assembler

Assemblers

Other

Comments