Weekly digest 1

This is the first of (hopefully) many weekly digests. Basically it will highlight the things that happened to x64dbg and related projects during the week before.

Improvements to the attach dialog

Forsari0 contributed this in pull request #994, the attach dialog will now also show the command line arguments for every process:

Disable debuggee notes when debugging

Previously it was possible to edit notes in the ‘Debuggee’ tab, even when you were not debugging anything. These notes would be forever lost so now you can no longer see that tab when not debugging.

Translation of the DBG

In addition to the Qt interface translation, torusrxxx added the infrastructure in the DBG to translate log strings. See pull request #998 for more details.

Search box locking in symbol view

Yet another contribution by Forsari0, in pull request #1003. He added a checkbox in the symbol view that allows you to save your search query when switching to other modules, allowing you to repeat the same query in various different modules:

Various GUI improvements

torusrxxx added various useful interface improvements in pull request #1004:

• Forward and Backward mouse buttons found in some models of mice.
• No confirmation when adding functions.
• Allocate memory from the dump menu.
• Placeholder tips for learners.
• Disable/Cancel log redirect (issue #939)
• Use Windows configuration for the singe step scroll size (issue #1000)

Here is an example of those placeholder tips:

Don’t freeze when the debuggee doesn’t close properly

In certain (rare) cases, the debuggee cannot be terminated correctly. TitanEngine will wait for EXIT_PROCESS_DEBUG_EVENT, but in some cases this event is not received and there will be an infinite debug loop, thus freezing x64dbg when trying to terminate the process. This has been worked around by setting the maximum wait time to 10 seconds.

Warn when setting a software breakpoint in non-executable memory

If you attempt to set a breakpoint in a data location, x64dbg will warn you and ask if you are certain about this. Usually the only thing that comes out of it is data corruption:

Signed and unsigned bytes in the dump

The dump views now include signed and unsigned byte views:

Fixed WOW64 redirection issues

See the dedicated blog post for more information.

Fixed invalid save to file sizes

When using the Binary -> Save to file option the savedata command would be executed but with the decimal size as argument, thus saving far too much data. This has been fixed.

Added imageinfo command

During a discussion in the Telegram channel, someone mentioned OllySEH and I decided to implement a similar command in x64dbg. The command imageinfo modbase will show you a summary of the FileHeader.Characteristics and OptionalHeader.DllCharacteristics fields:

Image information for ntdll.dll
Characteristics (0x2022):
  IMAGE_FILE_EXECUTABLE_IMAGE: File is executable (i.e. no unresolved externel references).
  IMAGE_FILE_LARGE_ADDRESS_AWARE: App can handle >2gb addresses
  IMAGE_FILE_DLL: File is a DLL.
DLL Characteristics (0x4160):
  IMAGE_DLLCHARACTERISTICS_DYNAMIC_BASE: DLL can move.
  IMAGE_DLLCHARACTERISTICS_NX_COMPAT: Image is NX compatible

Adding this command required a small change in TitanEngine (the DllCharacteristics field was not supported).

Updated Yara to 3.5.0

Quite recently the latest version of Yara (3.5.0) was released. This version is now also updated in x64dbg!

Work on GleeBug

GleeBug is the planned new debug engine for x64dbg. During the holidays there has been quite a bit of work with regards to memory breakpoints. See the commit log for in-depth information.

GleeBug has been in development for about 1.5 years now. It can currently replace TitanEngine for x64dbg (giving massive performance improvements), but not all features are implemented yet. The main blockers are currently:

• Memory breakpoints
• Extended (XMM, YMM) register support
• Breakpoint memory transparency

For reference, when tracing with TitanEngine, the events/s counter is around 250. When tracing with GleeBug, this counter comes near 30000 events/s in some cases!

Final words

That has been about it for this week. If you have any questions, contact us on Telegram/Gitter/IRC. If you want to see the changes in more detail, check the commit log.

You can always get the latest release of x64dbg here. If you are interested in contributing, check out this page.

64bit Debugging and the WoW64 File System Redirection

A small note on WoW64 Redirection on Windows

With the introduction to a 64bit Windows Operating System Microsoft introduced the Wow64 emulator. This is a combination of DLLs (aptly named wow64xxx.dll) that automatically handle the proper loading of x86 versions of system libraries for 32-bit processes on 64-bit Windows.

In addition to this, in order to prevent file access and registry collisions WoW64 automatically handles redirecting file operations for 32-bit applications requesting access to system resources. Notice on a 64bit version of Windows, when a 32-bit application requests to open a System file located in %windir%\system32, if that file is also located in the %windir%\SysWOW64 Windows File System Redirection kicks in and provides that application with the 32bit version of the application. This is done due to a combination of factors including pre-defined Registry settings and environment variables setup by WoW64 during application start up.

How this affected the x96dbg.exe loader

In the x64dbg package, a loader is provided as a convenience to the user in order to support Right-Click Context Menu debugging of applications and Desktop shortcuts. However up until recent commits 1 and 2 proper handling of redirection was not present. More information about this issue can be read here in Issue #899.

Due to the fact that x96dbg.exe is a 32-bit application, when a Right Click context menu is invoked to debug a 64-bit application in the System directory, File System Redirection will automatically provide it with the 32-bit version of the application. This redirection in fact affects the function GetFileArchitecture():

static arch GetFileArchitecture(const TCHAR* szFileName)
{
    auto retval = notfound;
    auto hFile = CreateFile(szFileName, GENERIC_READ, FILE_SHARE_READ, nullptr, OPEN_EXISTING, 0, nullptr);
    if(hFile != INVALID_HANDLE_VALUE)
    {
        unsigned char data[0x1000];
        DWORD read = 0;
        auto fileSize = GetFileSize(hFile, nullptr);
        auto readSize = DWORD(sizeof(data));
        if(readSize > fileSize)
            readSize = fileSize;
        if(ReadFile(hFile, data, readSize, &read, nullptr))
        {
            retval = invalid;
            auto pdh = PIMAGE_DOS_HEADER(data);
            if(pdh->e_magic == IMAGE_DOS_SIGNATURE && size_t(pdh->e_lfanew) < readSize)
            {
                auto pnth = PIMAGE_NT_HEADERS(data + pdh->e_lfanew);
                if(pnth->Signature == IMAGE_NT_SIGNATURE)
                {
                    if(pnth->FileHeader.Machine == IMAGE_FILE_MACHINE_I386)  //x32
                        retval = x32;
                    else if(pnth->FileHeader.Machine == IMAGE_FILE_MACHINE_AMD64)  //x64
                        retval = x64;
                }
            }
        }
        CloseHandle(hFile);
    }
    return retval;
}

The call to CreateFile will invoke the FS Redirector and if the file requested is a system resource and is a 64-bit application with an equivalent 32-bit version, Windows will return the 32-bit version of that application. So debugging notepad.exe in %windir%\system32 will become debugging notepad.exe in %windir%\SysWOW64 which is not what was intended.

The Fix

Luckily for us, Microsoft provides an easy way to bypass the default behavior of File System Redirection. The fix applied to the x96dbg.exe loader is one function that determines whether FS Redirection is supported, and a structure that facilitates disabling this and re-enabling it once done. Two conditions need to be met before we can determine whether FS Redirection can be disabled:

1. Are we running under WoW64 context? (Meaning is this a 32bit application running under 64-bit Windows) and
2. Does this OS support File System Redirection?

Checking for FS Redirection is a simple matter of seeing if the pertinent functions are available:

static BOOL isWowRedirectionSupported()
{

    BOOL bRedirectSupported = FALSE;

    _Wow64DisableRedirection = (LPFN_Wow64DisableWow64FsRedirection)GetProcAddress(GetModuleHandle(TEXT("kernel32")), "Wow64DisableWow64FsRedirection");
    _Wow64RevertRedirection = (LPFN_Wow64RevertWow64FsRedirection)GetProcAddress(GetModuleHandle(TEXT("kernel32")), "Wow64RevertWow64FsRedirection");

    if(!_Wow64DisableRedirection || !_Wow64RevertRedirection)
        return bRedirectSupported;
    else
        return !bRedirectSupported;

}

The structure that encapsulates the calls to these functions is defined like so:

struct RedirectWow
{
    PVOID oldValue = NULL;
    RedirectWow() {}
    bool DisableRedirect()
    {
        if(!_Wow64DisableRedirection(&oldValue))
        {
            return false;
        }
        return true;
    }
    ~RedirectWow()
    {
        if(oldValue != NULL)
        {
            if(!_Wow64RevertRedirection(oldValue))
                MessageBox(nullptr, TEXT("Error in Reverting Redirection"), TEXT("Error"), MB_OK | MB_ICONERROR);
        }
    }
};

Once we’ve determined that our conditions for FS redirection are met, we can disable it using a simple call to the member function DisableRedirect(). This must be invoked before we attempt to determine the files architecture and this can be seen here at Line #412

With these changes, x96dbg.exe can now properly allow a user to Debug redirected 64-bit applications as intended.

References

More reading material on WoW64 can be found from the resources below:

• WoW64 Implementation Details
• File System Redirector
• Wow64DisableWow64FsRedirection function
• Wow64RevertWow64FsRedirection function

User interface design principles

I just committed branch destination preview tooltip recently. When the project leader, Duncan, comes to me to express the users’ appreciation, I feel a bit surprised. I just feel it will be needed by our users, and it indeed is. A good user interface design is a key to x64dbg’s success. As users are demanding better user interface design, it is useful to share my ideas about what is a good user interface, and how to implement it.

There is significant difference between GUI design for x64dbg and other software. Imagine, what GUI do you wish to see on a software? You might recall your memory with IOS/Android, Google/Facebook/Twitter and other favourite software. They typically feature brief design, colorful text and buttons, large buttons and images to attract users. But come back to x64dbg, things are a bit different. Most issues on Github on GUI design are not about ugly buttons and images, but rather about accessing features and data from GUI. If you dislike the dull interface lacking images and color blending, you may still bear it. However when the function is not in the context menu when you need it most, you will probably feel very disappointed. The user expectation for x64dbg is different. They need a really powerful GUI, instead of a beautiful but feature-lacking GUI. So I usually favor feature improvement tasks over visual improvement.

In my opinion, a good graphical user interface has the following features:

Access any feature, anywhere

A feature without interface in the GUI is not complete. It is very disappointing to find that the most needed feature is inaccessible from the current user interface. To fit the users’ demand, we have to add various commands to the context menus everywhere where they are applicable. That’s a great amount of work. Disassembly and Dump views have a long list of context menu items, but the same cannot be said on other views. And when we add some new feature, for example, watch view, we have to add that feature to all the relevant menus. Failing to do so results in increased inconvenience.

To add feature in context menu more easily, we have written some very useful utility components. But since we want every feature to appear everywhere where it is applicable, it is still not enough. It will be a good idea to group the features together, so they can be added in groups, to a context menu. For example, we can group all address-related functions, such as breakpoint, dump, watch, etc, together in a package, so we can add all these features in the menu with little code, and also ensure no feature is missing from the context menu.

Offer to show the most needed data to user

We are always complaining the screen being too small to show the entire program and data. Protected programs tend to access non-local data and execute large routines. It is often the case that only a small portion of the screen is displaying useful things, and most of the program’s important states, are hidden in the remote part of memory not shown on the screen. To enlarge the sight of the user, I have introduced many features that can display a non-continuous range of memory, such as watch view, code folding, and recently introduced branch destination previewing. When used properly, they can help concentrate useful information together on the same screen, despite being separated by a large gap in address.

However, there is another thing that makes branch destination preview more success. It displays on mouse over, not on a context menu event. In this way, the tooltip will automatically appear as soon as the user is interested on a particular call instruction. By offering to show the most needed data to user, we save a lot of mouse clicks and keyboard actions. Without branch destination preview, the user would press “Enter”, and have a glimpse at the disassembly, then press “-“ or “*” to go back to the current location. It is far more complicated and less convenient than mouse over. Also, I provided an option to disable it.

By contrast, code folding requires the user to select a range and then click on the checkbox. Code folding has cost me more effort to implement than branch destination preview, but it is less useful. I don’t know how often you fold a section of disassembly, but I think many of you will seldom use it. Duncan added an ability to fold the entire function more easily, but I think that is very misleading (see the comment on #829) and misses the principle 3. That can probably lead to a further decreased usage of code folding. The proper usage of code folding is in a large loop with lots of unexecuted branches. By folding these inactive branches, you can display more active instructions on screen, thus gain a better overview of the operation of the large loop.

Of course, to implement these features, we have to make lots of non-standard controls. That requires quite an effort. But for better user satisfaction, we are continuously working.

Guide the user to do the right thing

x64dbg has so many features that sometimes the user might not know which action to take. A good user interface should make the most frequent and useful actions very easy, and make the less used features still accessible. Take exception handling debug commands for example. By gaining experience from Ollydbg, we know it is really time to change something. Skipping first-chance exceptions is almost always the wrong practice in reverse engineering. So Duncan made it the default behaviour to pass these exceptions to the debuggee. In this way, in case an exception is thrown by the debuggee and you don’t know what to do, you can just step and see what’s happening. But to advanced users, we also make it possible to skip first-chance exceptions.

Easy to understand and master

Graphical user interface is the reason for which many users choose x64dbg. GDB and radare are great, but they are less popular than Ollydbg and IDA Pro due to lack of graphical user interface. Good GUI design will greatly reduce the time to learn and master a piece of software for the user, and also increase the comfort of using the software daily. To improve the usability of the GUI of x64dbg, we have hundreds of different icons for the menu items, a rich color design scheme to help the user understand the debuggee better.

A good help system is also important. Although the current help system is better than before, we still receive user complaints about not knowing how to accomplish a certain goal. We need to improve the documentation by adding more examples, as well as adding more tips directly in the GUI to help the user use the software more correctly.

Dissatisfied with English user interface, I added internationalization support for x64dbg and first translated x64dbg to Chinese. The program has since been translated to many languages worldwide, helping its users get the native experience. You are always welcome to contribute to the translations of x64dbg at Crowdin.

User interface customization is important

The default layout, keyboard and color scheme may not suit everyone’s needs. A customizable workspace is better than the best fixed workspace. x64dbg has detachable views and resizable splitters, but if we can introduce a draggable panel, that would be even better.

To accelerate the most common operations, we have customizable key shortcuts for most actions. I contributed the favourites menu to allow further customization. It can bind a shortcut to custom tools and any single command, greatly improves production when some custom actions need to perform repeatedly.

Fast and responsive

Performance problems often occur in custom graphical user interface designs. We have spent effort on optimizing the performance with drawing, but the CPU usage is still high on the disassembly view. A non-responsive user interface gives the users very bad experience no matter how beautiful it is. We will continue improving the performance of drawing to make it responsive on most computer hardware.

Afterword

The continuous improvement of x64dbg relies on voluntary work of the community. We sincerely welcome you to contribute to x64dbg. Contributing is very easy. There are many easy issues on the issue list. Within a few hours, you will be able to help the users of x64dbg worldwide and have your name on the contributors list. Please check contribute.x64dbg.com for more information.