VMProtect unpacking vs devirtualization

Unpacking and devirtualization at a glance

Which one a sample needs is decided by how it was protected. Packing and virtualization are separate options inside VMProtect, and they leave separate problems behind.

The last row decides an engagement. An unpacker cannot produce instructions that were never stored as instructions.

What unpacking solves

Packing is a storage transformation. VMProtect's user manual defines packing as protecting program code by compressing the executable file or its libraries with non-typical algorithms, and states that the compressed fragments are unpacked completely or partially on the user's side when the application is executed.

Because the program itself must restore that code before it can run, the original instructions do exist in native form at some point. Unpacking is the analysis step that recovers them for inspection instead of leaving them as compressed data. For an analyst that is the difference between a file whose sections, strings, resources and imports can be read with ordinary tools, and a file whose contents look like noise.

Packing also changes what detection sees. MITRE ATT&CK notes that packing an executable changes its file signature in an attempt to avoid signature-based detection, which is one reason protected files draw attention regardless of what they do.

Packing sits alongside VMProtect's other loader-level options, such as import protection, memory protection, resource protection, debugger detection and detection of virtual environments. None of them translates code into another form.

What unpacking does not solve

Virtualization is not storage, so there is nothing for an unpacker to restore. The vendor's glossary defines virtualization as transforming part of an application's executable code into commands of a virtual machine whose command system, architecture and operational logic are unknown to an attacker, and states that virtualized fragments are executed by the virtual machine's interpreter without being transformed into machine language code of the physical processor.

That last clause is the whole answer. A virtualized function does not return to native instructions at any point, so an unpacker that did its job perfectly still leaves that function as an interpreter plus a block of bytecode.

Five jobs get confused with one another. Naming them keeps an engagement honest:

• Unpacking. Restores compressed or encrypted file contents so the image can be loaded and read.
• Import rebuilding. Restores the table of external functions the binary calls, so those calls read as named APIs rather than indirect jumps.
• Deobfuscation. Undoes transformations that left the code native but hard to read, such as substituted instructions, junk instructions and dead code.
• Virtual machine handler analysis. Works out what each handler of the embedded interpreter does, which describes the machinery rather than the protected function.
• Semantic devirtualization. Recovers what the virtualized function computes, and supplies evidence that the recovered version behaves like the original.

For a virtualized function, semantic devirtualization recovers the protected logic. Packed-only or mutated native functions may become understandable after unpacking or deobfuscation. The three jobs that are most often confused are compared on their own terms in devirtualization vs deobfuscation vs unpacking.

What virtualization changes

VMProtect compiles selected functions into bytecode for a virtual machine embedded in the binary. Its glossary defines bytecode as the code obtained after transcoding commands of the real processor into commands of the virtual machine, and the virtual machine as program code that executes that bytecode directly inside the protected application.

After that transformation a disassembler is accurate and unhelpful at the same time. It shows the interpreter's dispatch and handlers, which are real native code, and it shows the bytecode as data. Neither of them is the function that was protected.

Mutation is a different option again: the manual defines it as replacing an original command with an analogue, or with a set of commands producing the same result, which leaves the code native but harder to read. Ultra mode applies mutation and then virtualization, so the logic is obfuscated before it is translated.

MITRE ATT&CK keeps the same separation inside a single technique: software packing compresses or encrypts an executable, while virtual machine software protection translates an executable's original code into a special format that only a special virtual machine can run. Two behaviors, two different analysis problems.

When devirtualization is required

Devirtualization is required whenever the functions you need to understand were virtualized, rather than only packed or mutated. That is a property of the sample, not a preference. Three cases cover it:

• The functions you need were only packed, and recovering the loadable image is enough.
• They were mutated but left native, so the remaining work is deobfuscation and patient reading.
• They were virtualized, and only recovering their semantics answers a question about what they compute.

The vendor describes the same problem from the other side. Reverse engineering virtualized fragments, its manual says, comes down to building a disassembler with the same architecture as the processor the virtual machine imitates, and then analyzing the disassembled result. That is the work devirtualization does, and it is why it is a separate service rather than a better unpacker.

It also means the honest answer is sometimes neither. If the functions carrying the logic you need were never protected, no recovery stage is required at all. Identifying the protection and classifying the functions that matter therefore come first, which is the subject of the protected binary analysis workflow.

What each result gives you

The two results answer different questions, and they look different when they arrive.

Unpacking alone produces a loadable image. Strings and resources become readable, native functions appear as native code, and each virtualized function appears as a call into interpreter code with its bytecode nearby as data. The file is easier to work with and the protected logic is exactly as unavailable as before.

Devirtualization produces recovered semantics for those functions. Devirt accepts VMProtect 2.x and 3.x, and delivers named imports and typed API calls together with a function-level coverage ledger that records the state of every unit it counted.

A readable result is not automatically a proven one. Each unit in the ledger is proven, runtime-dependent, unresolved or permanently bounded, and the difference between those four states decides what a finding can support. That distinction is set out in coverage is a contract.

Authorized-analysis use cases

Every use below assumes you own the submitted binary, or hold authorization to analyze it, and that the results are used lawfully. That is a condition of the service rather than a formality. The boundaries are in the acceptable use policy.

Malware research is the clearest case. MITRE ATT&CK records VMProtect in the hands of tracked groups and families: APT38 packed implants with Themida, Enigma, VMProtect and Obsidium, and in one campaign APT41 used VMProtect to slow the reverse engineering of malicious binaries. An analyst who has to explain what such a sample does cannot stop at a loadable file.

The other cases are commercial. A security audit of software you are authorized to assess. Interoperability or maintenance work on software your organization owns, or licenses with a right to analyze. Recovery of software you own whose source code is gone.

In all of them the deliverable is an explanation that survives review, which is why the coverage record matters as much as the recovered code.

Is your VMProtect sample in scope?

Devirt accepts VMProtect 2.x and 3.x samples, and scope is confirmed per sample. The practical next step is to describe the file and the functions you need, rather than to guess which stage applies.

Background on how VMProtect protects code is in VMProtect research, and the recovery methods themselves are in devirtualization methods.