Reverse Engineering the Fictional Product Licensing System: A Comprehensive Journey

#

Introduction

#

In this comprehensive blog post, I document my extensive reverse engineering journey into the Fictional Product licensing system. What began as a curiosity about how commercial software licensing works evolved into a multi-week deep dive into cryptographic primitives, binary analysis, and the intricate architecture of modern license validation systems. This project ultimately resulted in a complete understanding of the license file format, cryptographic pipeline, and the successful generation of valid license files for all four Fictional Product components.

The journey was neither straightforward nor easy. Along the way, I encountered numerous hurdles—incorrect initial hypotheses, misaligned field formats, cryptographic misunderstandings, and frustrating debugging sessions. Yet each obstacle became a learning opportunity, and each failure brought me closer to the truth. In this post, I detail every significant discovery, every mistake made, every lesson learned, and ultimately present the complete scheme that was uncovered.

This analysis covers the Fictional Product Fictional Product version 1.3.1390.0, a 16MB x86-64 PIE (Position Independent Executable) ELF binary. The target was the licensing validation component, which protects the software from unauthorized usage. The goal was to understand how the licensing system works end-to-end and to be able to generate valid license files.

Executive Summary

#

Before diving into the detailed narrative, here is a high-level summary of what was accomplished and what was discovered:

What Was Achieved:

• Complete reverse engineering of the Fictional Product licensing system
• Understanding of the asymmetric cryptographic architecture
• Successful binary patching to replace the embedded RSA public key
• Development of a working license file generator
• Generation of valid license files for all four product components
• Successful execution of the patched binary with zero license errors

What Was Discovered:

• The license file uses a custom binary format with “WTF*” magic number
• RSA key transport is used for AES key exchange (not hardware-key-derived)
• 20 license fields are required (10 core + 10 device ID)
• The field format is type:key:value, not the more common key:type:value
• Four product types are supported: 102 (Access), 103 (Gateway), 104 (Advanced), 105 (Client)
• Error codes VE1-VE9 provide detailed diagnostic information about validation failures

The Harness: Creating an Isolated Analysis Environment

#

Before diving into the reverse engineering details, it’s important to understand the harness setup—a critical component of this project that made the analysis possible. The harness was designed to create a controlled, isolated environment for running the target binary, allowing us to intercept and manipulate the system’s view of the world.

Why a Harness Was Necessary

#

The target binary doesn’t run in isolation—it reads numerous files from the host system to perform hardware fingerprinting and license validation. The most critical of these is the machine-ID, which the binary uses to generate its hardware key for machine binding. The license system reads:

• /etc/machine-id or /var/lib/dbus/machine-id — The primary machine identifier (32 hex characters)
• /sys/devices/virtual/dmi/id/product_uuid or /sys/class/dmi/id/product_uuid — SMBIOS UUID
• Network interface MAC addresses
• CPU information
• Hostname

In a normal production environment, these values would be unique to each machine. For reverse engineering purposes, we needed control over these values to:

1. Ensure consistent behavior across test runs
2. Understand exactly what values the binary was reading
3. Replace sensitive or variable data with known values

Building the Harness

#

The harness was constructed as a minimal chroot environment with exactly the files and directories the binary needed:

harness/
├── target              # The main binary (16MB, x86-64, PIE ELF)
├── etc/
│   ├── machine-id     # Contains: 00000000000000000000000000000000
│   └── [other minimal etc files]
├── var/
│   └── lib/
│       └── product/
│           └── config/
│               ├── productaccess.lmao
│               ├── productgateway.lmao
│               ├── productsecurity.lmao
│               └── productclient.lmao
├── bin/
│   └── sh -> bash    # Shell for chroot execution
└── usr/
    └── [minimal library structure]

The Machine-ID Replacement

#

The most critical file in the harness is /etc/machine-id. In a real Linux system, this file contains a 32-character hexadecimal identifier unique to that installation. For our harness, I created a file containing all zeros:

00000000000000000000000000000000

This “null” machine-ID serves several purposes:

• Predictability: We know exactly what hardware key will be generated
• Reproducibility: Any analysis or test can be repeated with identical results
• Simplicity: No need to extract or generate a real machine-ID
• Security: The zero-value machine-ID doesn’t match any real system

The binary computes the hardware key as:

machine_id = open("/etc/machine-id").read().strip().upper()  # "00000000000000000000000000000000"
hardware_key = hashlib.sha256(machine_id.encode()).hexdigest()  # 33e3c3eafaa33a33c333f333ac33e33ce333bab3e333a33e3333a33ae333b33a

This produces a deterministic hardware key: 33e3c3eafaa33a33c333f333ac33e33ce333bab3e333a33e3333a33ae333b33a

Running the Binary in the Harness

#

The binary is executed within the harness using chroot to create an isolated filesystem view:

sudo env SHELL=/bin/bash chroot harness /usr/bin/env LANG=C LC_ALL=C \
    OPENSSL_CONF=/etc/ssl/openssl-legacy.cnf SHELL=/usr/bin/sh \
    /target -l debug

Key aspects of this command:

• chroot harness: Changes the root filesystem to the harness directory, isolating the binary from the real system
• LANG=C LC_ALL=C: Sets locale to C to avoid internationalization issues
• OPENSSL_CONF=/etc/ssl/openssl-legacy.cnf: Uses legacy OpenSSL configuration for compatibility with older cryptographic algorithms the binary was built against
• -l debug: Starts the binary in debug/verbose mode to see license loading messages

For remote debugging, gdbserver is used:

sudo env SHELL=/bin/bash chroot harness /usr/bin/env LANG=C LC_ALL=C \
    OPENSSL_CONF=/etc/ssl/openssl-legacy.cnf SHELL=/usr/bin/sh \
    /usr/bin/gdbserver 0.0.0.0:23456 /target -l debug

This allows Ghidra’s debugger to attach to the running process for dynamic analysis.

Benefits of the Harness Approach

#

The harness provided several significant advantages:

1. Controlled Environment: The binary can only see files we explicitly include in the harness directory structure
2. Reproducible Results: Every run produces identical results since the machine-ID and other files are constant
3. Safe Experimentation: We can modify files, patch binaries, and try different approaches without affecting the host system
4. Isolation: If something goes wrong (infinite loop, crash, etc.), the host system remains unaffected

This harness setup proved essential throughout the project. Every test, every license generation, and every debugging session ran within this controlled environment.

Phase 1: Initial Reconnaissance and Environment Setup

#

Understanding the Target

#

Before any meaningful analysis could begin, I needed to understand what I was working with. The target binary was placed into a “harness” directory. Dummy license files were generated, basing the filenames on the output from the target running with DEBUG loglevel.

harness/
├── target              # The main binary (patched)
├── target.orig         # Original unpatched binary backup
├── target.patched      # Copy of patched binary
├── etc/machine-id      # Contains: 00000000000000000000000000000000
├── bin/sh              # Symlink to bash
└── var/lib/product/config/
    ├── productaccess.lmao
    ├── productgateway.lmao
    ├── productsecurity.lmao
    └── productclient.lmao

The machine-id file was prepared with 00000000000000000000000000000000, which would become crucial later in understanding hardware binding. The presence of four separate license files (.lmao extension) suggested that the product suite consisted of four distinct components, each with its own licensing requirements.

Upon initial inspection of the target binary using the file command, I discovered:

• ELF 64-bit LSB executable, x86-64
• Dynamically linked (not statically compiled)
• Stripped of debugging symbols
• Position Independent Executable (PIE)
• Build ID present

The binary being stripped meant that function names would not be available during analysis, requiring me to rely on Ghidra’s function naming conventions (FUN_xxxxxx) or to assign meaningful names based on code analysis.

Setting Up the Analysis Environment

#

The primary analysis tool for this project was Ghidra, a powerful open-source reverse engineering framework developed by the NSA’s Research Directorate. I configured Ghidra with two MCP (Model Context Protocol) servers:

• CodeBrowser Server: Running on port 8080, used for static analysis, function listing, string searching, and cross-referencing
• Debugger Server: Running on port 9090, used for dynamic analysis, setting breakpoints, and examining runtime behavior

The initial setup proved challenging. The MCP bridge required careful configuration, and there were connectivity issues in the early sessions. At one point, I spent significant time troubleshooting why the Ghidra MCP tools were unavailable—only to discover the bridge wasn’t properly configured on the client side.

The debugging sessions required running the target binary in a chroot environment with specific environment variables:

sudo env SHELL=/bin/bash chroot harness /usr/bin/env LANG=C LC_ALL=C \
    OPENSSL_CONF=/etc/ssl/openssl-legacy.cnf SHELL=/usr/bin/sh \
    /usr/bin/gdbserver 0.0.0.0:23456 /target -l debug

The OPENSSL_CONF=/etc/ssl/openssl-legacy.cnf setting was particularly important because the binary was built against an older version of OpenSSL that used legacy cryptographic algorithms by default.

First Glimpses: Strings and Error Messages

#

One of the first analysis techniques I employed was string extraction. Even in stripped binaries, embedded strings often reveal significant information about functionality. I searched for license-related strings and found:

• “WTF*” - Appeared to be a file magic number
• “VE1”, “VE2”, “VE3” - Error codes related to license validation
• Product names: “Product Access”, “Product Gateway”, “Product Advanced Security”, “Product Client”
• “Evaluation” - Suggests evaluation license support
• Various error messages related to license parsing
• “/etc/machine-id” - Hardware identification mechanism
• “/var/lib/dbus/machine-id” - Fallback machine ID location
• “/sys/devices/virtual/dmi/id/product_uuid” - SMBIOS UUID location

These strings provided initial breadcrumbs that would guide the deeper analysis. The presence of multiple machine identification paths suggested the system used hardware fingerprinting for machine binding.

Understanding the Binary Structure

#

Through preliminary Ghidra analysis, I identified key structural elements of the binary:

Entry Point and Initialization:

• The binary has a standard ELF entry point that quickly branches to main
• Several initialization functions run before main (INIT functions)
• One such function (_INIT_223) was particularly important as it initialized the signature delimiter string

Key Function Locations:

• License-related functions clustered around addresses 0x00390000-0x00392000 and 0x00780000-0x007a0000
• Cryptographic operations concentrated in the 0x00780000-0x007c0000 range
• The binarypacker namespace functions were found at 0x007ac8a0

Memory Layout:

• Code section (.text) spans a significant portion of the 16MB binary
• Read-only data (.rodata) contains embedded strings and the RSA public key
• Data section (.data) contains runtime-initialized variables
• BSS section (.bss) contains uninitialized global variables

Phase 2: Understanding the License File Format

#

The First Breakthrough: Identifying the Magic Number

#

My initial analysis focused on understanding the .lmao file format. Through a combination of Ghidra analysis and trial-and-error with generated files, I discovered the file header structure:

12-byte header:
  [0-3]   Magic: 0xDEADBEEF = "WTF*" (little-endian)
  [4-7]   Version: must be <= 1
  [8-11]  Product Type ID (102-105)

The magic number “WTF*” (0xDEADBEEF in little-endian) was consistent across all four license files and appeared prominently in the binary’s parsing code. When I searched for this magic in the binary, I found it at address 0x007ac8a0, which was the entry point for the binarypacker::unpack function.

This was a crucial first discovery because it told me exactly where to look in the binary for the license parsing logic. From there, I could trace the code flow to understand the complete file format.

Key Discovery: Field Format Confusion

#

One of the earliest and most significant hurdles involved the field format within the license data. Initially, I assumed the format was key:type:value (similar to many configuration file formats). This assumption led to numerous failures.

Through careful decompilation of the parseLicenseComponents function at address 0x007877d0, I discovered the actual format was the reverse: type:key:value. The function used getline(stream, str, ':') three times:

1. First getline → TYPE code (e.g., b, i, d, s, ab, ai, ad, as)
2. Second getline → KEY name (e.g., product.id, evaluation)
3. Third getline (delimiter '\n') → VALUE

This seemingly small detail caused the VE8 error (“unknown type code”) because when the parser tried to interpret “product” as a type code, it failed since “product” wasn’t a valid type.

The confusion arose because I was thinking of license files in terms of common configuration formats like JSON, YAML, or INI files. However, the Fictional Product developers chose a more compact format optimized for parsing efficiency. By putting the type first, the parser can immediately dispatch to the correct field handler without needing to look up the key first.

The Type Code System

#

The license system supports eight distinct type codes:

Each type code corresponds to a specific C++ type in the underlying implementation:

• b maps to a boolean field handler
• i maps to a 32-bit integer field handler
• d maps to a Boost date_time ptime field handler
• s maps to a std::string field handler
• The array types (ab, ai, ad, as) map to STL vector containers

The Delimiter Discovery

#

Another critical discovery was the delimiter used to separate license data from its cryptographic signature. The binary initializes a delimiter string at runtime via _INIT_223, which stores “s:rocksolid:” at BSS address 0x0100f2e0. This delimiter appears inside the AES-decrypted plaintext, separating the actual license fields from the base64-encoded RSA signature.

The complete plaintext structure became:

<type:key:value\n lines>s:rocksolid:<base64_rsa_sha256_signature>

This delimiter was not present as a string constant in the binary’s read-only data section—instead, it was constructed at runtime, which initially made it difficult to locate. The use of runtime initialization suggests this might have been an attempt to obfuscate or protect this sensitive string, though it was ultimately discoverable through careful analysis.

Understanding the Header Structure Corrections

#

Early in my analysis, I made incorrect assumptions about the header structure. I initially believed the header was 24 bytes, contained a separate IV field, and had a signature field at the end. All of these assumptions were wrong.

What I Got Wrong:

• Header size: I thought it was 24 bytes; it’s actually 12 bytes
• IV field: I thought there was a separate IV in the file; the IV is inside the RSA blob
• Signature field: I thought there was a signature at EOF; the signature is inside the AES payload

The Correct Structure:

• 12-byte header: 4 bytes magic + 4 bytes version + 4 bytes product type
• No separate IV or signature fields—all cryptographic material is in encrypted blobs

These corrections came through iterative testing and careful examination of the binarypacker::unpack function’s read operations. Each time I generated a license file that failed, I would analyze the error and adjust my understanding accordingly.

Phase 3: Cryptographic Analysis

#

Initial Hypothesis: Hardware-Key-Derived AES

#

My first cryptographic hypothesis was that the AES encryption key was derived from the machine’s hardware key. The reasoning seemed sound:

1. The hardware key (SHA256 of machine-id) produces 64 hex characters
2. When converted to bytes, this is exactly 32 bytes—perfect for AES-256
3. This would provide machine-binding without storing any secrets
4. The hardware key algorithm was clearly implemented in the binary (I could see the SHA256 hash being computed)

I developed scripts based on this hypothesis and tested them extensively. The generator aes_key_hypothesis.py and early versions of the license generator were built on this assumption. The hypothesis felt right because it provided a neat explanation for how machine binding worked.

The Problem: Wrong Architecture

#

This hypothesis was completely wrong. The crypto pipeline was entirely different. The early analysis had correctly identified the hardware key algorithm:

import hashlib

machine_id = open("/etc/machine-id").read().strip().upper()  # 32 hex chars
hardware_key = hashlib.sha256(machine_id.encode()).hexdigest()  # 64 hex chars

However, this hardware key was NOT used as the AES encryption key. Instead, it served a different purpose entirely—one that would take much longer to fully understand.

Looking back, there were clues I missed:

• The key map stored RSAPublicKey objects, not raw AES keys
• The EVP_PKEY_get_size() call returned 512 (RSA key size), not 32 (AES key size)
• No key derivation functions (KDF) were found in the critical path

But at the time, these details didn’t register as contradictions to my hypothesis. I was seeing what I expected to see rather than what was actually there.

The Correct Architecture: RSA Key Transport

#

The actual cryptographic architecture uses asymmetric RSA key transport, a standard approach in secure licensing systems:

1. Random AES Key Generation: Each license file contains a randomly generated 32-byte AES-256 key and 16-byte IV (48 bytes total)
2. RSA Encryption of Key Material: These 48 bytes are encrypted using RSA private key encryption with PKCS#1 v1.5 padding
3. Binary’s RSA “Decryption”: The binary uses its embedded public key to “decrypt” this blob—which is mathematically equivalent to verifying a signature
4. AES Decryption: The recovered 48 bytes are split into AES key (32B) and IV (16B)
5. Payload Decryption: The license data is AES-256-CBC decrypted
6. Signature Verification: An RSA+SHA256 signature over the plaintext data is verified

This architecture is elegant because:

• No secret key material needs to be embedded in the binary (only a public key)
• Each license has its own unique AES key
• The license issuer needs only the private key to generate licenses
• The public key in the binary cannot be used to create new licenses

RSA Key Details

#

The embedded RSA public key provided crucial information:

• Key Size: 4096-bit RSA
• Key Format: 550 bytes DER (Distinguished Encoding Rules)
• File Offset: 0x00d02080
• Key ID: 102 (0x66) — the only key in the binary’s red-black tree map

The binary stores this key in an RSAPublicKey object with the following structure:

+0x00: vtable ptr (Fictional::Licensing::RSAPublicKey)
+0x08: key_id (uint32 = 102/0x66)
+0x18: EVP_PKEY* (OpenSSL, type=6/RSA)
+0x30: string "YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXo=" (32 chars)
+0x78: string length = 13
+0x80: "Foobarbaz 1" (SSO inline string)

The object uses C++ small string optimization (SSO) for the “Foobarbaz 1” string, storing it inline within the object rather than on the heap.

Key Functions in the Crypto Pipeline

#

Several critical functions handle the cryptographic operations:

Deep Dive: The RSA Blob Operation

#

The RSA “decryption” of the AES key blob is perhaps the most confusing aspect of this system. Let me explain it in detail:

When the binary processes the license file, it reads a 512-byte blob (for 4096-bit RSA). This is mathematically equivalent to having signed the 48 bytes with PKCS#1 v1.5 padding, but without any hash algorithm. It’s a raw RSA exponentiation.

When the binary processes this blob, it calls EVP_PKEY_decrypt() with the public key. This performs the inverse operation—raw RSA exponentiation with the public exponent. The result is the original 48 bytes of AES material.

This is why I say the operation is “effectively signature verification”—because mathematically, a raw RSA private-key operation followed by a public-key operation just recovers the original data. It’s the same mathematical relationship as signing and verifying, just without using a hash algorithm.

Phase 4: Binary Patching and Key Replacement

#

The Problem: No Matching Private Key

#

The binary contained an embedded RSA public key, but I didn’t have the corresponding private key. Without the private key, I couldn’t:

1. Generate the RSA-encrypted AES blob
2. Sign the license data

This presented a significant challenge. I had two options:

1. Find the original private key (unlikely to be available)
2. Replace the embedded public key with my own

Solution: Binary Patching

#

I chose option 2. The approach was straightforward in concept but required careful execution:

1. Generate a new 4096-bit RSA key pair:

   openssl genrsa -out /tmp/our_private.pem 4096
   openssl rsa -in /tmp/our_private.pem -pubout -out /tmp/our_pubkey.der -outform DER

2. Verify the new public key matches the required size (550 bytes for 4096-bit RSA)

3. Backup the original binary:

   cp harness/target harness/target.orig

4. Patch the binary at file offset 0x00d02080:

   • This was the location of the embedded RSA public key
   • Overwrite with my public key DER

The patching worked perfectly. The binary now loaded my public key, and I could use my private key to generate licenses that would pass verification.

Key Generation and Verification

#

The key generation process required ensuring the new public key was exactly 550 bytes to match the original. I verified this:

with open('/tmp/our_pubkey.der', 'rb') as f:
    pubkey_data = f.read()
    print(f"Public key size: {len(pubkey_data)} bytes")
    assert len(pubkey_data) == 550, "Public key must be exactly 550 bytes"

The original embedded key was extracted for reference:

# Extract original public key from binary
dd if=harness/target of=/tmp/rsa_pubkey.der bs=1 skip=$((0xe02080)) count=550
openssl rsa -in /tmp/rsa_pubkey.der -pubout -out /tmp/rsa_pubkey.pem -inform DER

Important Security Note

#

This patching approach is only possible because the binary contains a public key, not a private key. In a properly designed licensing system, only the public key would ever be distributed with the software. The corresponding private key would be held securely by the license issuer. This is exactly what Fictional had done—they just made the mistake of not implementing verification of the included public key.

The fact that I was able to patch the binary highlights an important security principle: the security of asymmetric licensing systems depends entirely on keeping the private key secret. Once someone can replace the public key, they can become their own license issuer.

An Elegant Alternative: The LD_PRELOAD Approach

#

While binary patching worked, it had some drawbacks:

• The binary was modified on disk (albeit backed up)
• The patch location was specific to this binary version
• Any binary updates would require re-patching

A more elegant solution emerged: instead of patching the binary file directly, we use an LD_PRELOAD shared library to intercept the OpenSSL function that loads the public key and replace it at runtime. This approach is cleaner and keeps the original binary completely unmodified.

How It Works

#

The target binary uses OpenSSL’s d2i_PUBKEY() function to load its embedded RSA public key. This function parses DER-encoded public key data and returns an EVP_PKEY pointer. We created a shared library that:

1. Intercepts calls to d2i_PUBKEY() using the LD_PRELOAD mechanism
2. Detects when the binary is loading its embedded key by checking for a unique 8-byte fingerprint in the key’s modulus
3. Substitutes our own public key DER in place of the original
4. Returns the modified EVP_PKEY to the calling code

The key fingerprint we look for is at DER offset 33 (the start of the RSA modulus):

0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88

This allows us to distinguish the license key from other RSA keys the binary might load (such as TLS certificates).

The LD_PRELOAD Library (preload_pubkey.c)

#

/*
 * preload_pubkey.c - LD_PRELOAD library to replace the embedded RSA public key
 *
 * Hooks d2i_PUBKEY() from OpenSSL. When the binary loads its embedded 4096-bit
 * RSA public key (550 bytes DER at static address), we detect it by checking
 * the modulus fingerprint and substitute our own key.
 *
 * Build:
 *   gcc -shared -fPIC -o preload_pubkey.so preload_pubkey.c -ldl
 *
 * Usage:
 *   LD_PRELOAD=/preload_pubkey.so /target -l debug
 */

#define _GNU_SOURCE
#include <dlfcn.h>
#include <string.h>
#include <stdio.h>

typedef void EVP_PKEY;
typedef EVP_PKEY *(*d2i_PUBKEY_fn)(EVP_PKEY **a, const unsigned char **pp, long length);

/* Our replacement 4096-bit RSA public key (550 bytes DER) */
static const unsigned char our_pubkey_der[550] = { /* ... key data ... */ };

/* Fingerprint from the original embedded key's modulus (offset 33) */
static const unsigned char orig_fingerprint[8] = {
    0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88
};

#define DER_SIZE       550
#define FINGERPRINT_OFF 33

EVP_PKEY *d2i_PUBKEY(EVP_PKEY **a, const unsigned char **pp, long length)
{
    static d2i_PUBKEY_fn real_d2i_PUBKEY = NULL;

    if (!real_d2i_PUBKEY) {
        real_d2i_PUBKEY = (d2i_PUBKEY_fn)dlsym(RTLD_NEXT, "d2i_PUBKEY");
    }

    /* Check if this call is loading the embedded license pubkey */
    if (length == DER_SIZE && pp && *pp &&
        memcmp(*pp + FINGERPRINT_OFF, orig_fingerprint, sizeof(orig_fingerprint)) == 0)
    {
        fprintf(stderr, "[preload] intercepted d2i_PUBKEY — replacing embedded key\n");

        /* Point to our replacement DER instead */
        const unsigned char *our_ptr = our_pubkey_der;
        EVP_PKEY *result = real_d2i_PUBKEY(a, &our_ptr, DER_SIZE);

        /* Advance the caller's pointer so bookkeeping stays consistent */
        *pp += length;

        return result;
    }

    /* Not our target key — pass through unchanged */
    return real_d2i_PUBKEY(a, pp, length);
}

Building and Using the LD_PRELOAD Library

#

# Build the shared library
gcc -shared -fPIC -o preload_pubkey.so preload_pubkey.c -ldl

# Copy to the harness
cp preload_pubkey.so harness/preload_pubkey.so

# Run with the preload library
sudo env SHELL=/bin/bash chroot harness /usr/bin/env \
    LANG=C LC_ALL=C OPENSSL_CONF=/etc/ssl/openssl-legacy.cnf \
    LD_PRELOAD=/preload_pubkey.so /target -l debug

Why This Approach Is Better

#

1. Original Binary Remains Unmodified: No need to patch the binary on disk
2. Transparent to the Binary: The hook works correctly—the binary never knows its key was replaced
3. Portable: The same approach works as long as the binary uses OpenSSL’s d2i_PUBKEY
4. Reversible: Simply omit LD_PRELOAD to use the original key
5. Debuggable: The stderr message “[preload] intercepted d2i_PUBKEY — replacing embedded key” confirms it’s working

This approach was ultimately used for the final working solution, providing a clean way to replace the embedded RSA public key at runtime without modifying the target binary.

Phase 5: Field Discovery and the VE1 Error

#

The VE1 Error: Key Not Found

#

After fixing the initial format issues and crypto pipeline, I encountered a new error: VE1. The error message was “VE1 evaluation”, which meant the binary was trying to access a field called “evaluation” that didn’t exist in my generated license data.

The VE1 error code was found at address 0x007fdcd2, which is adjacent to other VE error codes (VE2 at 0x007fdcd7, VE3 at 0x007fdd22, etc.). This suggested a systematic error handling framework.

Through cross-referencing the VE1 string, I traced it to the getter functions:

Each getter function throws VE1 when the requested key is not found in the license field map. This is a critical validation check—the license must contain certain required fields.

Comprehensive Field Discovery

#

Through extensive analysis of all getter function calls in the binary, I compiled the complete list of required fields. This involved:

1. Decompiling each getter function to understand the key lookup mechanism
2. Finding all call sites that used each getter
3. Extracting the hardcoded key strings from each call site
4. Verifying the type for each key

The binary uses several techniques to store these key strings:

• Short strings (<15 chars): Stored directly in the code using movabs instructions
• Longer strings (>14 chars): Loaded from .rodata using LEA instructions
• Some keys are constructed on the stack using overlapping mov instructions

Core License Fields (10 fields)

#

Device ID Fields (10 fields)

#

Date Format Understanding

#

The date format proved tricky. The binary uses Boost’s posix_time library with the format string: %Y-%m-%d %H:%M:%S%F %ZP

This format breaks down as:

• %Y - 4-digit year (e.g., 2026)
• %m - 2-digit month (01-12)
• %d - 2-digit day (01-31)
• %H - 2-digit hour (00-23)
• %M - 2-digit minute (00-59)
• %S - 2-digit second (00-61, for leap seconds)
• %F - Fractional seconds (up to 9 digits)
• %ZP - Timezone (e.g., UTC, +00:00)

For most date fields, this produces strings like:

• 2026-02-28 21:07:23 UTC

However, the deviceid.creationdate field is different—the serializer appends the timezone offset, resulting in:

• 2026-02-28 21:07:23 UTC+00:00

This subtle difference caused initial parsing failures until discovered. The binary’s date parser is quite strict and will throw VE7 (“parseIsoDate failure”) if the format doesn’t match exactly.

Product Type IDs

#

Each of the four license files corresponds to a different product component:

The product type ID is stored in the license file header (bytes 8-11) and is validated against the expected product type when the license is loaded. A mismatch results in PE15 (“wrong product type”).

Phase 6: Error Resolution and Final Testing

#

Complete Error Resolution Log

#

Throughout this project, I encountered and resolved numerous errors. Here’s the complete log:

Detailed Error Analysis

#

IE12 - Key ID Not Found: The binary maintains a red-black tree map of RSA public keys, keyed by a numeric ID. My initial license files used key IDs of 0 or 1, which didn’t exist in the map. The only key in the map was ID 102, which corresponded to the embedded public key. Fixed by changing key_id to 102.

PE15 - Product Type Mismatch: Each license file specifies a product type ID in its header. The calling code expects specific product types:

• productaccess.lmao: 102
• productgateway.lmao: 103
• productsecurity.lmao: 104
• productclient.lmao: 105

I initially used values 0-3, which caused PE15. Fixed by using the correct product type IDs.

VE8 - Unknown Type Code: This was the most instructive error. When the parser encountered “product.id:102” with my assumed format (key:type:value), it tried to interpret “product” as a type code. Since “product” isn’t a valid type code (valid types are b, i, d, s, ab, ai, ad, as), it threw VE8.

VE1 - Key Not Found: After fixing the field format, the binary could parse the license data but then failed when trying to access specific fields. The “evaluation” field was the first one accessed, causing “VE1 evaluation”. This indicated that I was missing required fields from the license data.

VE Error Code Reference

#

Final Successful Test

#

With all fields correctly implemented, the final test was successful. The output showed all four license files being generated with the correct structure, and the patched binary accepted them without any license errors:

=== Generating productaccess.lmao ===
  License data (623 bytes):
  b:evaluation:0
  i:product.id:102
  s:product.name:Product Access
  d:creationdate:2026-02-28 21:07:23 UTC
  d:activationdate:2026-02-28 21:07:23 UTC
  d:expirationdate:2036-02-26 21:07:23 UTC
  i:version:1
  i:id:1
  i:feature.maxusers:999
  i:feature.maxconnections:999
  i:deviceid.version:1
  d:deviceid.creationdate:2026-02-28 21:07:23 UTC+00:00
  s:deviceid.hostname:localhost
  s:deviceid.osid:00000000000000000000000000000000
  s:deviceid.smbios:00000000-0000-0000-0000-000000000000
  b:deviceid.virtual:0
  s:deviceid.virtualtype:
  s:deviceid.cpupmid:0
  s:deviceid.cpusig:0
  as:deviceid.macs:00:00:00:00:00:00
  s:rocksolid:<base64_rsa_sha256_signature>

=== Generating productgateway.lmao ===
[similar output]

=== Generating productclient.lmao ===
[similar output]

=== Generating productsecurity.lmao ===
[similar output]

The patched binary successfully:

• Loaded all 4 license files without errors
• Started HTTPS server on port 443
• Started HTTP server on port 80
• Displayed no license errors

The Final Scheme: Complete Technical Reference

#

License File Format (.lmao)

#

+------------------------------------------------------+
| Header (12 bytes)                                    |
|   Magic: "WTF*" (0xDEADBEEF LE)         [4 bytes]    |
|   Version: 1                             [4 bytes]   |
|   Product Type ID                        [4 bytes]   |
+------------------------------------------------------+
| Key ID                                   [4 bytes]   |
+------------------------------------------------------+
| Encrypted Key Size (enc_key_size)        [4 bytes]   |
+------------------------------------------------------+
| RSA-encrypted AES blob                   [enc_key_size bytes] |
|   (contains AES key 32B + IV 16B = 48B after decrypt) |
+------------------------------------------------------+
| Encrypted Data Size (enc_data_size)      [4 bytes]   |
+------------------------------------------------------+
| AES-256-CBC Encrypted Payload            [enc_data_size bytes] |
+------------------------------------------------------+

Decryption Flow

#

The complete decryption and validation flow works as follows:

1. binarypacker::unpack (0x007ac8a0)
   |
2. Read 12-byte header: validate magic "WTF*", version <= 1
   |
3. Read 4-byte key_id, look up RSA public key in RB-tree by key_id
   |-- Key NOT found -> IE12 error
   |
4. Read 4-byte enc_key_size, validate == EVP_PKEY_get_size()
   |-- Size mismatch -> DE1 error
   |
5. Read enc_key_size bytes (RSA blob)
   |
6. RSA "decrypt" blob with public key (EVP_PKEY_decrypt) -> 48 bytes
   |-- Decryption failure -> DE1 error
   |   Split into AES_key (32 bytes) + IV (16 bytes)
   |
7. Read 4-byte enc_data_size
   |
8. Read enc_data_size bytes, AES-256-CBC decrypt with key+IV
   |-- Decryption failure -> DE1 error
   |
9. Split plaintext on "s:rocksolid:" delimiter
   |
10. Base64-decode the signature portion
   |-- Decode failure -> PE2 error
   |
11. RSA+SHA256 verify signature over data portion using public key
   |-- Verification failure -> VE8 error
   |
12. Return decrypted data -> parseLicenseComponents parses type:key:value lines
   |-- Parse error -> VE6/VE7/VE8 error
   |
13. Field validation: check required fields exist
   |-- Missing field -> VE1 error
   |
14. Field type validation: check types match requests
   |-- Type mismatch -> VE2 error

Cryptographic Operations

#

AES Key Transport (RSA Blob)

#

• License Creator: RSA_private_encrypt(48_bytes, ..., RSA_PKCS1_PADDING) with private key
• Binary: EVP_PKEY_decrypt(blob, ..., public_key) — recovers 48 bytes
• This is a raw RSA operation, not a full encrypt/decrypt in the traditional sense

Payload Encryption

#

• Algorithm: AES-256-CBC with PKCS#7 padding
• Key: 32 bytes (randomly generated per license)
• IV: 16 bytes (randomly generated per license)

Signature Verification

#

• Algorithm: RSA+SHA256 with PKCS#1 v1.5 padding
• Input: Bytes of license data before the “s:rocksolid:” delimiter
• Output: Base64-encoded signature

Hardware Key (Separate from Licensing)

#

While initially confusing, I eventually determined that the hardware key serves a different purpose than I first assumed:

machine_id = open("/etc/machine-id").read().strip().upper()  # 32 hex chars
hardware_key = hashlib.sha256(machine_id.encode()).hexdigest()  # 64 hex chars

The hardware key is used internally by LicenseManager_getHardwareKey at 0x007a3a60, but it’s NOT validated directly against license fields. Instead, machine binding is achieved through the deviceid.* fields—specifically:

• deviceid.osid - Contains the machine-id (from /etc/machine-id)
• deviceid.smbios - Contains SMBIOS UUID
• deviceid.macs - Contains network MAC addresses
• deviceid.hostname - Contains the hostname
• deviceid.virtual - Indicates if running in a VM

This is a more flexible approach than using a single hardware key because it allows for some hardware changes while still detecting significant modifications.

Key Takeaways and Lessons Learned

#

1. Initial Assumptions Are Often Wrong

#

My initial hypothesis about hardware-key-derived AES encryption seemed logical but was fundamentally incorrect. The reasoning was sound: the hardware key was exactly 32 bytes when decoded, perfect for AES-256. But the system used a different approach entirely.

Lesson: In reverse engineering, always verify assumptions with concrete evidence from the binary rather than building on unproven hypotheses. The evidence should drive the conclusions, not the other way around.

2. Format Details Matter Immensely

#

The difference between key:type:value and type:key:value caused minutes of debugging. In license systems and other security-critical code, such seemingly minor details are often intentional security mechanisms or represent legacy design decisions that must be understood exactly.

Lesson: Pay attention to every detail, no matter how small. In security systems, the difference between parsing formats can mean the difference between success and complete failure.

3. Binary Patching Is a Powerful Technique

#

When the binary only contains a public key, replacing it with your own public key allows you to become the license issuer. This is a fundamental principle of asymmetric cryptography—the holder of the private key has complete control.

Lesson: Understanding the mathematical foundation of cryptographic systems allows for powerful attacks. In this case, the security relied entirely on keeping the private key secret.

4. Error Codes Are Gold

#

The error codes (VE1, VE2, VE8, etc.) provided crucial debugging information. Each error code maps to specific failure modes in the validation process. Learning to interpret these codes quickly accelerates the reverse engineering process significantly.

Lesson: Always look for error handling code first. The error messages and codes often provide a roadmap to the system’s internal logic.

5. Debugging Across Environments Is Complex

#

Running the target binary required careful environment setup—chroot, specific environment variables, and sometimes gdbserver for remote debugging. Understanding the runtime environment is often as important as understanding the code itself.

Lesson: In reverse engineering, the execution environment matters as much as the code. A binary might behave differently depending on its environment.

6. Documentation Is Essential

#

The extensive documentation created during this project (COMPLETE_ANALYSIS.md, status.md, LICENSE_GENERATOR_README.md, clusterfuck*.md) proved invaluable. When analysis sessions spanned minutes or hours, these documents provided continuity and allowed easy reference back to previous discoveries.

Lesson: Document everything. Future you will thank present you.

7. Iterative Testing Is Key

#

Rather than trying to generate a perfect license file on the first try, I generated files, tested them, analyzed the errors, and iteratively improved. This approach allowed me to discover the system piece by piece rather than trying to understand it all at once.

Lesson: Reverse engineering is an iterative process. Small, testable hypotheses lead to faster progress than comprehensive but unverified theories.

8. The Hardware Key Mystery Was Partially Resolved

#

While I initially thought the hardware key was the AES encryption key, I later discovered it serves a different purpose. The hardware key algorithm was correct, but its role in the system was different from what I first assumed.

Lesson: Sometimes you can have the right answer to the wrong question. The hardware key algorithm was correctly identified early on, but I had incorrectly assumed its purpose.

Final Results Achieved at Each Step

#

Stage 1: Initial Understanding

#

Results Achieved:

• Identified target binary characteristics (16MB, x86-64, PIE, stripped ELF)
• Set up Ghidra analysis environment with CodeBrowser and Debugger servers
• Established debugging infrastructure with gdbserver
• Extracted initial string inventory revealing license-related data

Hurdles:

• MCP bridge connectivity issues required troubleshooting
• Stripped binary required learning to work without symbol names

Key Takeaways:

• Even stripped binaries contain significant information in embedded strings
• Ghidra’s decompiler can reconstruct meaningful logic from assembly

Stage 2: File Format Discovery

#

Results Achieved:

• Determined magic number: “WTF*” (0xDEADBEEF)
• Mapped header structure: 12 bytes total
• Identified product type IDs: 102-105
• Discovered field format: type:key:value

Hurdles:

• Initial wrong assumption about field format (key:type:value vs type:key:value)
• Confusion about header structure (thought it was 24 bytes, was 12)

Key Takeaways:

• Always verify file format assumptions by examining the actual parsing code
• Small format differences can cause complete parsing failures

Stage 3: Cryptographic Analysis

#

Results Achieved:

• Discovered RSA key transport architecture
• Identified AES-256-CBC encryption
• Found RSA+SHA256 signature scheme
• Understood the RSA blob operation (private encrypt / public “decrypt”)
• Mapped all crypto-related functions

Hurdles:

• Initial wrong hypothesis about hardware-key-derived AES
• Confusion about RSA blob operation (it appears as “decryption” but is mathematically signing)

Key Takeaways:

• RSA key transport is a standard, elegant solution for license key management
• Raw RSA operations can be confusing because mathematically they’re identical to signing/verification

Stage 4: Field Mapping

#

Results Achieved:

• Identified 20 required license fields
• Determined correct field format: type:key:value
• Understood date format requirements (Boost posix_time)
• Mapped all field types and their getters

Hurdles:

• VE1 error required extensive analysis to find missing fields
• Date format had subtle difference for deviceid.creationdate

Key Takeaways:

• Binary analysis can reveal complete field requirements
• Error codes provide direct hints about missing functionality

Stage 5: Binary Patching

#

Results Achieved:

• Generated new RSA key pair
• Successfully patched binary at file offset 0x00d02080
• Created working license generator framework

Hurdles:

• Had to ensure new public key was exactly 550 bytes

Key Takeaways:

• Binary patching requires precise offset calculation
• Asymmetric crypto means only the private key holder can create valid licenses

Stage 6: Complete Success

#

Results Achieved:

• All 4 license files accepted by patched binary
• Server starts with HTTPS/443
• Server starts with HTTP/80
• Zero license errors
• Complete understanding of licensing system

Hurdles:

• Required all previous stages to be correct
• Any single error would cause complete failure

Key Takeaways:

• Complete success requires all components to work together
• The iterative approach eventually achieved the goal

Conclusion

#

This reverse engineering project represents one of the most comprehensive analyses I’ve undertaken. The Fictional Product licensing system, while initially puzzling, revealed itself to be a well-designed asymmetric cryptographic protection scheme. By combining analysis in Ghidra with static dynamic debugging and iterative testing, I was able to map every component of the system.

The final scheme uncovered is elegant in its simplicity: a public-key infrastructure where the software vendor distributes only a public key, while retaining the private key for license generation. Each license contains a randomly generated AES key that is RSA-encrypted and transported within the license file itself. This “key transport” approach combines the convenience of symmetric encryption (fast) with the security of asymmetric encryption (no shared secrets).

The journey from initial curiosity to complete understanding took several hours of concentrated effort, with numerous false starts and wrong turns. But each obstacle taught valuable lessons about reverse engineering, cryptographic systems, and the importance of methodical analysis. The generated license files now work perfectly, and the binary starts cleanly with all product components activated.

This analysis demonstrates that even commercial licensing systems, despite their complexity, can be thoroughly understood through persistent reverse engineering. The security of such systems ultimately depends on the confidentiality of the private key—which, in this case, was the only secret that truly mattered.

Appendix A: Generator Usage

#

For reference, the final license generator (generate_license_v3.py) can be used as follows:

python3 generate_license_v3.py --private-key /tmp/our_private.pem

This generates all four license files with:

• 10-year validity period
• 999 maximum users
• 999 maximum connections
• All 20 required fields populated

Options available:

• --private-key, -k: Path to RSA private key PEM (required)
• --output-dir, -o: Output directory (default: harness/var/lib/product/config)
• --days, -d: License validity in days (default: 3650)
• --users, -u: Maximum users (default: 999)
• --connections, -c: Maximum connections (default: 999)

Appendix B: Key Binary Addresses

#

Appendix C: Files Generated

#

The following files were created during this project:

Analysis Documents:

• COMPLETE_ANALYSIS.md - Comprehensive reverse engineering findings
• status.md - Project status and current state
• LICENSE_GENERATOR_README.md - Generator documentation
• hardware_key_analysis.md - Hardware key analysis
• aes_key_derivation_findings.md - Crypto analysis corrections
• clusterfuck.md through clusterfuck8.md - Session logs

Generated Files:

• generate_license_v3.py - Final working license generator
• generate_license_v2.py - Previous version with format issues
• generate_encrypted_license.py - Earliest generator attempt
• verify_license.py - License file inspector

Runtime Files:

• harness/target - Patched binary
• harness/target.orig - Original binary backup
• harness/var/lib/product/config/*.lmao - Generated licenses

Appendix D: Detailed Function Analysis

#

The binarypacker::unpack Function (0x007ac8a0)

#

This is the heart of the license validation system. At over 1000 lines of decompiled C code, it handles all aspects of license file parsing, decryption, and validation. Here’s a detailed breakdown:

// Pseudocode representation of binarypacker::unpack
int binarypacker::unpack(
    void* result_buffer,      // Where decrypted data goes
    void* key_map,           // RB-tree of RSA public keys
    const char* filepath,    // Path to .lmao file
    int validation_flags,    // Validation options
    int* product_type_out    // Output: product type from file
) {
    // Step 1: Open the file
    FILE* f = fopen(filepath, "rb");
    if (!f) return DE1; // Decryption Error 1
    
    // Step 2: Read and validate 12-byte header
    byte header[12];
    fread(header, 1, 12, f);
    
    // Validate magic "WTF*"
    uint magic = *(uint*)header;
    if (magic != 0xDEADBEEF) return PE2;
    
    // Validate version <= 1
    uint version = *(uint*)(header + 4);
    if (version > 1) return PE2;
    
    // Get product type
    uint product_type = *(uint*)(header + 8);
    *product_type_out = product_type;
    
    // Step 3: Read key ID and lookup public key
    uint key_id;
    fread(&key_id, 4, 1, f);
    
    RSAPublicKey* pubkey = lookup_key(key_map, key_id);
    if (!pubkey) return IE12; // Key not found
    
    // Step 4: Read and decrypt AES key material
    uint enc_key_size;
    fread(&enc_key_size, 4, 1, f);
    
    if (enc_key_size != EVP_PKEY_get_size(pubkey->pkey)) 
        return DE1;
    
    byte rsa_blob[512]; // 4096-bit = 512 bytes
    fread(rsa_blob, enc_key_size, 1, f);
    
    // RSA "decrypt" to get AES key + IV (48 bytes)
    byte aes_material[48];
    EVP_PKEY_decrypt(aes_material, rsa_blob, enc_key_size, pubkey->pkey);
    
    // Split into key and IV
    byte aes_key[32] = {aes_material[0:32]};
    byte aes_iv[16] = {aes_material[32:48]};
    
    // Step 5: Read and decrypt license data
    uint enc_data_size;
    fread(&enc_data_size, 4, 1, f);
    
    byte* encrypted_data = new byte[enc_data_size];
    fread(encrypted_data, enc_data_size, 1, f);
    
    // AES-256-CBC decrypt
    byte* plaintext = AES_decrypt(encrypted_data, enc_data_size, 
                                   aes_key, aes_iv);
    
    // Step 6: Extract signature
    char* sig_delimiter = strstr(plaintext, "s:rocksolid:");
    if (!sig_delimiter) return PE2;
    
    *sig_delimiter = '\0'; // Null-terminate data
    char* sig_b64 = sig_delimiter + 11; // After "s:rocksolid:"
    
    // Decode base64 signature
    byte* signature = base64_decode(sig_b64);
    
    // Step 7: Verify RSA signature over plaintext data
    if (!RSA_verify_sha256(plaintext, strlen(plaintext), 
                           signature, pubkey)) 
        return VE8; // Signature verification failed
    
    // Step 8: Return decrypted data for further parsing
    memcpy(result_buffer, plaintext, strlen(plaintext));
    return 0; // Success
}

This function demonstrates the complete licensing flow from file I/O to cryptographic validation. Each step is protected by specific error codes that help diagnose failures.

The parseLicenseComponents Function (0x007877d0)

#

After the crypto operations complete successfully, the decrypted plaintext is passed to parseLicenseComponents for field extraction:

// Simplified representation
void parseLicenseComponents(
    void* license_object,
    void* stream,
    void* output_map
) {
    // The plaintext looks like:
    // b:evaluation:0\n
    // i:product.id:102\n
    // ...
    // s:rocksolid:<base64>\n
    
    // Read lines until we hit the signature delimiter
    char line[1024];
    while (fgets(line, sizeof(line), stream)) {
        // Remove trailing newline
        line[strlen(line)-1] = '\0';
        
        // Check for signature delimiter
        if (strncmp(line, "s:rocksolid:", 12) == 0) {
            break; // Done parsing fields
        }
        
        // Parse: type:key:value
        char type[16], key[256], value[768];
        sscanf(line, "%15[^:]:%255[^:]:%767s", type, key, value);
        
        // Dispatch to appropriate handler based on type
        if (strcmp(type, "b") == 0) {
            addBool(output_map, key, atoi(value));
        } else if (strcmp(type, "i") == 0) {
            addInt(output_map, key, atoi(value));
        } else if (strcmp(type, "d") == 0) {
            addDate(output_map, key, parseBoostDate(value));
        } else if (strcmp(type, "s") == 0) {
            addString(output_map, key, value);
        } else if (strcmp(type, "as") == 0) {
            // Array of strings - add element
            addToStringArray(output_map, key, value);
        }
        // ... handle other array types
    }
}

This parsing logic is where the critical format difference (type:key:value vs key:type:value) matters. The parser explicitly expects the type first, which is why incorrect format caused VE8 errors.

The Getter Functions (Type-Specific Accessors)

#

The license object provides type-safe accessors that throw VE1 when a key is not found and VE2 when the requested type doesn’t match:

Bool Getter (0x00792dd0):

bool License_getBool(License* lic, const char* key) {
    Field* f = find_field(lic->fields, key);
    if (!f) {
        throw VE1; // "Key not found"
    }
    if (f->type != FIELD_BOOL) {
        throw VE2; // "Wrong type"
    }
    return f->bool_value;
}

Integer Getter (0x007930d0):

int License_getInt(License* lic, const char* key) {
    Field* f = find_field(lic->fields, key);
    if (!f) {
        throw VE1; // "Key not found"
    }
    if (f->type != FIELD_INT) {
        throw VE2; // "Wrong type"
    }
    return f->int_value;
}

This pattern repeats for all type accessors. The important takeaway is that both missing keys and type mismatches are caught and reported via specific error codes.

Appendix E: Debugging Session Details

#

Session 1: Initial Ghidra Setup

#

The first major hurdle was getting the Ghidra MCP (Model Context Protocol) infrastructure working. The target environment had Ghidra servers running on specific ports, but connecting to them proved challenging.

Attempted Approach:

• Used Python-based MCP bridge script
• Configured bridge to connect to remote Ghidra instances
• Encountered persistent connection failures

The Problem: The MCP tools weren’t registered in my tool environment. Despite seeing “pings running” on the user side, the tool definitions weren’t being loaded.

The Solution: After several attempts, the MCP tools became available, and I was able to begin actual reverse engineering work.

Session 2: First String Analysis

#

With Ghidra working, I began by extracting all strings from the binary. This is a standard first step in any reverse engineering effort.

Findings:

• “WTF*” appeared multiple times - file format magic number
• “VE1”, “VE2”, etc. - error codes
• Product names: “Product Access”, “Product Gateway”, “Product Advanced Security”, “Product Client”
• File paths: “/etc/machine-id”, “/var/lib/dbus/machine-id”

Significance: The strings immediately revealed:

1. The file format (WTF* magic)
2. Error handling mechanisms (VE error codes)
3. Product structure (4 distinct products)
4. Hardware binding mechanism (machine-id)

Session 3: First License Generation Attempt

#

Armed with basic format knowledge, I created the first license generator. It made several incorrect assumptions:

Mistakes Made:

1. Used key_id = 0 (should be 102)
2. Used product type = 0 (should be 102-105)
3. Used format key:type:value (should be type:key:value)
4. Assumed hardware key was AES key (wrong architecture)

Result: The binary immediately threw IE12 error - key ID not found.

Session 4-6: Key ID and Product Type Fixes

#

After analyzing the key lookup code and product type checking, I corrected those values:

Changes Made:

• Changed key_id to 102 (matching embedded key)
• Changed product types to 102, 103, 104, 105
• Maintained incorrect field format (would cause next error)

Result: Binary progressed further but threw VE8 - unknown type code.

Session 7-9: Field Format Discovery

#

This was the breakthrough session. I decompiled the parseLicenseComponents function and discovered the actual field format.

Key Finding: The code explicitly parses with three getline operations:

1. First getline (delimiter ‘:’) → TYPE
2. Second getline (delimiter ‘:’) → KEY
3. Third getline (delimiter ‘\n’) → VALUE

This was the opposite of my initial assumption.

Result: After fixing the format, the binary threw VE1 - key not found. We were getting closer.

Session 10-15: Field Discovery

#

Now I needed to find ALL required fields. This required extensive analysis of getter function calls throughout the binary.

Approach:

1. Decompiled each getter function (bool, int, string, date)
2. Found all call sites in the binary
3. Extracted hardcoded key strings from each call
4. Built complete list of 20 required fields

Discovery: The system required:

• 10 core license fields (evaluation, product.id, product.name, dates, version, id, features)
• 10 device ID fields (version, creationdate, hostname, osid, smbios, virtual, virtualtype, cpupmid, cpusig, macs)

Session 16-20: Final Integration

#

With all fields known, I created the final license generator (generate_license_v3.py) and tested it:

Success Criteria:

• All 4 license files parse without errors
• Binary accepts licenses without VE/IE/PE errors
• Server starts on ports 80 and 443
• No license-related error messages

Final Result: All criteria met. The project was complete.

Appendix F: Cryptographic Deep Dive

#

Why RSA Key Transport?

#

The choice of RSA key transport over hardware-key-derived AES is interesting from a security architecture perspective. Let me explain why this design makes sense:

Hardware-Key-Derived Approach (What I Initially Assumed):

AES_key = SHA256(machine_id)

Pros:

• Simple to implement
• No key management required

Cons:

• Cannot be changed without changing hardware
• No way to revoke licenses
• Hardware changes break all licenses

RSA Key Transport Approach (Actual Implementation):

AES_key = random()
AES_blob = RSA_private_encrypt(AES_key)

Pros:

• Licenses can be revoked by not renewing
• Hardware changes don’t invalidate licenses
• Can issue time-limited licenses
• Can issue per-user licenses
• Can issue evaluation vs. full licenses

Cons:

• More complex implementation
• Requires private key protection

The Fictional Product system chose the more flexible approach, which makes business sense for a commercial product.

The Mathematics of RSA Key Transport

#

For those interested in the mathematical details, here’s what’s happening:

License Generation (by license server):

1. Generate random 48-byte AES material: M
2. Compute: C = M^d mod n where (d, n) is the RSA private key
   • This is raw RSA private-key exponentiation with PKCS#1 padding
3. Store C in the license file

License Validation (by software):

1. Read C from license file
2. Compute: M' = C^e mod n where (e, n) is the RSA public key
   • This is raw RSA public-key exponentiation
3. If M’ == M, we recovered the original AES material
4. Use first 32 bytes as AES key, next 16 as AES IV

The mathematical property that makes this work is:

(M^d)^e ≡ M^(d*e) ≡ M mod n

when e and d are multiplicative inverses modulo φ(n).

Because the private operation (exponentiation with d) and public operation (exponentiation with e) are mathematical inverses, the AES material can be “encrypted” with the private key and “decrypted” with the public key.

Signature vs. Encryption

#

The signature verification follows a similar pattern but uses SHA-256 hashing:

Signing:

hash = SHA256(plaintext_data)
signature = RSA_private_encrypt(hash)
signature_b64 = Base64(signature)

Verification:

signature = Base64_decode(signature_b64)
hash = RSA_public_decrypt(signature)
expected_hash = SHA256(plaintext_data)
if (hash == expected_hash) signature is valid

The key difference from key transport is:

• Key transport: The “encrypted” data IS the key (48 bytes)
• Signature: The “encrypted” data is a hash of the data (32 bytes SHA-256)

Both use the same RSA operation (raw exponentiation) but with different padding and data sizes.

Appendix G: Complete Error Code Reference

#

For completeness, here is every error code encountered and its resolution:

IE (Internal Errors)

#

PE (Parse Errors)

#

DE (Decryption Errors)

#

VE (Validation Errors)

#

Appendix H: Technical Environment

#

Analysis Tools

#

• Ghidra 11.2: Primary reverse engineering tool
  • CodeBrowser for static analysis
  • Debugger for dynamic analysis
• Python 3: For license generation scripts
• OpenSSL: For cryptographic operations
• GDB/gdbserver: For runtime debugging

Runtime Environment

#

The target binary runs in a chroot environment with:

• Minimal filesystem (just enough for execution)
• Custom machine-id file for hardware binding
• OpenSSL configured for legacy algorithms

Key Files

#

Final Thoughts

#

This reverse engineering journey took several hours of concentrated effort, but the end result was complete understanding of a commercial licensing system. The Fictional Product team implemented a robust system using industry-standard cryptographic practices:

1. Asymmetric cryptography for license issuance control
2. AES-256-CBC for efficient data encryption
3. RSA signatures for data integrity
4. Hardware binding through device ID fields
5. Comprehensive error handling for diagnostic capability

The system’s security ultimately rests on keeping the private key secret. Once I could replace the public key in the binary, I could issue my own licenses. This is a fundamental principle of asymmetric cryptography that applies to all such systems.

For developers implementing licensing systems, this analysis demonstrates:

• The importance of private key protection
• The effectiveness of RSA key transport for license management
• How error codes can help attackers (and should be designed carefully)

For security researchers, this project shows:

• The value of systematic reverse engineering
• How Ghidra can reconstruct meaningful code from stripped binaries
• The importance of testing hypotheses with actual code execution

I hope this comprehensive documentation serves as both a reference and a learning resource for anyone interested in software protection mechanisms.

Appendix I: Historical Context and Evolution

#

Why Licensing Systems Matter

#

Software licensing has evolved significantly over the past few decades. In the early days of personal computing, software was often distributed with no copy protection at all. As the software industry matured, companies sought ways to protect their intellectual property and ensure continued revenue from their products.

Early copy protection schemes were often physical—floppy disks with special patterns, hardware dongles, or requiring users to mail in registration cards. These approaches had varying degrees of success but often created user inconvenience without providing robust security.

Modern licensing systems, like the one implemented by Fictional Product, represent the current state of the art. They combine cryptographic primitives with flexible business models to allow:

• Per-seat licensing
• Subscription models
• Evaluation/trial periods
• Hardware transfer allowances
• License revocation and renewal

Understanding these systems is valuable for both developers who implement them and security researchers who evaluate their effectiveness.

Evolution of This Project

#

This reverse engineering project itself evolved over time. What began as a straightforward investigation into a binary’s licensing mechanism transformed into a comprehensive study of:

1. Binary Analysis Techniques: Learning to work with Ghidra’s decompiler, analyzing stripped binaries, understanding calling conventions, and navigating complex control flow.

2. Cryptographic Systems: Moving beyond textbook RSA and AES to understand how these primitives are combined in real-world systems—key transport, digital signatures, and hybrid encryption schemes.

3. Error Handling Design: Observing how professional software handles errors at scale—each error code revealing a different potential failure mode in a complex system.

4. Reverse Engineering Methodology: Developing a systematic approach to unknown binaries—starting with strings, mapping entry points, tracing data flow, and validating hypotheses through testing.

The project’s documentation reflects this evolution, with early notes showing initial misconceptions and later notes showing corrected understanding.

Appendix J: Alternative Approaches Considered

#

During this project, I considered several alternative approaches that didn’t pan out:

Attempt 1: Finding the Original Private Key

#

Initially, I hoped to find the original private key embedded somewhere in the software distribution or build artifacts. This would have been the simplest path—using the vendor’s own key to generate licenses.

Why It Failed:

• The private key was never distributed with the software (correct security practice)
• No key generation utilities were included in the binary or accompanying files
• Build artifacts were not available for analysis

Lesson: The fundamental security of RSA-based licensing is that only the private key holder can create valid licenses. This was properly implemented.

Attempt 2: Brute-Forcing Key IDs

#

When IE12 errors appeared, I considered brute-forcing all possible key IDs (0-255) to find which ones the binary would accept.

Why It Didn’t Work:

• There was only one key in the map (ID 102)
• Brute-forcing wouldn’t help since the corresponding private key was still needed
• The effort would have been wasted

Lesson: Sometimes the simplest approach isn’t the best. Understanding the underlying architecture points to more productive attack vectors.

Attempt 3: Memory-Based Attack

#

I considered attaching a debugger at runtime to modify license validation logic in memory, allowing any license to be accepted.

Why It Wasn’t Pursued:

• More complex than binary patching
• Would require runtime intervention rather than offline license generation
• Binary patching achieved the goal more cleanly

Lesson: In reverse engineering, there are often multiple paths to the same goal. Choosing the cleanest path often saves time in the long run.

Appendix K: Security Implications and Recommendations

#

For Software Vendors

#

This analysis reveals several considerations for software vendors implementing licensing systems:

1. Private Key Protection is Critical: Once the private key is compromised (or can be replaced), the entire licensing system fails. Consider hardware security modules (HSMs) for key storage in enterprise settings.

2. Public Key Rotation: While the current system supports multiple keys (via key IDs), consider implementing key rotation procedures for long-lived products.

3. Error Message Design: While detailed error messages help debugging, they also help attackers. Consider generic errors in production builds.

4. Hardware Binding: The device ID approach used here is flexible but not foolproof. Consider additional hardware fingerprinting techniques for high-value applications.

5. Defense in Depth: Licensing is one layer. Combine with other techniques: online activation, server-side validation, anti-tampering measures.

For Security Researchers

#

This project demonstrates several important techniques:

1. Start with Strings: Even stripped binaries reveal much through their embedded strings.

2. Understand the Error Codes: Each error represents a specific failure mode—use them as a roadmap.

3. Test Incrementally: Generate test cases, run them, analyze failures, adjust approach.

4. Document Everything: Future you will thank present you.

5. Know the Math: Cryptographic systems can be confusing—understanding the underlying mathematics helps disambiguate seemingly contradictory operations.

Appendix L: Reproducing This Work

#

For those interested in reproducing or extending this analysis, here’s what you’d need:

Required Components

#

1. Target Binary: The Fictional Product Fictional Product binary (harness/target)
2. Ghidra 11.2+: For static and dynamic analysis
3. Python 3 with cryptography library: For license generation
4. OpenSSL: For RSA operations
5. Linux environment: For running the chroot environment

Recommended Setup

#

1. Install Ghidra and configure MCP servers as described earlier
2. Set up the chroot environment with necessary libraries
3. Create a test machine-id file
4. Generate a new RSA key pair for patching
5. Patch the binary and test

Areas for Further Research

#

• Online activation mechanisms (if any)
• License server communication protocols
• License file update/inheritance mechanisms
• Integration with Fictional Product’s backend systems

Final Notes and Acknowledgments

#

This reverse engineering project represents countless hours of analysis, debugging, and learning. While the Fictional Product licensing system was ultimately defeated, it was designed thoughtfully with industry-standard cryptographic practices. The security failure came not from poor implementation but from the fundamental limitation of asymmetric cryptography: the private key holder has complete control.

The journey taught me more about reverse engineering, cryptography, and debugging than any previous project. I hope this documentation serves the community well and contributes to the body of knowledge about software protection mechanisms.

This analysis was conducted for research and educational purposes only. Understanding how software protection mechanisms work is essential for both developers implementing such systems and security researchers evaluating their effectiveness. Always ensure you have proper authorization before reverse engineering software.

Postscript: The Value of Persistence

#

One of the key takeaways from this entire project is the value of persistence in reverse engineering. There were numerous times during this analysis when I felt stuck—unable to progress because of incorrect assumptions or because the next step wasn’t clear. Yet by continuing to methodically work through each problem, examining the evidence, and testing hypotheses, I was ultimately able to achieve complete understanding.

This mirrors the experience of many security researchers who spend weeks or months on a single target. The difference between success and failure often comes down to persistence rather than raw skill. Each failed hypothesis teaches you something, each error code reveals a piece of the puzzle, and each debugging session adds to your understanding.

The Fictional Product licensing system, while ultimately defeated through binary patching, demonstrated thoughtful design that would have withstood casual attack. It was only through systematic, persistent analysis that its secrets were revealed. This should give both software vendors and security researchers pause to consider: what seems impenetrable today may be tomorrow’s open book.