Reverse Engineering (3/5): dynamic analysis with strace, GDB and Frida

Static analysis tells you what code could do; dynamic analysis shows what it actually does with real input. Professionals switch between the two constantly: form a hypothesis in Ghidra, confirm it in a debugger within seconds. Part 3 of the Reverse Engineering for Developers series covers black-box tracing, debuggers and instrumentation with Frida.

Start with black-box tracing

Before attaching a debugger, watch the program from the outside:

A single ltrace ./crackme AAAA often reveals strcmp("AAAA", "s3cr3t") — the cheapest win in reverse engineering.

Debuggers

• GDB + pwndbg or GEF (Linux): plugins that show registers, stack and disassembly on one screen.
• x64dbg (Windows): the free standard for user mode.
• LLDB (macOS, iOS): Apple's native debugger.
• WinDbg for kernel and crash dumps; rr on Linux records and replays, so you can run backwards to the moment memory was corrupted.

break *0x401a30          # breakpoint at an address
break strcmp             # breakpoint on a function
watch *(int*)0x404040    # stop when memory changes
run AAAA
ni / si                  # step one instruction
x/16gx $rsp              # dump the stack
finish                   # run until the function returns
set $rax = 1             # patch the return value

PIE binaries move on every run: use starti then vmmap (pwndbg) to find the base, or disable ASLR in your lab with setarch -R ./target.

Instrumentation with Frida

Frida injects a JavaScript engine into a running process so you can hook any function without recompiling. It runs on Windows, macOS, Linux, Android and iOS.

// hook.js — log every strcmp call
Interceptor.attach(Module.getGlobalExportByName('strcmp'), {
  onEnter(args) {
    this.a = args[0].readUtf8String();
    this.b = args[1].readUtf8String();
  },
  onLeave(ret) {
    console.log(`strcmp("${this.a}", "${this.b}") = ${ret.toInt32()}`);
  }
});

frida -f ./target -l hook.js       # spawn and instrument
frida-trace -i "recv*" -p 1234     # auto-generate hooks for a running PID

Use Frida to dump decrypted buffers after a decrypt function returns, log arguments to internal functions you found in Ghidra (module base + offset), or replace a return value. API names can change between Frida versions — check the docs for the version you install.

The static + dynamic loop

1. Find a candidate function in Ghidra (for example, the one referencing "invalid license").
2. Note its offset from the image base.
3. Set a breakpoint or Frida hook there and feed controlled input.
4. Inspect arguments and return values; rename and retype in Ghidra.
5. Repeat with the next function in the call chain.

Risks and caveats

• Running is risky: run unknown samples only in a snapshotted VM with shared folders off and the network isolated (host-only or simulated services). FLARE-VM and REMnux ship ready-made toolkits.
• Anti-debugging: many programs detect debuggers and change behaviour — see part 5.
• Permission: only instrument software you own or are authorised to test.

How to apply it

• Dev: solve the crackme from part 2 three ways — ltrace only, a breakpoint on the compare instruction in GDB, and a 10-line Frida script that prints the expected value.
• Tech Lead: Frida and strace are also great production debugging tools for your own apps: see which files and endpoints the app touches without rebuilding.
• PM: a shared lab VM image (snapshotted, tools preinstalled) gets newcomers productive in hours instead of days.