A comprehensive practical guide to reading x86/x64 assembly for malware analysis, covering registers, memory operands, flags and branches, calling conventions (Windows x64, System V AMD64, 32-bit x86), key instruction families, recovering high-level structures like loops and switch statements, recognizing Windows API call patterns (dynamic resolution, memory unpacking, process injection), TEB/PEB access, a fully worked WriteFile example, a repeatable analysis workflow, common interpretation mistakes, and a checklist for analysts. The guide also introduces AIDebug, an open-source companion tool with a Learning Mode that compiles real C functions into disassembly and Ghidra pseudo-code for hands-on practice.
Table of contents
2. Using AIDebug as the practical lab companionGitHub - anpa1200/AIDebug: AI-assisted malware reverse-engineering debugger with ATT&CK, YARA, IOC…Questions this post answers
How do I identify which registers hold function arguments in Windows x64 assembly versus Linux System V AMD64?
On Windows x64, the first four integer or pointer arguments pass in RCX, RDX, R8, and R9, with a 32-byte shadow space reserved by the caller and RSP kept 16-byte aligned before each call. On System V AMD64 (most Linux and Unix systems), the argument order differs, using RDI as the first argument register instead of RCX. Mixing up the two conventions leads to misreading identical instruction bytes. Reverse engineers comparing calling conventions across platforms can track ABI details like these on daily.dev.
What sequence of Windows API calls indicates a process might be injecting code into another process?
A cross-process injection pattern typically chains OpenProcess to get a process handle, VirtualAllocEx to reserve remote memory, WriteProcessMemory to populate that memory, and finally CreateRemoteThread or another execution mechanism to run it. Because security tools and legitimate administration software use similar APIs, classifying the behavior requires checking the target process, access rights, transferred content, and protection flags. Security engineers tracing injection patterns can follow malware analysis techniques like this on daily.dev.
How can I tell if an assembly loop is treating a value as signed or unsigned during malware analysis?
The conditional jump mnemonic reveals the interpretation: unsigned comparisons use instructions like jae, jb, or ja, while signed comparisons use jl, jle, jg, or jge. For example, a loop bounded by jae after a cmp against a counter treats both values as unsigned, which matters for correctly reconstructing buffer lengths, file sizes, and boundary checks rather than assuming a signed greater-than relationship. Analysts debugging boundary conditions in disassembled code can dig into cases like this on daily.dev.