Modern x64 Reverse Engineering for AI-Assisted Malware Analysis // Karim Nathoo

In-Person | November 2-4 | 3 Days

BOOK NOW

ABSTRACT

This practical, hands-on course teaches modern reverse engineering and code analysis for students who want to take their malware analysis, incident response, vulnerability research, or penetration testing skills to the next level.

AI-assisted analysis tools are rapidly changing the way analysts approach reverse engineering. Large language models, decompiler integrations, and AI-enabled workflows can help explain code, summarize functionality, generate hypotheses, and accelerate repetitive analysis tasks. However, these tools are only useful when the analyst has enough core reverse engineering knowledge to guide the AI, provide useful context, and validate the results.

This course teaches the foundational reverse engineering skills required to use modern AI-assisted analysis workflows effectively. Students will analyze 64-bit Windows binaries using free and accessible tools such as Ghidra, IDA Free, x64dbg, WinDbg, Python, and common PE analysis utilities. The emphasis is on practical analyst tradecraft: understanding compiler-generated code, recognizing common programming constructs, reasoning about Windows API usage, validating decompiler output, and confirming findings through debugging and runtime observation.

The course is delivered in an applied format using realistic malware analysis and offensive security case studies based on problems the instructor has encountered in the field. Students will move from foundational exercises into practical analysis scenarios involving real-world code, malware behaviors, obfuscation, encoding, and anti-analysis techniques.

A major theme throughout the course is human-in-the-loop reverse engineering: using AI as an assistant, not an oracle. Students will learn how to ask better analysis questions, provide useful context from disassembly and decompiler output, interpret AI-generated explanations, identify hallucinations or incorrect assumptions, and confirm findings using debuggers, traces, scripts, and manual analysis.

INTENDED AUDIENCE

• Malware analysts
• Incident responders
• Threat hunters
• Vulnerability researchers
• Penetration testers
• Detection engineers
• SOC analysts seeking deeper technical understanding
• Security practitioners who want to understand how software actually behaves at the code level
• Analysts who want to use AI-assisted reverse engineering tools more effectively and safely

This course is especially useful for students who want to build the foundation required for more advanced training in malware analysis, exploit development, vulnerability research, firmware analysis, or AI-assisted reverse engineering.

COURSE OUTLINE

• Introduction to modern x64 assembly language
• x64 registers, stack usage, calling conventions, and function prologues/epilogues
• Windows PE file format and common PE analysis workflows
• Introduction to the Windows API from a reverse engineering perspective
• Principles of code disassembly and decompilation
• Recognizing common programming constructs in assembly and decompiler output
• Working with strings, imports, exports, resources, cross-references, and data references
• Static analysis using free and accessible tools such as Ghidra and IDA Free
• Debugging principles for reverse engineers
• Dynamic analysis using tools such as x64dbg and WinDbg
• Using decompiler output effectively without blindly trusting it
• Identifying compiler artifacts, library code, wrapper functions, and optimized code patterns
• Dealing with encoding, encryption, hashing, and data transformation routines
• Custom scripting and automation with Python and reverse engineering tool APIs
• Introduction to AI-assisted reverse engineering workflows
• Prompting AI tools with useful reverse engineering context
• Using AI to explain functions, rename variables, summarize behavior, and generate analysis hypotheses
• Validating AI-generated analysis against disassembly, decompiler output, debugger state, and runtime behavior
• Recognizing when AI tools hallucinate, overstate conclusions, or misinterpret compiler artifacts
• Introduction to anti-reverse engineering techniques and countermeasures
• Building analyst-quality notes and reports from reverse engineering findings

Course Format

Students will complete a series of guided exercises designed to teach the core concepts required for modern reverse engineering. Early labs will focus on understanding 64-bit code, navigating disassembly and decompiler output, recognizing common patterns, and using debuggers effectively.

Once students have developed a practical foundation, they will progress to realistic analysis problems based on malware analysis, penetration testing, and vulnerability research scenarios. These exercises will require students to combine static analysis, dynamic analysis, scripting, and AI-assisted workflows to understand program behavior.

Most exercises and labs will use a 64-bit Windows environment. Although some concepts apply to other architectures and operating systems, the course will focus primarily on practical Windows x64 reverse engineering.

Prerequisites

Hardware and Software Requirements

Students must provide their own laptop computer for the course.
The course will use a virtual machine containing reverse engineering tools, lab binaries, and malware analysis exercises. Some labs may involve real malware or malware-like samples.

Students must have:

• A laptop capable of running a 64-bit Windows virtual machine
• At least 16 GB of RAM, with 32 GB recommended
• At least 100 GB of free disk space
• Administrative privileges on the host laptop
• The ability to disable or configure host-based security tools where necessary for the lab environment
• VMware Workstation Pro, VMware Fusion Pro, or another approved virtualization platform installed before class
• Internet access for selected AI-assisted workflow exercises, unless an offline or local AI environment is provided

Knowledge Prerequisites

• Experience using C/C++
• Experience with reverse engineering tools (Ghidra, IDAPro, etc) and debuggers
• Basic knowledge of Windows internals
• Familiarity with exploitation techniques and types of vulnerabilities

Students should be comfortable with general security concepts and basic programming concepts. Prior experience with malware analysis, penetration testing, incident response, or vulnerability research is helpful but not required.

Students should understand basic programming concepts such as variables, functions, loops, conditionals, and data structures. They should also be comfortable using Windows and basic command-line tools.

Prior assembly language or reverse engineering experience is helpful but not required. The course is designed to bring students up to speed quickly, while still giving them practical skills they can apply immediately after class.

YOUR INSTRUCTOR: Karim Nathoo

Karim Nathoo is a freelance computer security consultant providing specialized security services to government, military and private sector clients. Karim has extensive experience in high assurance ethical hacking, incident response and security product evaluation, including the application of binary code analysis and reverse engineering. Karim has delivered professional services for international clients in Asia, Europe, Canada and the United States. Karim has experience ranging from working with R&D teams in cutting edge technical environments to providing executive level risk management briefings and proof of concept demonstrations.

Karim has performed security assurance and engineering engagements for organizations such as Apple, Microsoft, France Telecom, Cloakware Corporation, Creative Labs, Motorola, Verizon, Nokia, Philips Semiconductor, SONY BMG, SUN Microsystems, QNX Software Systems and numerous Canadian and US Government agencies.

Specialties: Penetration testing, code analysis, reverse engineering, software security evaluation, custom software development, malware analysis, incident response, product evaluation, and security engineering

OTHER IN-PERSON TRAINING COURSES