Librería Portfolio

Hack your antivirus software to stamp out future vulnerabilities
The Antivirus Hacker´s Handbook guides you through the process of reverse engineering antivirus software. You explore how to detect and exploit vulnerabilities that can be leveraged to improve future software design, protect your network, and anticipate attacks that may sneak through your antivirus´ line of defense. You´ll begin building your knowledge by diving into the reverse engineering process, which details how to start from a finished antivirus software program and work your way back through its development using the functions and other key elements of the software. Next, you leverage your new knowledge about software development to evade, attack, and exploit antivirus software-all of which can help you strengthen your network and protect your data.

While not all viruses are damaging, understanding how to better protect your computer against them can help you maintain the integrity of your network.

Discover how to reverse engineer your antivirus software
Explore methods of antivirus software evasion
Consider different ways to attack and exploit antivirus software
Understand the current state of the antivirus software market, and get recommendations for users and vendors who are leveraging this software
The Antivirus Hacker´s Handbook is the essential reference for software reverse engineers, penetration testers, security researchers, exploit writers, antivirus vendors, and software engineers who want to understand how to leverage current antivirus software to improve future applications.



Table of Contents

Introduction xix

Part I Antivirus Basics 1

Chapter 1 Introduction to Antivirus Software 3

What Is Antivirus Software? 3

Antivirus Software: Past and Present 4

Antivirus Scanners, Kernels, and Products 5

Typical Misconceptions about Antivirus Software 6

Antivirus Features 7

Basic Features 7

Making Use of Native Languages 7

Scanners 8

Signatures 8

Compressors and Archives 9

Unpackers 10

Emulators 10

Miscellaneous File Formats 11

Advanced Features 11

Packet Filters and Firewalls 11

Self-Protection 12

Anti-Exploiting 12

Summary 13

Chapter 2 Reverse-Engineering the Core 15

Reverse-Engineering Tools 15

Command-Line Tools versus GUI Tools 16

Debugging Symbols 17

Tricks for Retrieving Debugging Symbols 17

Debugging Tricks 20

Backdoors and Configuration Settings 21

Kernel Debugging 23

Debugging User-Mode Processes with a Kernel-Mode Debugger 25

Analyzing AV Software with Command-Line Tools 27

Porting the Core 28

A Practical Example: Writing Basic Python Bindings for Avast for Linux 29

A Brief Look at Avast for Linux 29

Writing Simple Python Bindings for Avast for Linux 32

The Final Version of the Python Bindings 37

A Practical Example: Writing Native C/C++ Tools for Comodo Antivirus for Linux 37

Other Components Loaded by the Kernel 55

Summary 56

Chapter 3 The Plug-ins System 57

Understanding How Plug-ins Are Loaded 58

A Full-Featured Linker in Antivirus Software 58

Understanding Dynamic Loading 59

Advantages and Disadvantages of the Approaches for Packaging Plug-ins 60

Types of Plug-ins 62

Scanners and Generic Routines 63

File Format and Protocol Support 64

Heuristics 65

Bayesian Networks 66

Bloom Filters 67

Weights-Based Heuristics 68

Some Advanced Plug-ins 69

Memory Scanners 69

Non-native Code 70

Scripting Languages 72

Emulators 73

Summary 74

Chapter 4 Understanding Antivirus Signatures 77

Typical Signatures 77

Byte-Streams 78

Checksums 78

Custom Checksums 79

Cryptographic Hashes 80

Advanced Signatures 80

Fuzzy Hashing 81

Graph-Based Hashes for Executable Files 83

Summary 85

Chapter 5 The Update System 87

Understanding the Update Protocols 88

Support for SSL/TLS 89

Verifying the Update Files 91

Dissecting an Update Protocol 92

When Protection Is Done Wrong 100

Summary 101

Part II Antivirus Software Evasion 103

Chapter 6 Antivirus Software Evasion 105

Who Uses Antivirus Evasion Techniques? 106

Discovering Where and How Malware Is Detected 107

Old Tricks for Determining Where Malware Is Detected: Divide and Conquer 107

Evading a Simple Signature-Based Detection with the Divide and Conquer Trick 108

Binary Instrumentation and Taint Analysis 113

Summary 114

Chapter 7 Evading Signatures 117

File Formats: Corner Cases and Undocumented Cases 118

Evading a Real Signature 118

Evasion Tips and Tricks for Specific File Formats 124

PE Files 124

JavaScript 126

String Encoding 127

Executing Code on the Fly 128

Hiding the Logic: Opaque Predicates and Junk Code 128

PDF 129

Summary 131

Chapter 8 Evading Scanners 133

Generic Evasion Tips and Tricks 133

Fingerprinting Emulators 134

Advanced Evasion Tricks 136

Taking Advantage of File Format Weaknesses 136

Using Anti-emulation Techniques 137

Using Anti-disassembling Techniques 142

Disrupting Code Analyzers through Anti-analysis 144

More Anti-Anti-Anti. 147

Causing File Format Confusion 148

Automating Evasion of Scanners 148

Initial Steps 149

Installing ClamAV 150

Installing Avast 150

Installing AVG 151

Installing F-Prot 152

Installing Comodo 153

Installing Zoner Antivirus 154

MultiAV Configuration 154

peCloak 158

Writing the Final Tool 160

Summary 162

Chapter 9 Evading Heuristic Engines 165

Heuristic Engine Types 165

Static Heuristic Engines 166

Bypassing a Simplistic Static Heuristic Engine 166

Dynamic Heuristic Engines 173

Userland Hooks 173

Bypassing a Userland HIPS 176

Kernel-Land Hooks 178

Summary 180

Chapter 10 Identifying the Attack Surface 183

Understanding the Local Attack Surface 185

Finding Weaknesses in File and Directory Privileges 185

Escalation of Privileges 186

Incorrect Privileges in Files and Folders 186

Incorrect Access Control Lists 187

Kernel-Level Vulnerabilities 187

Exotic Bugs 188

Exploiting SUID and SGID Binaries on Unix-Based Platforms 189

ASLR and DEP Status for Programs and Binaries 190

Exploiting Incorrect Privileges on Windows Objects 193

Exploiting Logical Flaws 196

Understanding the Remote Attack Surface 197

File Parsers 198

Generic Detection and File Disinfection Code 199

Network Services, Administration Panels, and Consoles 199

Firewalls, Intrusion Detection Systems, and Their Parsers 200

Update Services 201

Browser Plug-ins 201

Security Enhanced Software 202

Summary 203

Chapter 11 Denial of Service 207

Local Denial-of-Service Attacks 208

Compression Bombs 208

Creating a Simple Compression Bomb 209

Bugs in File Format Parsers 212

Attacks against Kernel Drivers 213

Remote Denial-of-Service Attacks 214

Compression Bombs 214

Bugs in File Format Parsers 215

Summary 215