Malware Analysis Patterns
Common Behaviors
Persistence Mechanisms
- •Registry Run keys (Windows)
- •Scheduled tasks / cron jobs
- •Service installation
- •DLL hijacking
- •Boot sector modification
Anti-Analysis Techniques
- •Debugger detection (ptrace, IsDebuggerPresent)
- •VM detection (CPUID, registry, timing)
- •Sandbox detection (user activity, artifacts)
- •Code obfuscation
- •Packed/encrypted payloads
C2 Communication
- •HTTP/HTTPS beaconing
- •DNS tunneling
- •Custom protocols
- •Domain generation algorithms (DGA)
- •Dead drops (pastebin, social media)
Analysis Techniques
String Decryption
XOR decryption:
python
def xor_decrypt(data, key):
return bytes([b ^ key[i % len(key)] for i, b in enumerate(data)])
# Single byte key
decrypted = xor_decrypt(encrypted, b'\x42')
# Multi-byte key
decrypted = xor_decrypt(encrypted, b'secret')
RC4 decryption:
python
def rc4_init(key):
S = list(range(256))
j = 0
for i in range(256):
j = (j + S[i] + key[i % len(key)]) % 256
S[i], S[j] = S[j], S[i]
return S
def rc4_crypt(S, data):
S = S.copy()
i = j = 0
result = []
for byte in data:
i = (i + 1) % 256
j = (j + S[i]) % 256
S[i], S[j] = S[j], S[i]
result.append(byte ^ S[(S[i] + S[j]) % 256])
return bytes(result)
Payload Extraction
LD_PRELOAD hooking:
c
// hook_memfd.c
#define _GNU_SOURCE
#include <dlfcn.h>
#include <stdio.h>
int memfd_create(const char *name, unsigned int flags) {
int (*real_memfd)(const char*, unsigned int) = dlsym(RTLD_NEXT, "memfd_create");
int fd = real_memfd(name, flags);
fprintf(stderr, "[HOOK] memfd_create('%s') = %d\n", name, fd);
return fd;
}
ssize_t write(int fd, const void *buf, size_t count) {
ssize_t (*real_write)(int, const void*, size_t) = dlsym(RTLD_NEXT, "write");
// Dump to file for analysis
FILE *f = fopen("/tmp/captured_payload", "ab");
fwrite(buf, 1, count, f);
fclose(f);
return real_write(fd, buf, count);
}
Compile and use:
bash
gcc -shared -fPIC -o hook.so hook.c -ldl LD_PRELOAD=./hook.so ./malware
Memory Forensics
Volatility 3 basics:
bash
# List processes vol -f memory.dmp windows.pslist # Dump process memory vol -f memory.dmp windows.memmap --pid 1234 --dump # Find injected code vol -f memory.dmp windows.malfind
Indicators of Compromise (IOCs)
Network IOCs
- •IP addresses
- •Domain names
- •URLs
- •User-Agent strings
- •JA3/JA3S fingerprints
Host IOCs
- •File hashes (MD5, SHA256)
- •File paths
- •Registry keys
- •Mutex names
- •Service names
Documentation Template
markdown
## IOCs ### Network | Type | Value | Context | |------|-------|---------| | IP | 192.168.1.100 | C2 server | | Domain | evil.com | DGA generated | ### Host | Type | Value | Context | |------|-------|---------| | Hash | abc123... | Main payload | | Path | /tmp/.hidden | Dropped file | | Mutex | Global\XYZ | Infection marker |
Anti-Debug Bypass
ptrace Detection
c
// Malware checks:
if (ptrace(PTRACE_TRACEME, 0, 0, 0) == -1) {
exit(1); // Being debugged
}
// GDB bypass:
(gdb) catch syscall ptrace
(gdb) commands
> set $rax = 0
> continue
> end
Timing Checks
c
// Malware checks: t1 = rdtsc(); // ... code ... t2 = rdtsc(); if (t2 - t1 > threshold) exit(1); // Debugger detected // Bypass: Patch comparison or NOP the check