Preface
PartⅠ.Data
1.Sensors and Detectors: An Introduction
Vantages: How Sensor Placement Affects Data Collection
Domains: Determining Data That Can Be Collected
Actions: What a Sensor Does with Data
Conclusion
2.Network Sensors
Network Layering and Its Impact on Instrumentation
Network Layers and Vantage
Network Layers and Addressing
Packet Data
Packet and Frame Formats
Rolling Buffers
Limiting the Data Captured from Each Packet
Filtering SpeciFic Types of Packets
What Iflt''s Not Ethernet?
NetFlow
NetFlow v5 Formats and Fields
NetFlow Generation and Collection
Further Reading
3.Host and Service Sensors: Logging Traffic at the Source
Accessing and Manipulating LogFiles
The Contents of Logfiles
The Characteristics of a Good Log Message
Existing Logflles and How to Manipulate Them
Representative Logflle Formats
HTTP: CLF and ELF
SMTP
Microsoft Exchange: Message Tracking Logs
Logfile Transport: Transfers,Syslog,and Message Queues
Transfer and Logfrle Rotation
Syslog
Further Reading
4.Data Storage for Analysis: Relational Databases,Big Data,and Other Options
Log Data and the CRUD Paradigm
Creating a Well—Organized Flat File System: Lessons from SiLK
A Brieflntroduction to NoSQL Systems
What Storage Approach to Use
Storage Hierarchy,Query Times,and Aging
PartⅡ.Tools
5.The SiLK Suite
What Is SiLK and How Does It Work?
Acquiring and Installing SiLK
The DataFiles
Choosing and Formatting Output Field Manipulation: rwcut
Basic Field Manipulation: rwfrlter
Ports and Protocols
Size
IP Addresses
Time
TCP Options
Helper Options
Miscellaneous Filtering Options and Some Hacks
rwfileinfo and Provenance
Combining Information Flows: rwcount
rwset and IP Sets
rwuniq
rwbag
Advanced SiLK Faalities
pmaps
Collecting SiLK Data
YAF
rwptoflow
rwtuc
Further Reading
6.An Introduction to R for Security Analysts
Installation and Setup
Basics of the Language
The R Prompt
R Variables
Writing Functions
Conditionals and Iteration
Using the R Workspace
Data Frames
Visualization
Visualization Commands
Parameters to Visualization
Annotating a Visualization
ExportingVisualization
Analysis: Statistical Hypothesis Testing
Hypothesis Testing
Testing Data
Further Reading
7.Classification and Event Tools: IDS,AV,and SEM
How an IDS Works
Basic Vocabulary
Classifler Failure Rates: Understanding the Base—Rate Fallacy
Applying ClassiFication
Improving IDS Performance
Enhancing IDS Detection
Enhanang IDS Response
Prefetching Data
Further Reading
8.Reference and Lookup: Tools for Figuring Out Who Someone ls
MAC and Hardware Addresses
IP Addressing
IPv4 Addresses,Theu Structure,and Significant Addresses
IPv6 Addresses,Their Structure and Significant Addresses
Checking Connectivity: Using ping to Connect to an Address
Tracerouting
IP Intelligence: Geolocation and Demographics
DNS
DNS Name Structure
Forward DNS Querying Using dig
The DNS Reverse Lookup
Using whois to Find Ownership
Additional Reference Tools
DNSBLs
9.More Tools
Visualization
Graphviz
Communications and Probing
netcat
nmap
Scapy
Packet Inspection and Reference
Wireshark
GeoIP
The NVD,Malware Sites,and the C*Es
Search Engines,Mailing Lists,and People
Further Reading
PartⅢ.Analytics
10.Exploratory Data Analysis and Visualization
The Goal of EDA: Applying Analysis
EDA Workflow
Variables and Visualization
Univariate Visualization: Histograms,QQ Plots,Boxplots,and Rank Plots
Histograms
Bar Plots(Not Pie Charts)
The Quantile—Quantile(QQ)Plot
The Five—Number Summary and the Boxplot
Generating a Boxplot
Bivariate Description
Scatterplots
Contingency Tables
Multivariate Visualization
Operationalizing Security Visualization
Further Reading
11.On Fumbling
Attack Models
Fumbling: Misconfiguration,Automation,and Scanning
Lookup Failures
Automation
Scanning
Identifying Fumbling
TCP Fumbling: The State Machine
ICMP Messages and Fumbling
Identifying UDP Fumbling
Fumbling at the Service Level
HTTP Fumbling
SMTP Fumbling
Analyzing Fumbling
Building Fumbling Alarms
Forensic Analysis of Fumbling
Engineering a Network to Take Advantage of Fumbling
Further Reading
12.Volume and Time Analysis
The Workday and Its Impact on Network Traffic Volume
Beaconing
File Transfers/Raiding
Locality
DDoS,Flash Crowds,and Resource Exhaustion
DDoS and Routing Infrastructure
Applying Volume and Locality Analysis
Data Selection
Using Volume as an Alarm
Using Beaconing as an Alarm
Using Locality as an Alarm
Engineering Solutions
Further Reading
13.Graph Analysis
Graph Attributes: What Is a Graph?
Labeling,Weight,and Paths
Components and Connectivity
Clustering Coeffiaent
Analyzing Graphs
Using Component Analysis as an Alarm
Using Centrality Analysis for Forensics
Using Breadth—First Searches Forensically
Using Centrality Analysis for Engineering
Further Reading
14.Application Identification
Mechanisms for Application Identification
Port Number
Application Identiflcation by Banner Grabbing
Application Identification by Behavior
Application Identification by Subsidiary Site
Application Banners: Identifying and Classifying
Non—Web Banners
Web Client Banners: The User—Agent String
Further Reading
15.Network Mapping
Creating an Initial Network Inventory and Map
Creating an Inventory: Data,Coverage,and Files
Phase Ⅰ: The First Three Questions
Phase Ⅱ: Examining the IP Space
Phase Ⅲ: Identifying Blind and Confusing Traffic
Phase Ⅳ: Identifying Clients and Servers
Identifying Sensing and Blocking Infrastructure
Updating the Inventory: Toward Continuous Audit
Further Reading
Index