Security Onion – from traffic to analyst

In the past months I’ve been using Security Onion in relation to one of my school projects and lately to my internship. Security Onion has a lot of useful programs, on which you could literally spend days to configure to work properly on the same server. The fact that it just works, does not save you the headaches of figuring out how it works and tweaking it to suit your needs. (but it saved me from a lot of headaches caused by other issues).

In order to understand how it works I had to get the big picture in mind, and there’s no better way to do that than by a visual representation followed by an appropriate description. The present article will show and explain the roadmap that Security Onion creates between the network traffic that it’s monitoring to the user that analyzes and classifies the data.

so-architecture

Legend:

  • blue boxes – traffic capturing/analysis programs
  • geen boxes – user interfaces
  • yellow boxes – “intermediate” programs
  • circles – data formats
  • cylinders – databases

Description:

In order to understand how the system is working I’m going to split it down to 4 descriptions, related to the specific programs and the “chain” that they’re involved in.

Snorby – is a web interface built in ruby on rails that shows a nice overview of Snort alerts. It can be accessed by default on HTTPS on por t 444

  • snort processes the network traffic, saves some usage info in a snort.stat log file (/nsm/sensor_data/HOSTNAME-IFACE/snort.stats) and outputs the results in unified2 format (/nsm/sensor_data/HOSTNAME-IFACE/snort/snort.unified2.TIMESTAMP)
  • the unified2 log files are being processed by Barnyard2 which passes the information to syslog-ng, the Snorby database and to Sguil
  • Snorby uses the info from the Snorby database to represent it visually

ELSA – is a collection of perl scripts which uses syslog-ng, patterndb, sphinx search and mysql to gather massive amounts of logs and to query through them (really fast btw). It can receive logs from multiple location and in Security Onions case it receives logs from BRO, Snort and Syslog

  • the snort information that has been passed by Barnyard2 to syslog-ng is being processed by ELSA in realtime
  • BRO processes the network traffic and outputs the result in log files (/nsm/bro/logs/current/TYPE.log)
  • the log files location is defined as a log source in syslog-ng for ELSA, and ELSA processes them directly as they are being saved to disk by BRO
  • ELSA writes to the ELSA database, where it stores usage information (query history and others)

Sguil and Squert – these 2 work hand-in-hand, as Squert is a web-interface for Sguil which provides access to realtime events, session data, and raw packet captures

  • the snort information that has been passed by Barnyard2 to Sguil is being shown in Sguils GUI
  • Sguil writes the information into the Sguil database
  • Squert uses the information in the Sguil database to display information in the web-interface

Other

  • netsniff-ng saves the network traffic from the monitoring interface to pcap files (nsm/sensor_data/HOSTNAME-IFACE/dailylogs/DATESTAMP) that have a a default size of 150MB
  • capME is being by Snorby to get from a specific alert to the traffic that generated it