ArgusEye Documentation

Table of Contents

About this document
Installation
Required Perl modules
Additional tools required
Configuration
arguseye.ini
argusconfig.ini
Colors
Preferences dialog
Selecting Files
File type detection
Reading files
Ra input filter and time ranges
Reading TCP flags
Reading user captured data bytes
Writing an output file
Argus Management
Configuration Manager
Configuration File Editor
SSH Configuration
Security Considerations
Argus Live Reading
Transaction views
Column configuration
Details window
Hostname and whois lookups
Marker feature
Display filters
Generate filter expressions
Performance considerations
Reading large files
Why Perl?

About this document

This is the documentation for ArgusEye.

Author: Philipp E. Letschert <phil@datenspionage.de>

License: GNU Free Documentation License

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation.

Installation

There is no special installation routine necessary. Just unpack the archive and copy the folder to the desired location (e.g. /usr/local/arguseye). You can start ArgusEye by executing arguseye from any place in the filesystem.

ArgusEye does not need extended user privileges to run (as long as you can access logfiles from a normal user account).

Required Perl modules

ArgusEye depends on Perl Modules that are not part of the standard Perl distribution. Most vendors have ready-to-use packages of additional modules available which can be installed using a package manager.

Another way (recommended) to install/upgrade additional Perl modules is using the CPAN shell (http://www.cpan.org/):

perl -MCPAN -e shell
...
install Glib
install Gtk2
install Gtk2::GladeXML
install Gtk2::Ex::Utils
install Gtk2::Ex::Dialogs
install Gtk2::Ex::Simple::List
install Config::IniFiles
install Parse::Yapp

Please note, that installation of the Glib and Gtk2 Perl modules from source requires a compiler and the packages libglib2.0-dev and libgtk2.0-dev present on your system.

Additional tools required

ArgusEye depends on some additional system commands:

  • file(1), gzip(1) and bzip2(1) for determining the input file type and reading compressed files

  • ra(1) and racount(1) from the argus-clients distribution for reading Argus files.

  • argus(8) from the argus distribution for reading tcpdump files.

  • ssh(1) for remote Argus management

Configuration

ArgusEye uses the .arguseye directory in the users home directory to store its configuration. In this directory you will find the following files:

  • arguseye.ini: Plain text configuration file.

  • argusconfig.ini: Configuration of Argi for live-reading and management

  • fields_order, filters_display, filters_ra and filters_time: This is serialized Perl data, which is automatically generated on changes within the GUI. Do not edit theese files manually.

arguseye.ini

You can edit this file by hand or use the 'Preferences' dialog from the GUI. New versions of ArgusEye may have a different configuration file, in this case your previous configuration will be backed up with the version number suffixed.

The default configuration file looks like this:

#
# ArgusEye configuration file
#
[arguseye]
version=0.0.4
debug=0
read_tcp_flags=0
read_user_data=0
digits=2

[argus]
ra=/usr/local/bin/ra
argus=/usr/local/sbin/argus
default_dir=/var/log/argus

[colors]
marker=fff7a2
tcp=b1d6e6
rtp=b1b9e6
rtcp=c7b1e6
udp=afffaf
icmp=ffafaf
ipv6-icmp=ffafaf
igmp=cdb6a3
man=cccccc
arp=ffc885

argusconfig.ini

This file holds the configuration settings of remote Argi. You can edit this file by hand or using the configuration manager. Like the arguseye.ini file the configuration is divided into sections, with the section name as name of the configuration. Each section has six available parameter:

#
# argusconfig.ini - sample argus configurations
#

# live reading and management enabled
[test.rc.39]
host=localhost
port=561
user=root
key=/home/arguseye/.ssh/localhost
argus=/usr/local/sbin/argus
config=/etc/argus/argus.conf

# only live reading, no management
[dotcom]
host=argus.dot.com
port=561
user=
key=
argus=
config=
  • host: Hostname or IP address of the Argus sensor.

  • port: Port on which the Argus listens for incoming connections from argus-clients programs. Default port is 561.

  • user: The user for the SSH connection to the remote host. The user should have privileges for starting and stopping argus (usually root).

  • key: Path to the SSH keyfile holding the private key for connecting to the remote Argus host.

  • argus: Path to the argus executable on the remote host.

  • config: Path to the argus configuration file on the remote host.

For live-reading you must specify host and port. For Argus management you must specify host, user, key, argus and config.

Colors

The transaction rows are colored, depending on the protocol values, specified in the [color] section of the configuration file. If no color for a specific protocol is found, the row is displayed white. You can add additional colors to the configuration file using the protocol name and a hex color string (like a HTML color, but without the preceding '#').

Preferences dialog

The 'Edit->Preferences' dialog can be used to configure ArgusEye:

The 'Settings' pane holds the options, that can be configured in arguseye.ini.

In the 'Display Fields' pane you can configure the fields (columns), that are displayed when a new transaction view opens. The 'Defaults' button gives a hardcoded value, that is similar to the default output of the ra(1) command-line.

Selecting Files

ArgusEye supports reading of the following file-formats:

  • Argus Data Files (compress, gzip and bzip2 compressed)

  • tcpdump capture files (e.g. Wireshark, compress, gzip and bzip2 compressed)

You can select multiple files via the 'Select Files' button or 'File->Select Files' menu:

File type detection

If the Argus file formats are not recognized by ArgusEye, the file(1) command does not know about these formats. You can manually add the missing descriptions to /etc/magic:

# Magic local data for file(1) command.
# Insert here your local magic data. Format is described in magic(5).

# Argus 2.0
8       belong          0xE5617ACB      Argus data
>32     byte            x               - version %d
>33     byte            x               \b.%d

# Argus 3.0
8       belong          0xE5712DCB      Argus data
>40     byte            x               - version %d
>41     byte            x               \b.%d

Reading files

When the files to read have been selected, you can start the reading process by clicking the 'Read Data' button or selecting 'Read Transactions' from the 'Transactions' menu. For each file ArgusEye starts ra(1) or argus(5) commands and parses the output. A progress bar at the bottom of the ArgusEye window reports on the state of this process.

You can stop the reading process at any point, using the 'Stop' button. A dialog will ask, if you want to view the partial retrieved data.

If you are interested in the syntax of the commands for debugging or scripting reasons, you can check the 'Print debug output to stdout' option in the Preferences dialog: All external Argus commands are now printed to the console window before execution.

Ra input filter and time ranges

You can limit the number of transactions to be read by using a ra(1) input filter and time range:

The ra(1) filter syntax is validated while typing. This feature is work-in-progress, so an invalid syntax (indicated red) does not necessarily mean, the expression will be rejected by ra(1) and vice versa. Therefore the state of that entry has no effect on wether the reading process is started or not. Please drop a line if you find any oddities.

All expressions, that have sucessfully executed, are stored in the ArgusEye configuration directory, and can be selected for later use from the drop-down menu of the filter entry.

An explanation of the filter syntax for transactions and time ranges can be found in the ra(1) man page.

Reading TCP flags

For TCP transactions Argus keeps track of TCP state changes and records the actual seen TCP flags. To read theese values, ArgusEye does additional runs of ra(1). You can enable/disable this option via the Read TCP flags and state changes in the Preferences dialog. Since this will increase file reading time, it is disabled by default.

The flag values can be found in the following three columns:

  • aflags: Argus state change flags

  • sflags: TCP source flags

  • dflags: TCP destination flags

The flags are displayed as a single character each. An expanded translation can be found in the 'Details' window of a transaction.

Reading user captured data bytes

If Argus is configured with the option ARGUS_CAPTURE_DATA_LEN set to a value > 0, Argus captures a number of user data bytes from the packet stream (see argus.conf(5) man page). If this data is available in the Argus file you can read it into ArgusEye by specifying a number to the 'Bytes of user data to read' option. Since this will increase file reading time, it is disabled by default.

The largest Argus record possible is 64K, so the combined user data capture can't be bigger than this, although practically it should be something that is useful. 128 bytes is a good choice.

Writing an output file

not implemented yet

Argus Management

You can use ArgusEye for management of remote Argi. Before you can connect to an remote host, you have to create a configuration for it. You can do this by editing the argusconfig.ini file as described above or by using the configuration manager.

Configuration Manager

The configuration manager opens by clicking the 'Argus' button in the main window, or by selecting 'File->Add Argus' from the menu:

A list, sorted by name, shows the available configurations. At first start the status of managed Argi is unknown. The status of all configurations is updated using the 'Update' button. You can add or edit configurations using the appropriate buttons:

You have to specify at least name, host and port to add a new entry to the list. This enables the live-reading feature for the specified host. All other entries are optional and only needed for management operations. If you have entered information for user, key, argus and config you can get the status information using the 'Test' button. If the test succeeded the Argus can be started/stopped/restarted using the appropriate buttons.

IMPORTANT: Management is only possible for Argus 3.x daemons. The remote Argus is started with the path to the configuration file as the only argument, so all your options have to be in the configuration file, no additional command-line arguments are supported.

Configuration File Editor

not implemented yet

SSH Configuration

ArgusEye has no own SSH functionality and depends on OpenSSH for connecting to remote hosts. ArgusEye does not use passwords, but keys for authentication. You have to set up the key configuration for remote hosts manually, before they can be used in ArgusEye. This is a well-known procedure for most network administrators and various tools exist for aiding in this process. A common setup looks like this:

## create a RSA key pair with a keylength of 2048 bits and empty passphrase
local$ ssh-keygen -t rsa -b 2048

Generating public/private rsa key pair.
Enter file in which to save the key (/home/arguseye/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/arguseye/.ssh/id_rsa.
Your public key has been saved in /home/arguseye/.ssh/id_rsa.pub.

## give some meaningful names to the keypair e.g. name of the remote host
local$ cd .ssh
local$ mv id_rsa remote_host
local$ mv id_rsa.pub remote_host.pub

## install the public key at the remote host
local$ scp remote_host.pub root@remote_host:.ssh/

local$ ssh root@remote_host

remote_host# cd .ssh
remote_host# cat remote_host.pub >> authorized_keys
remote_host# ^D

## back at the local host, try if connecting to the remote host is possible
## using the private key as identity
local$ ssh -i /home/arguseye/.ssh/remote_host root@remote_host
remote_host#

## if the prompt at the remote host comes up without authentication dialogs
## or prompting for a password, the setup is ready for use in ArgusEye

When 'Print debug output to stdout' is enabled in ArgusEye, all SSH commands used for management are printed to the terminal window before execution.

Security Considerations

Enabling root access, using SSH keys without a passphrase, has a serious impact on the security of the remote host. You should use this feature in ArgusEye only if you exactly know what you are doing. There are ways to restrict key usage and methods for protecting your private keys. Please refer to more specific SSH security guides, available on the web...

ArgusEye knows the locations of your private keys and stores them in the argusconfig.ini file. This should be mentioned here, even if a malicious attacker has far better methods for finding your private keys, than searching for this configuration file...

Argus Live Reading

If you have enabled remote access to Argus by setting ARGUS_ACCESS_PORT (see the argus.conf(5) manpage), you can select the appropriate entry from the configuration manager and click the 'Start' button. A new transaction view opens and updates with each new transaction from the sensor.

The transaction view is the same as for files, with the difference of three buttons in the top right corner:

  • 'Stop/Start' for stopping and restarting the live-reading process.

  • 'Reset View' for clearing all entries from the view

  • 'Toggle automatic scrolling' ...should be self-explanatory

Transaction views

If all files have read successfully, a new transaction view opens in the main window's notebook. You can open multiple views for different files and ra(1) filter combinations.

Column configuration

The columns in the transaction view represent the available ra(1) data fields. A longer description of the column fields meaning can be found via a tooltip on the column header. When a new transaction view opens, it displays the default column set, that has been configured in the 'Preferences->Display Fields' dialog.

Columns can be resized and sorted on left-click. You can also change the ordering of columns: Just hold down the left mouse button on a column header and drag the column to its new position.

You can change wich columns to be displayed by right-clicking on a column header:

The popup menu provides the following actions:

  • Remove Column (name): Removes the actual column from the view.

  • Show Default Columns: Shows the default column set, that has been configured via the 'Preferences->Display Fields' dialog.

  • Show All Columns: All available columns.

  • Add Column: Opens a submenu to select a single column from all columns, that are currently not displayed. The new column is inserted right after the column that was selected for opening the column configuration menu.

Details window

If you want to see the details of a single transaction without scrolling the whole row, you can open a Details window by double-clicking the transaction row or selecting 'Show Details' from the menu:

The details window shows only the information that actually exists, e.g. an entry for 'Loss' is only visible if loss was recorded for that transaction. The information for 'Argus TCP state changes' and 'TCP flags' only show up when the file was read with the option 'Read TCP flags and state changes' set in the 'Preferences->Settings' dialog. The information for 'User captured data bytes' only show up, if the file was read with the option 'Bytes of user data to read' set to a value > 0 in the 'Preferences->Settings' dialog, and either the suser or duser field is not empty.

Hostname and whois lookups

If you right-click on a cell, that contains an IP address the additional menu items 'nslookup <IP>' and 'whois <IP> show up. On activation, the corresponding system commands with their default server configuration are run for that IP. If a result is available, a new window opens displaying the result. Since the lookups may take some time, the commands are run in background and it's possible to queue additional lookups for other addresses.

Marker feature

Sometimes it is useful to observe transactions for specific IP addresses, without applying a display filter. To make this easier you can highlight single addresses with a marker feature. A marker can be set via right-click on a cell, that contains an IP address (saddr and daddr):

All cells that contain the selected IP address are highlighted with the configured 'marker' color. You can remove a marker by setting a new marker on a different address or selecting 'Remove Marker' from the menu.

Display filters

The transaction view display filter is work in progress. The filter syntax is different from the ra(1) filter syntax, and has few expressions yet. Later versions should have a display filter syntax, that matches the ra(1) filter syntax to increase filtering options and reduce confusion.

As with the Ra input filter, the syntax of the expression is validated on-the-fly.

At moment you can specify protocol, ports and IP-addresses. Expressions can be combined with 'and' or 'or'. Protocol identifiers include arp, rarp, esp, tcp, udp, rtp, rtcp, man and icmp. Ports can be specified via sport, dport and port (sport or dport). IP addresses can be specified via saddr, daddr and addr (saddr or daddr).

Examples:

  • tcp and dport 80 or udp and dport 53

  • daddr 141.1.1.1

  • icmp and arp and addr 141.1.

Generate filter expressions

For some protocols, it is possible to generate filter expressions for other programs via right-click and selecting 'Generate Filter Expressions'. A new window opens and displays the expressions for tcpdump(8), iptables(8) (Linux) and pf(4) (BSD):

This is a rather incomplete feature, its basic purpose is to have something for copy and paste, when the transaction information should be used for other programs.

Most notably, the tcpdump(8) filter syntax is compatible to the ra(1) syntax and you can just copy the expression to the 'Ra Filter Expression' entry and re-read your files.

Performance considerations

One goal for ArgusEye Development is 'scalable for enterprise use'. It is confirmed that Argus runs stable in large-scale scenarios and reports Gigabytes of log data per hour. It would be nice if ArgusEye could help in processing large data like that, what is definitely not possible at moment, so most network administrators have to rely on Argus command line tools and scripting.

At moment I am developing ArgusEye in a few rather small network environments. It is important to recieve feedback of users from large-scale networks, who would like to use a GUI for their daily work with Argus, to learn what requirements for larger data exist. So please drop a line, if you would like to help in development!

Reading large files

The current way of reading files in ArgusEye is opening the file with ra(1) and parsing the whole output into an array for storage. Depending on file size there are two implications:

  • The speed of processing files depends on your CPU ressources available. Reading a file may take several minutes. You save a little bit of time when you configure ArgusEye not to read TCP flags und user captured data, which is the default.

  • The maximum file size depends on the available memory. Due to Perls rather large overhead for array data the situation is even worse. So for reading Argus logfiles larger than ~50MB you should specify a good input filter, to limit the number of transactions processed by ArgusEye. A good choice as a default is 'not man' to skip all Argus management records.

    An unconfirmed guess on the maximum number of transactions, that can be handled in an ArgusEye view is a few 100000. Your mileage may vary, but as soon as your system runs out of memory and starts swapping, Bad Things will happen.

Possible Solutions:

  • Porting ArgusEye to C/C++. This is a step I would consider, if everything else fails (See section 'Why Perl?'). Even in C/C++ problems would arise if you try to load billions of rows into a GtkTreeView.

  • Improve the design of the ArgusEye code and its Gtk2 details. I already made some effort to increase performance, but I feel there is still room for improvements. Unfortunately I don't have the experience on all of the little details involved and currently I'm happy if it just 'works'.

  • Only load transaction rows, that actually fit into the window. For this to work, a library would be needed, that allows reading forward and backward from arbitrary positions in an Argus file. But scrolling large files only wouldn't help, features for sorting and filtering are needed as well. Probably the Argus client tools can be of help here but at moment I don't have a complete idea of implementing this.

  • Using limits on the number of transactions read and displayed and load the missing parts on demand.

  • Other solutions?

Why Perl?

If performance is an issue, Perl is probably not the best choice. Perl data structures have a rather large memory footprint, Perl objects are slow and Perl has all the disadvantages of an interpreted language. So why is ArgusEye written in Perl?

  • On todays computer hardware, performance of different languages is no longer a big issue, as it was back in the older days. As Microsoft continues pushing the hardware requirements for the average home user and with quad-core CPUs available and 1 Gig of memory as a bare minimum, it's no real problem if the code executes a bit slower. This is especially true for applications that don't do low-level processing and are just GUI frontends.

  • Perl is nice for rapid prototyping. You can develop lots of new features in a short time with few lines of code. And if done right it is also usable for large and stable applications. (If you are an experienced Perl Hacker and see something terrible wrong in the code, please forgive me: I just don't know better. Please drop a line and report on possible improvements...)

  • Perl runs on many different operating systems. The code of ArgusEye will run on different platforms without modification (at least it should do, but in reality there are probably some pitfalls I'm currently not aware of, especially because ArgusEye makes some assumptions on the underlying Unix environment)

  • The perl-gtk team is doing a great job in porting Gtk2 and Glib to perl. Writing Gtk applications in Perl is just more fun, than in C or C++.

  • ArgusEye has by no means to be stuck to Perl. If you can't live with a Perl GUI for Argus and have the required experience and time for porting to C, please get in touch. That is project I wouldn't start on my own and without help...

  • Perl6 will fix everything!


generated from DocBook source at 2007-02-01+01:00 14:26:54+01:00