TriSentry, a Unix Intrusion Detection System

by Glenn Graham
11/14/2002

Intrusion detection isn't anything new. Throughout the history of computer networking, administrators have worked to find ways of tracking system security breaches and identifying the culprits behind them. Network administrators have a wide range of sophisticated tools to improve auditing, and to report and block intrusion. The TriSentry suite, from Psionic Technologies, is one such free tool.

TriSentry is composed of three applications. PortSentry detects, reports, and blocks port scanning. LogSentry examines system logs for security violations and generates administrative reports. HostSentry monitors user logins and activities, detecting breaches and unusual behavior.

PortSentry

Port scanning, or using an automated tool to probe a host for open TCP ports, is increasingly common on the Internet. Though intended to help administrators secure their networks, port scanners may also be used by malicious users to find vulnerable services. Common exploits include daemons such as portmapper and various database applications.

The PortSentry application detects and blocks these scans. It works well on Unix and Unix-like systems. Using the principle of socket binding, PortSentry listens on a series of unused ports. If a remote host attempts to connect, its IP address is added to a blackhole list, and cannot continue its scan or make any other TCP requests. Blocking is performed either through the appropriate firewall software (ipchains or iptables on Linux, ipfw on *BSD) or through the routing table (the route utility on most Unixes).

Installing and Configuring PortSentry

Download the PortSentry distribution from the Psionic site. As with all new installations, review the README file for the latest instructions.

Pass the OS name (e.g. freebsd, linux, or solaris) to the make command to configure the build. For a complete list of available options, type make with no argument. Executing make install as the root user will install the binaries and configuration files with the appropriate permissions.

PortSentry is also available through the FreeBSD ports collection, in /usr/ports/security/portsentry.

After installing, edit the configuration file. By default, it's located at /usr/local/etc/portsentry.conf. This file is well-commented; configuration is vital to the correct operation or PortSentry. For our example, we'll set the listening ports and configure the firewall and route blocking options for a FreeBSD box with IPFW compiled into the kernel. It's important to remember that when using the firewall option under FreeBSD, you must have IP_FW compiled into the kernel.

PortSentry will listen for TCP and UDP packets on all ports specified in the TCP_PORTS and UDP_PORTS directives, respectively. The ports must be expressed as a list of comma-separated values, with no embedded spaces. This example has had lines broken to fit on the page, but represents one logical line per directive:

TCP_PORTS="1,7,9,11,15,70,79,109,111,119,138,139,143,\
   512,513,514,515,540,635,1080,1524,2000,2001,4000,4001,5742,6000,\
   6001,6667,12345,12346,20034,27665,30303,32771,32772,32773,32774,\
   31337,40421,40425,49724,54320"

UDP_PORTS="1,7,9,66,67,68,69,111,137,138,161,162,474,513,517,518,\
   635,640,641,666,700,2049,31335,27444,34555,32770,32771,32772,\
   32773,32774,31337,54321"

In order to block malicious users with a kernel-level packet filter, use the KILL_ROUTE directive. PortSentry will replace the string $TARGET$ with the source address, executing the result as a standard command. On FreeBSD with ipfw, the configuration will resemble:

kill_route="/sbin/ipfw add 1 deny all from $TARGET$:255.255.255.255 \ 
   to any"

In the absence of some sort of packet filtering, KILL_ROUTE can use the venerable route command as a last-ditch backup. It will only work for TCP attacks, though. The command will look something like:

KILL_ROUTE="route add -net $TARGET$ -netmask 255.255.255.255 \
   127.0.0.1 -blackhole"

If you have iptables, ipchains, pf, or ipfw, use that instead. See the copious comments in the configuration file for more details.

The KILL_RUN_CMD directive is also useful. It specifies a command to run after blocking a scanner. I use a custom notification alarm to mail the system administrator. It's a simple shell script, in /usr/local/etc/notify. In the interests of security, it's owned by root and wheel, and has no write permissions.

#!/bin/sh

echo "My computer has been attacked" | \
   mail -s "Strobe Attack on My System" you@your-email.com

Enabling this in portsentry.conf is as you'd expect:

KILL_RUN_CMD="/usr/local/etc/notify"

This option may also be used to execute a return attack on the host, although I don't recommend doing this :-)

Occasionally, we need to allow certain hosts the ability to scan our system without being blacklisted. PortSentry checks the file portsentry.ignore for these hosts. It should normally contain entries for 127.0.0.1 and localhost. TCP-based blocked hosts are added to portsentry.blocked.tcp and UDP-based blocks are added to Portsentry.blocked.udp. Their contents will persist until the next system restart. Finally, the portsentry.history file contains a running history of all blocked hosts. This file remains constant even after a reboot. It may also be exported to a database for further analysis.

Starting and Testing PortSentry

Starting PortSentry is easy:

/usr/local/bin/portsentry -tcp &

This example uses the classic TCP detection mode with the -tcp switch. Other options may be added. For example, -sctp enables Stealth Scan mode, on Linux. -udp enables UDP mode.

Once PortSentry is running, you should test it. Various port scanners I use are nmap, ISS (Internet Security Scanner), and Strobe (available with Slackware). We will use nmap for this example.

NOTE: If the test succeeds, you will not be able to reconnect from the probing host. Plan ahead!

Scanning with nmap is easy:

nmap -O target_host

To ensure the program has blocked the attacking IP, try re-connecting to the target host. It should fail. On the host, tail the message log to ensure the attacker is identified and blocked:

Sep 19 01:50:19 striker portsentry[129]: attackalert: \
   Host 192.168.0.1 has been blocked via dropped route using command: \
   "/sbin/ipfw add 1 deny all from 192.168.0.1:255.255.255.255 to any"
Sep 19 01:50:19 striker portsentry[129]: attackalert: \
   Connect from host: 192.168.0.1/192.168.0.1 to TCP port: 9
Sep 19 01:50:19 striker portsentry[129]: attackalert: \
   Host: 192.168.0.1 is already blocked. Ignoring
...

When using route to drop the IP, you can examine the routing table with netstat --nr. Blocked hosts will show routes pointing to the IP you designated as your blackhole.

PortSentry 2 Betas

The recent releases of PortSentry beta versions 2.0b have seen some notable improvements. These beta versions still require some work, but are worth playing with if you need these features:

Stealth scan detection for all Unix platforms
Elimination of "Classic" port binding
Expansion for upcoming packet anomaly detection routines

Download PortSentry 2.0b1 from the Psionic Web site. (It is not yet available as a BSD port.) You will need to edit the portsentry_config.h file to match the file paths on your system. Follow the standard compilation and configuration instructions for version 1.1. (Remember to pass your architecture name to the make command.)

If you run into compilation errors during your build, re-check the paths in portsentry_config.h and compile again. This compilation is known to work under Linux and FreeBSD. Be sure to read the README file included in the installation.

Summing up PortSentry

I've been using PortSentry since its early release and find it to be an excellent complement to the TriSentry suite of applications. Though I prefer version 1.1 in a production environment, I have confidence that version 2 will become a favorite when it is officially released.

Pages: 1, 2