Network Your Shell Scripts with Netpipes

by Robert Bernier
05/27/2004

There's no advantage to a sysadmin programming his own utility when he can do the same job (though perhaps less efficiently) in a quarter of the time with a quick and dirty shell script.

Unfortunately, bash scripts can't solve all problems. Take, for example, making socket connections to send and receive data streams. Conventional wisdom states that you'll have to use an existing service such as FTP, or create your own using a programming language such as Perl or C. I don't like these solutions: I don't like the idea of having a service that only one person uses. I don't like using full-blown programming languages on a server just to make a couple of simple socket connections. What I want is a command-line utility that anybody can use to set up TCP connections without worrying about root privileges. I want something that works with standard input and output, is easy to use in a script, and has a permissions system to restrict who can use it.

netpipes is a suite of utilities for shell-script writers that builds on the idea of conventional pipes to allow different processes to communicate and share data using both TCP and Unix domain-based sockets across the network! Not only does it duplicate the pipe's behavior, but it uses a novel technique called Session Control Protocol (SCP) that provides a simple mechanism for creating multiple, lightweight connections over a single TCP session connection. You can have many datastreams at the same time instead of just one.

This article will introduce you to a few of the netpipes command line utilities. Refer to the man pages for more information.

There are six command-line utilities in the netpipes suite:

faucet: The server end of a BSD network pipe.
hose: The client end of a BSD network pipe.
encapsulate: Multiplexes several datastreams or channels over a single channel.
timelimit: Caps how long a spawned subprocess can exist.
sockdown: Selectively shuts down all or a portion of a socket connection.
getpeername (or getsockname, in Debian): Returns information about the socket connection.

Installing netpipes from Debian's testing distribution is easy enough:

apt-get install netpipes

Netpipes is also available from the FreeBSD ports collection.

`faucet`

This utility runs the server end. This example invocation sets up a server connection on port 2000 and gives a simple "hello world."

~$ faucet 2000 --out echo hello world

The --out switch means that the server will spit out the message hello world as soon as a connection is made. You can also run this in the background as a daemon by adding the --daemon switch:

~$ faucet 2000 --out --daemon echo hello world

We can test this out using telnet:

~$ telnet localhost 2000
Trying 127.0.0.1...
Connected to wolf.
Escape character is '^]'.
hello world
Connection closed by foreign host.

The server can also carry out more than one instruction:

~$ faucet 2000 --out sh -c 
	"echo hello world; echo hi again; sockdown 1 1; ls / > /tmp/temp.txt"

This example echoes two statements to the client, hello world and hi again. It then closes the output connection using sockdown, another netpipe utility. The next instruction sends the output to the server's own file system in the file temp.txt.

Note: It's a good idea to enclose multiple commands within the bash shell command sh -c to avoid confusion.

`hose`

hose is the client utility. We can contact the first server example in the following manner:

~$ hose localhost 2000 --in cat
hello world
~$

The --in switch means that the client expects incoming information from the server. cat is of course the concatenate utility that takes input and puts it out as standard output.

The client can also send a datastream to the server. The server echoes the string hello it's my turn.

server: ~$ faucet 2000 --in sh -c "cat"
client: ~$ hose localhost 2000 --out sh -c "echo -e hello it\\047s my turn"

`hose` and `faucet` Combinations

This table shows several examples of simple piping -- one-way communications between the client and server.

Server (`faucet`)	Client (`hose`)	result
`faucet 2000 --out echo hello world`	`hose localhost 2000 --netslave`	hello world (comes out on the server's terminal console)
faucet 2000 --in echo I am hit, `date`	`hose localhost 2000 --netslave`	I am hit, Tue Mar 9 18:51:31 EST 2004 (comes out on the client's terminal console)
`faucet 2000 --in sh -c "grep bernier - > temp1.txt"`	`hose localhost 2000 --in --out sh -c "ps aux"`	Creates a file called temp1.txt on the server with information provided by the client.
`faucet 2000 --out sh -c "ps aux"`	`hose localhost 2000 --in --out sh -c "cat <&0>temp2.txt`	Creates a file called temp2.txt on the client with information provided by the server. Notice the use of redirection in `cat <&0> temp2.txt`.

Two-Way Server/Client Communications

Piping feeds the output of one process as input to another, but lacks the ability of running bidirectional communications. You can, however, create two-way data flow.

netpipes permits you to choose whether it's the client or server that sends, or receives, data upon connection by using the --in and --out switches in a specific order on both the client and server commands. For example, if the client sends data as soon as it connects to the server, then the first switch, reading from left to right, is --out. The server must complement this by using the --in switch.

Pages: 1, 2