HA-OSCAR: Five Steps to a High-Availability Linux Cluster

by Ibrahim Haddad and Chokchai Leangsuksun, Stephen L. Scott
02/03/2005

The HA-OSCAR project's primary goal is to improve the existing OSCAR, Beowulf architecture, and cluster management technology systems (including OSCAR, ROCKS, and Scyld) while providing high-availability and scalability capabilities for Linux clusters. The OCG recognized the project as an official working group, along with the current OSCAR and Thin-OSCAR working groups. HA-OSCAR introduces several enhancements and new features to OSCAR, mainly in the areas of availability, scalability, and security. The new features in the initial release are head node redundancy and self-recovery for hardware, service, and application outages.

This document provides a systematic installation guide for system administrators, as well as a detailed explanation of what happens during the installation. This guide assumes familiarity with basic Linux administration commands. Prior knowledge of OSCAR installation and administration will be useful.

Supported Distributions and System Requirements

The HA-OSCAR team has tested HA-OSCAR to work with OSCAR 2.3, 2.3.1, and 3.0 based on Red Hat 9.0. The test environment for the installation discussed in this article is as follows:

Head node: Two dual Xeon 2.4GHz machines, each with 1GB of RAM, a 40GB HD, and two network interface cards (NICs)
Client node: Four dual Xeon 2.4GHz machines, each with 1GB of RAM, a 40GB HD, and two NICs
Switch: D-Link 10/100Mbps switch

This article assumes that you have built the cluster with OSCAR beforehand. If this is not the case, please refer to the OSCAR project page for the OSCAR installation procedure.

The primary and standby servers should have homogeneous hardware, and each server should have at least two network interface cards. The network interfaces must support PXE boot, and they must all connect to the local switch (two for redundancy purposes).

HA-OSCAR Architecture

Figure 1 illustrates the HA-OSCAR architecture.

Figure 1. HA-OSCAR architecture

HA-OSCAR consists of the following major system components:

A primary server, which receives and distributes requests to specified clients. Each server has three network interface cards: one connected to the Internet by a public network address, and the other two connected to a private LAN, which consists of a primary Ethernet LAN and a standby LAN. However, the beta release only supports one private NIC.
A standby primary server, which activates its services, monitors the primary server, and anticipates taking over for the primary server when it detects a failure in the primary server.
Multiple clients, dedicated to computation.
Local LAN switches, to provide local connectivity among head and client/compute nodes.

Each head node must have at least two NICs: eth0 and eth1. One of the NICs is a public interface to the outside network and the other is a private interface to its local LAN and towards computing nodes. The exact configuration depends on how a user wants to connect eth0 and eth1 to either the public or private network. Our example assumes that eth0 is a private interface and eth1 is the public interface.

Figure 2 shows the sample network configuration of a HA-OSCAR head node.

sample network configuration for head nodes
Figure 2. Sample HA-OSCAR network configuration for head nodes

Cluster Installation Procedure

The HA-OSCAR team has developed an easy-to-install package with a GUI interface. When the system has OSCAR installed, download HA-OSCAR.

You must be root to be able to install HA-OSCAR. Once you uncompress the package, start the installation by typing the following command:

% ./haoscar_install <interface>

The interface directive is the private network interface for the primary head, normally eth0.

The installation wizard should pop up (as shown in Figure 3). The HA-OSCAR installation wizard will walk the user through a complete installation process consisting of the following steps:

Installing the HA-OSCAR package (SW staging on the primary head).
Building a standby server image (cloning the primary server).
Configuring the initial standby server.
Setting up the network and creating a boot image on the standby server.
Completing the installation.

The following sections describe the HA-OSCAR wizard installation process and provide visuals for the associated screens.

HA-OSCAR Package Installation

In step 1, this wizard will install all of the required packages to the OSCAR cluster server and prepare the environment.

Figure 3. HA-OSCAR installation wizard

Fetching (Cloning) Image for Standby Server

The first step will take less than one minute to complete. Step 2 is for building a standby server image from the primary node. When you click the button Building Image for Standby server, the wizard will pop up another window requesting a server image name. Normally, you can leave the default value and just press the Fetch image button (shown in Figure 4) to fetch an image for the standby server. This step will take several minutes.

This is an important step in cloning a standby server image from a primary one. For a stringent downtime requirement, we recommend a separate image server for image repository and recovery purposes.

Figure 4. Fetching or cloning a server image