1.3 The Beowulf Cluster

The most commonly accepted definition of a Beowulf Cluster is :

• Built with commodity-of-the-shelf (COTS) hardware and components
• A private compute TCP/IP network (although today the use of proprietary high-speed, high-bandwidths interconnects such as Myrinet, Infiniband and Quadrics is becoming common)
• Use of Linux as the operating system
• Supports message passing programs build with MPI/PVM

Figure 1-2 Schematic layout of a typical cluster and its components.


A Head Node (also commonly called a frontend, master or management node) with two network interface cards (NIC), with eth1 connecting to the public network and eth0 connecting to the internal and private cluster network. The compute nodes are connected via eth0.

For certain applications, a separate high-speed, high-bandwidth application-message passing network may be necessary. This is typically Gigabit ethernet, Myrinet, Quadrics or Infiniband.

Typical usage of the Beowulf Cluster will have the user login into the head node of the cluster, and edit, compile his code or run his application. Running a HPC application typically involve the use of a batch scheduler such as GridEngine, Torque, Platform LSF or LAVA. The use of a batch scheduler ensures fair access to the cluster, better utilization of the cluster and easier manageability of jobs. The head node does not usually participate in the computation.

The freely available and open source operating system (OS) – Linux, is the most commonly used OS for Beowulf Clusters. There are clusters build with Solaris, FreeBSD and Windows, but these are less common.

Early Beowulf Clusters were built from desktop PCs, however with the availability of low-cost 1U and 2U dense servers and Blade servers, more and more system integrators, original equipment manufacturers (OEM), and do-it-yourself (DIY) Beowulf Cluster builders are opting for these instead.

From a software software perspective, a cluster consists of the Linux operating system a cluster scheduler and middleware which integrates and presents the cluster as a single entity, a parallel programming environment, application libraries such as MPI, cluster management tools and drivers. This is shown below.



Figure 1-3: The HPC Cluster Software Stack



1.4 Digression: Network of Workstation and alternatives

NOW
Network of Workstation (NOW) is the concept of aggregating the spare and idle CPU cycles of the many desktop PCs found in many organizations such as computing laboratory or classroom. These PCs are often not dedicated to the computing tasks and often only suitable for what is commonly called embarrassingly parallel applications. Middleware or scheduler softwares that enable these NOW include Platform LSF, UD and Condor which runs well on PCs running Microsoft Windows and Linux. Jobs are submitted through Condor for example, and Condor will hunt for a suitable resource and schedule the jobs to be run on an idle PC.

OpenMosix
The OpenMosix project is another form of PC class cluster system. It is a Linux kernel extension to make a network of PCs or servers to behave as a single-system (what is known as single-system image). Processes can migrate from one node to another node depending on the load of the nodes and this optimization of all the resources is happening continuously.

Applications need not be re-written to run on an OpenMosix cluster. As in NOW, OpenMosix clusters are suitable for embarrassingly parallel applications or serial applications (which then gets load-balanced automatically to a least loaded node).

OpenMosix is a Linux only solution and as it requires a kernel extension, many commercial ISV applications is not officially supported (yet).

OpenMosix clusters are suitable for organizations running only Linux applications and program code which are not from commercial ISVs who typically certify their applications only on Red Hat Linux or SuSe Linux. We see the biggest obstacle to OpenMosix (or similar technology) adoption in the enterprise and commercial market is the fact that a non-standard Linux kernel is used.

Note: The openMosix Project has officially closed as of March 1, 2008.


BProc
The Beowulf Distributed Process Space or BProc project is based on a set of kernel modifications, utilities and libraries to present to the user a single-system-image cluster.

Users can start processes remotely in a Beowulf-styled cluster without rsh, rlogin or ssh mechanism. These remotely started process appear on the head node process table in the cluster via BProc's global PID mechanism. So a listing of the Linux TOP command shows both the head node processes and all remote processes started with BProc.

This global PID mechanism allows for remote process management using the standard UNIX process control facilities.

In the BProc convention, the head node is the machine distributing pieces of its process space and the machines accepting pieces of it to run are the compute nodes.

As the BProc cluster makes use of kernel modifications, the support of commercial HPC applications can be limited. There are work-arounds to many commercial applications to make it work on a BProc cluster. It is a good system to consider if you are highly experienced with Linux and can do without commercial ISV support if you are using their software. It is also ideal if you run mostly applications written by yourself or which you have access to source code.

__________________
Laurence Liew
HPCCommunity.org Manager