A VISION FOR CLUSTER SECURITY

Funded by TRECC, NCSA's Cluster Security research team is developing a software tool for visually monitoring the security of computing clusters.

Clusters of computers are used by businesses and research centers because they are capable of performing trillions of calculations each second, storing trillions of bytes of data, and communicating quickly over high-speed networks.

NVisionCC Main View. The color and shape of the tick mark indicates the security status of each node of the HPC cluster. Together, all the tick marks indicate the security posture of the cluster as a whole. The cluster administrator can drill down for specific information on a particular node by mousing over or clicking on a tick mark. view larger

Unfortunately, these same capabilities also make clusters an attractive target for malice and mischief, as hackers try to hijack the computing power to run their own calculations (such as password cracking), to launch denial-of-service attacks against Internet sites, or to warehouse unauthorized files (such as illicit copies of copyrighted material).

TRECC is currently funding and supporting efforts by NCSA's Cluster Security (Cluster-Sec) research team to address this need by researching and developing a software tool to monitor cluster security.

The Cluster-Sec team, led by NCSA senior systems security engineer Bill Yurcik, has developed NVisionCC. NVision is an innovative software tool that collects and synthesizes data from heterogeneous sources and presents the information through an easy-to-understand visual interface. (The "CC" denotes "cluster computing.") On a single screen, NVisionCC provides an overview of the cluster and generates alerts that pinpoint specific nodes where the data indicates a potential security breach. NVisionCC is currently installed on the 12-node cluster at TRECC's facility in West Chicago.

"We are grateful for the people who provide us the opportunity to use TRECC cluster as our testbed," says Yurcik. "We monitor processes running on the cluster, open ports as well as file change from each of the node in the cluster. Our preliminary results show that NVisionCC is an effective security tool for a system administrator to use in protecting a cluster with minimal performance impact on the user jobs."

The NVisionCC main view for a 12-node cluster with its associated process alarm legend for each tick mark shape and color. view larger

A key step in the development of NVisionCC was the team's realization that the many nodes on a cluster actually fall into a small number of classes. Most of the nodes in a cluster are compute nodes (which are allocated to users to run serial or parallel jobs), some are head nodes (used to access the cluster, compile software, and submit and monitor jobs), some are storage nodes (to hold datasets), and some are management and monitoring nodes (which typically are accessible only to the cluster administrators). Instead of trying to focus on a large cluster consisting of hundreds or thousands of nodes, an administrator can instead easily comprehend a small number of node classes that are typically homogeneous.

Profiles can be set for each class of node, defining the parameters of acceptable, secure use for that type of node, including the processes that are allowable on that type of node, the ports that can be used, etc.

The individual nodes in a cluster can be separated into these classes, and NVisionCC then compares the incoming data on each node to the acceptable profile for that node category. In this fashion multiple compute nodes, for example, can be quickly compared to a single profile and any activity that falls outside the defined profile can be flagged for a system administrator to examine in more detail.

A text summary of the four process alarms found on the monitored cluster. In this way, alarms can be categorized and prioritized. Alarms are not only listed on this interface but are logged to an external syslog file for further analysis. view larger

NVisionCC currently includes:

1. a Process Monitor Module that tracks the processes running on each node,
2. a Port Scanner Module that scans each node for open network ports,
3. a File Integrity Module that validates the identity of disk files, particularly those files that hackers frequently try to alter for their own ends.

Real-time data from these three modules are compared to the configured profiles for each type of node and visualized on an interface plug-in extension of Clumon, a cluster performance monitoring tool developed at NCSA that is widely used on clusters worldwide.

As the team continues to develop NVisionCC, they plan to include a traffic analyzer, which will compare network traffic with the cluster communication pattern and will correlate the network traffic with the job scheduler, and a log analyzer, to analyze the cluster's system logs.

Return to November 2004 Newslink Table of Contents

If you would like to receive an email reminder when an issue is placed online, please add your name to the TRECC News mailing list.