1 Abstract

This document is a best practices guide for Platform EGO (or EGO for short) to orchestrate 3rd party application middleware. It identifies why 3rd party applications need to be integrated with EGO and orchestrated by EGO, what applications would benefit from this integration, and finally how to to it.


2 Benefit of Orchestrating 3rd party application with EGO

EGO can virtualize distributed heterogeneous resources and can coordinate the sharing of resources across applications, users and workloads. It is currently used to provide resources to HPC workload managers such as Platform LSF and Platform Symphony. These HPC workload managers ask EGO for some resources in the cluster, and use these resources to run some HPC applications.

There is also another kind of requirement for 3rd party applications, such as web services, 3rd party application server clusters, etc. These applications are usually managed in a simple way, even manually. But now it is expected that these applications can be managed automatically, and from a central place. These applications are also expected to have better resource utilization and be more reliable. A good solution for these requirements would be to use EGO to orchestrate these applications.

Any 3rd party applications can be run as an EGO service, i.e., they are managed and monitored by EGO. In this way, the life-cycle of these applications would be easily controlled in a central way (via EGO CLI or GUI), EGO also provides fail-over mechanism for the services it manages, by which the services provided by these applications would be more reliable. In addition, with the sophisticated EGO policies and PTM (Performance Threshold Manager is a condition and action engine, by which the action configured by user will be enforced when a specified condition is met), the resources in the cluster would be utilized by these applications in a better way.



We have used EGO to orchestrate a 3rd party application cluster: Hadoop cluster, the result turned out we can set up multiple Hadoop clusters in a physical cluster, and these Hadoop clusters can expand/shrink on demand.

3 How to Use EGO to orchestrate a 3rd party application

As mentioned in the last section, the key point of orchestrating 3rd party applications with EGO is to run them as an EGO service. So in this section, the procedure for running a 3rd party applications as an EGO service will be introduced.

3.1 Define an EGO Service

Each EGO service has its own definition in the form of an xml configuration file which defines the name of the service, how to run the service (the binary to be executed), the minimal and maximal number of service instances can be run, the resource requirement of running service, etc. Here is a sample service definition:
According to this XML configuration, EGO will automatically start 3 ~ 15 MyApp.exe instances for EGO service MyService, one instance per host. The slots in the resource group ComputeHosts are needed to run the instances of this service, a slot is a logical resource allocation unit by EGO. The resource sharing policy with other applications will be controlled by the resource plan defined for /DeptA.

Since EGO can do fail-over for the services it manages, the 3rd party application, wrapped as an EGO service will be more reliable. When the host on which an application instance runs is down due to for some failure, EGO will try to re-run that instance on another appropriate host, and this process is transparent to users.

3.2 Resource Group and Consumer

In EGO cluster, hosts are grouped into resource groups. User can define different resource groups for different purposes. Each host in a resource group has some slots defined which will be used by consumers to run activties (such as run an EGO service). Multiple consumers can consume the slots in one resource group, resource plan should be defined for a resource group to control how many slots each consumer can consume.

After wrapping 3rd party application as EGO service, some dedicated resource groups can be defined to run these applications. For example, an application needs some commercial libraries to run, and these libraries are only installed in some hosts in the cluster. So these hosts should be grouped into a dedicated resource group, i.e., only the hosts in this resource group can be used to run that application. Here is a sample resource group definition:
In the above definition, a resource group is defined whose name is MyResourceGroup, and only the hosts which have a library (libx) installed will be grouped into this resource group by EGO automatically.

In addition, the resource plan of this resource group should be defined which makes use of sophisticated EGO policies. Here is the sample resource plan for MyResourceGroup:
In the above definition, there are two consumers (/DeptA and /DeptB) can consume the slots in MyResourceGroup, and they shares the slots in 2:1 ratio. In this example, the slots of these two consumer can be used to set up two application clusters, and the size of the clusters can be adjusted on demand by changing the resource plan dynamically. So the resources in the physical environment can be utilized in a more flexible way.

3.3 PTM orchestrator

PTM is a condition and action engine provided by Platform in the form of Java libraries, it provides a framework based on which user can write their own PTM orchestrator which is used to orchestrate their own applications according to some pre-defined conditions.

The conditions PTM orchestrator cares are defined in an xml configration file, here is a sample:
Here two conditions are defined, one is when the CPU load is greater than 50%, ADD action will be enforced, this ADD action may be starting more instances for a 3rd party application; another one is when the CPU load is less than 10%, REMOVE action will be enforced, this REMOVE action may be stopping some existing instances for a 3rd party application.

Based on PTM framework, there are two kinds of methods that should be implemented: information collection method and action execution method;

1. Information collection method is used to collect the specific information (such as CPU load) which will be used to judge whether the pre-defined conditions are met or not. Here EGO API can provide great support on collecting host-based information, such as CPU load, memory, etc, and users are also free to choose other methods to collect application-based information, such as using JMX API to collect JVM heap utilization.
2. The Action execution method, is used to execute specified actions when a condition is met. The EGO API can also provide great support on EGO Service Controller, such as adding/removing an instance of an EGO service.

So with the help of PTM orchestrator, the 3rd party application cluster can expand/shrink on demand, the resources in the physical environment can be utilized in a better way.

4 Experience

The solution proposed in this doc still depends on the specific 3rd party application cluster to support expanding/shrinking that cluster. That means, when EGO is used to orchestrate a 3rd party application cluster, node joining/leaving in that cluster should not be done without taking account of the specific characteristic of that cluster. For example, a node in that cluster may not be removed in a special period, if EGO removes the node during that period, data loss or system instability may occur. So in this case, the solution needs to be aware of some characteristic of the 3rd party application cluster it targets, if PTM orchestrator is adopted in the solution, it can be used to take care of this point.

5 Summary

Platform EGO provides a flexible mechanism to manage and monitor application services. By running 3rd party application as EGO services, the 3rd party application cluster can be more reliable, and managed in a easier and more flexible way. With the help of EGO policies and PTM, the 3rd party application can utilize the resources in the physical environment in a more efficient manner.