Thread: MXJ and LSF Optimization

Hi I am running a 6.2 LSF cluster.

I have MXJ set for all the queues and the jobs are running OK.
My cluster effeciency is at 50% or below.

1) Is there a way to optimize the queues jobs with setting MXJ's on the queue ?
2) Is LSF intelligent enough to close the queue by itself if I do not set the MXJ ?
3) What else can I set or tweek so I can get rid of MXJ's in the queue and probably improve effeciency ?
4) Am I way under operating ? what is the general cluster effeciency in the industry?
5) What shouuld i be looking at to improve the cluster effeciency ?

Hi Aleem,

I've asked an LSF expert to answer this question.

- Ajith

LSF have lots of paramter to tune cluster performance according to different application requirement.

So in order to answer your question , I must know the following information from you

1 why you set MXJ in queue . what issue do you want to resove with MXJ
reduce job load in some hosts or job starvation
2 what's your cluster's job volume .how many hosts in your clusters
3 can you append the following command output
bjobs -l -u all
bqueues -l
bhosts -l

If you don't want to set MXJ, you can use load thresholds on the host to determine when to close the host (the queue stays open). You can set these load thresholds in the lsb.hosts file (look in the LSF Configuration Reference guide under lsb.hosts).

The load index to use for the threshold depends on the workload you are running, since different workloads affect the different metrics differently, but you can start trying with metrics like 'r1m' or 'ut' or even some combination of 'r1m' and 'pg'.

It's important that you have JOB_ACCEPT_INTERVAL set to at least 1, so you allow the new job sent to a host to affect the load and give mbatchd time to use the updated load to schedule the next job.