Thread: Location and Usage of OpenMP Wrapper

Originally posted by: JPummill, Fri Dec 15, 2006 5:12 pm

Can someone give me some quick tips on setting up an OpenMP job to run thru LSF? I am using the Intel OpenMP implementation.

I don't have test code yet. I just need to verify that, in fact, OpenMP is installed correctly and functional on this system via LSF.

Originally posted by: ddunlap, Thu Dec 28, 2006 9:12 pm

Well, it appears you didn't get any "quick" tips, in the 2 weeks since you posted this. Here are some of my thoughts on the topic.

This applies to any case of setting up parallel (MPI) jobs through LSF, regardless of whether you're using OpenMP, mpich-gm, or something
else...

First, I'll "cut to the chase". If you've got everything set up right, you should be able to do the following:

bsub -a <job_type> ... mpirun.lsf ./a.out

OpenMP example (run 4-way OpenMP job):
bsub -a openmp -n 4 mpirun.lsf ./a.out

What this means:
- the bsub "-a" option tells LSF to call "esub.<job_type>" before
submitting the job. LSF ships with several esub's, located in
$LSF_SERVERDIR. For OpenMP, the esub is called "esub.openmp".
This should set up the required variables for running the job
- in the case of OpenMP, the esub mostly just adds "pam" to the
command line
- mpirun.lsf is a script which uses the variables set by the esub, along
with run-time variables, to figure out how to run the job correctly
- of course, in this example the executable is "./a.out"

If you have problems:
1) make sure that you can run the job outside of LSF
Example (ksh):
export OMP_NUM_THREADS=4
./a.out
2) write your own simple LSF script to run the job the same way you
just did, then "bsub < script"
Example script:
#BSUB -J myjob
#BSUB -n 4
export OMP_NUM_THREADS=4
./a.out
Example bsub:
bsub < script
3) gradually add to the script, to make it look like what the esub
winds up with. For OpenMP:
Example script:
#BSUB -J myjob
#BSUB -n 4
export OMP_NUM_THREADS=4
pam ./a.out

If you follow this, you should be able to debug where things start to break.
Hopefully, it'll just work. The OpenMP "integration" is less complicated than some of the others...

Hope this helps,
Dale
_________________
Dale Dunlap
Technical Consultant

Originally posted by: JPummill, Thu Feb 08, 2007 2:49 pm

Well, the OpenMP app runs just fine from the command line. However, LSF exits with the following...

mpirun.lsf: LSF_PJL_TYPE is undefined. Exit ...

I used the following per the above instructions for a submission...

bsub -a openmp -n 2 mpirun.lsf ./omp_mm

The esub is where it should be as well.

Something obvious I am doing wrong?

Originally posted by: ddunlap, Thu Feb 08, 2007 3:15 pm
Sorry, I gave you some bad advice earlier.

The command for submitting your job should be:

bsub -a openmp -n 2 ./omp_mm

(drop the "mpirun.lsf"). Sorry about that.

The mpirun.lsf script is used for other MPI implementations, and depends on the LSF_PJL_TYPE variable being set.

Try dropping mpirun.lsf, and let me know how that goes. Also, you can RTFM (like I should have done): look in "Using Platform LSF HPC" -- there's a page on OpenMP jobs.

Hope this helps,
Dale
_________________
Dale Dunlap
Technical Consultant

Originally posted by: JPummill, Thu Feb 08, 2007 3:53 pm

Hey, thanks Dale!

That worked. And, yes, I should definitely RTFM. I've become dependant on asking questions rather than searching for the answers myself.

Nonetheless, appreciate your quick (and accurate) response!

-Jeff