Thread: SAGA LSF Adaptor - Introduction & Status Report

Hi everybody,

my name is Ole and I'm the developer of the Platform LSF adaptor for SAGA - the Simple API for Grid Applications. I'd like to give you a quick introduction and status report.

SAGA is an open standard defined and maintained by the Open Grid Forum that describes an interface for high-level Grid application programming. Me and my colleagues at the Center for Computation & Technology are working on the C++ reference implementation of the SAGA standard as well as various middleware adaptors (including GT4, Condor, GridSAM and of course LSF). You can find out more about SAGA here.

The Platform LSF adaptor is currently in a usable state but it is not yet complete. It operates on top of the LSF command line tools and implements basic functionality for job submission, control and monitoring. Let me give you some simple C++ code examples to show you what is already possible with the LSF adaptor and how easy it is to interact with LSF programatically through SAGA:

First, let's start with the probably most common task - submitting a job to LSF:

Code:

using namespace saga::job;
        
saga::url js_url("lsf://localhost");
saga::job::service js (js_url);
    
saga::job::description jd;
jd.set_attribute (attributes::description_executable, "/bin/sleep");
jd.set_attribute (attributes::description_queue, "SHORT");
jd.set_attribute (attributes::description_job_contact, "oweidner@localhost");

saga::job::job j1 = js.create_job (jd);
j1.run ();

std::cout << "Job ID: " << j1.get_job_id() << std::endl;

The few lines of code submit a job to the LSF server running on "localhost" using the "SHORT" queue. The output will look like this:

Code:

Job ID: [lsf://cn01/]-[117223]

Now you might want to query the job for its status. That's quite easy as well:

Code:

saga::url js_url("lsf://localhost");
saga::job::service js (js_url);

saga::job::job job = js.get_job("[lsf://cn01/]-[117223]");
std::cout << job_id << ": "
          << saga::job::detail::get_state_name(job.get_state())
          << std::endl;

The output might look like this:

Code:

[lsf://cn01/]-[117224]: Running

Another thing you can already do is getting a list of all jobs and their states known to the LSF server:

Code:

using namespace saga::job;

saga::url js_url("lsf://localhost");

saga::job::service js (js_url);
std::vector<std::string> job_ids =js.list();

for(int i=0; i < job_ids.size(); ++i)
{
    saga::job::job j = js.get_job(job_ids[i]);
    std::cout  << j.get_job_id() <<" : "
               << j.get_state()) << std::endl;
}

Here's the output on my local machine:

Code:

[lsf://cn01/]-[117212] : 2
[lsf://cn01/]-[117218] : 2
[lsf://cn01/]-[116921] : 6
[lsf://cn01/]-[117100] : 2
[lsf://cn01/]-[117068] : 2
[lsf://cn01/]-[117090] : 2
[lsf://cn01/]-[117180] : 2
[lsf://cn01/]-[116968] : 2
[lsf://cn01/]-[116969] : 2
[lsf://cn01/]-[117066] : 2

I hope that gives you a first idea how SAGA works. We're planning to have a first production version of the LSF adaptor in our upcoming 1.1 release which will be available at SAGA :: A Simple API for Grid Applications - HOME. It should be available right before Supercomputing 2008 - around November, 15th.

BTW: We're at boot 596 if you want to stop by ;-)

Regards,

Ole

This is great! I'll definitely be coming by to see the demo at SC08.

Since LSF doesn't provide any real file management capability, what do the file-related packages of the SAGA library map to? Is there some simple functionality provided by default in the SAGA library?

Originally Posted by csmith

This is great! I'll definitely be coming by to see the demo at SC08.

Since LSF doesn't provide any real file management capability, what do the file-related packages of the SAGA library map to? Is there some simple functionality provided by default in the SAGA library?

SAGA comes with two bindings for the file package right now:

(1) The 'local' file adaptor which can access local files and directories using the Boost::Filesystem library (platform independent).

(2) The Globus GridFTP adaptor which supports remote and third-party file transfer using the GridFTP protocol and security context.

How do you usually transfer files from/to a remote LSF cluster in the "Platform world"?

ssh/scp is a very typical way of moving files around.