Thread: HPC Cluster used for Animation Rendering

At Purdue University. They use lab machines as well as campus machines.

YouTube - HPC and Education

Something Symphony can be applied to?

Yes, rendering/raytracing has been discussed as a potential applications for Symphony.

The guy in the video (2:25), refered to Maya and 3D Studio Max, these are a couple of popular 3D graphics software widely used throughout the industry as well as academically.

most people doing renderfarms either use in-house batch schedulers or commercial batch schedulers such as Platform LSF.

Most render jobs takes a long time per frame, while Symphony I believed was designed for very short (milliseconds - seconds) jobs.

A more suitable product would be LSF for example. Symphony - would not be a good fit here - unless I am missing something here.

I agree with beowulf, that Symphony is optimized for short tasks.

If it would be possible to render sub-frames, to shorten the task time and spread processing over many more hosts, then Symphony could be useful.

We could get almost real-time rendering if we had enough hosts in the cluster.

One idea would be to put all the graphics information (layers, masks, etc) for one frame into session common data and pass some info to each task so that it could figure out what part of the frame to render.

I think that there are some use cases for more interactive usage. For example, if an artist is lighting a scene, they might want to run in an interactive mode so that they can quickly see the results of placing light sources. In this case, being able to render the scene in parallel would provide the interactivity that somebody sitting at a monitor would be looking for.

Sometimes I think we need to start seeing Symphony as a parallel programming toolkit more than just a task processing engine, where a Symphony application session is a "parallel job".

I agree we need to see Symphony used as a parallel or multi-core programming toolkit. As INTEL and AMD releases new quad-six-eight cores CPUs, it will only get more challenging to program these new CPUS to get the maximum benefits.

While multi-threading has been around, low-costs clusters or the upcoming trend of personal supercomputers like the Tyan Typhoons or Cray CX1, with its distributed memory - will only require easier to use multi-core distributed programming SDKs - where traditional multi-threading SDKs and techniques fail.

I agree as well, but there are (at least) 2 different ways this could be done.

One is allowing service tasks to be parallel, for example threaded. That would be one simple way of taking advantage of multicore (however on general-purpose cores, such as on the UltraSPARC T2, it could be taken into advantage more simply still by scheduling more tasks on the multicore host) or perhaps linking into some platform-specific API. This seems to make sense on things like the CBE or GPUs.

Another is to have the tasks effectively communicate (a traditional interpretation of the term `parallel job'. I have made a suggestion with 2 variants Extending Symphony for Numerical Applications. What do think of those, abnd what other ways could you make Symphony a parallel programming toolkit?

Rendering is hardware intensive. The studios use wireframes up to the last moment because some of the single video frames, using heavy iron, might take hours. Blender does set up with RenderFarm to allocate tasks amongst a network of computers.

Most render jobs takes a long time per frame, while Symphony I believed was designed for very short (milliseconds - seconds) jobs.

A more suitable product would be LSF for example. Symphony - would not be a good fit here - unless I am missing something here.