Sunflow distributed rendering with ProActive Scheduler

14 August 2009 by admin

sunflow-logoSunflow is an open source rendering engine for photo-realistic image synthesis. Its integration with the ProActive Scheduler system enables an easy-to-use distributed rendering of 3D models.

Main Features

  • Fully Java Open Source
  • Easy installation and minimal configuration
  • Direct integration into the Sunflow GUI
  • Automatic file-transfer management
  • Fault-Tolerance

Description

This integration makes the installation and usage as simple as possible but here is a little explanation for minimal understanding. If you are already familiar with the ProActive Scheduler concepts skip this section.

Executions of parallel tasks on distributed resources (what we call ‘nodes’), such as networks of desktops or clusters, require a main system for managing resources and handling task executions. The ProActive Scheduler is connected to a Resource Manager providing the resource abstraction and enables users to submit jobs, containing one or several tasks that will be executed on available resources. A job is the entity to be submitted to the scheduler. It is composed of one or more tasks. The task is the smallest schedulable entity, it is included in a job and will be executed in accordance with the scheduling policy on the available resources. For more information you can consult the official ProActive Scheduler documentation.

For the management and advanced control of the Scheduler a remotely connectable GUI is available that offers additional interactions and statistics (a console admin client is also available). Please refer to the Scheduler GUI documentation. Same kind of GUI is available for the Resource Manager, please refer to the Resource Manager GUI documentation.

distributed-rendering-architecture_medium

Figure 1: Distributed rendering architecture

The Sunflow scene is decomposed into a fixed number of buckets, then each bucket can be rendered independently from the others. A single bucket rendering process is formulated as a task, therefore the whole scene can be seen as a job. However, the user can increase the number of buckets to be rendered per a single task.

Once a remote task is finished, it is sended back through the Scheduler to the Sunflow engine and appears on main screen immediately.

scene_decomposition

Figure 2: Example of 6 buckets per task decomposition

If you look at the figure 2 you can see that the scene is roughly decomposed into 60 buckets, with 6 buckets per task the job will have 10 tasks. This is almost the perfect decomposition if there are 10 computing nodes, since it will require only 10 communications because by default the Scheduler will try to use all available resources. However, if you would like to watch intermediate results in real time and this way be able to quickly detect obvious incorrectness and to cancel the on-going rendering process, simply decrease the number of buckets per task. Depending on the complexity of the scene and resources availaibility, the user can change the bucket/task ratio directly from the Sunflow GUI.

Detailed Features

  • The source code is included in the modified sunflow.jar archive.
  • The user can submit/cancel a job and change the number of buckets per task directly from the Sunflow GUI.
  • The user doesn’t need to specify required files (textures, bump maps, images etc.) manually, the rendering tasks will dowload missing files automatically.
  • The ProActive Scheduler offers resource fault-tolerance by using a database to store activities. For example, if a resource shuts down during a rendering process the rendering task will be executed elsewhere.

Be sure to have at least Java SE 5 installed on any computer you want to use. The JDK is required, because the Scheduler uses the database only available in the JDK bundle. You can get the latest release of Java SE from the official Java web site.

Downloads

Installation, Configuration and Execution for Windows Users

  • From the command-line, go to the Scheduler bin\windows directory and run the startScheduler_newDB.bat script to start the Scheduler with an embedded Resource Manager once started it will print its URL
  • There are 2 way to create computing nodes on remote computers:
    • First way is to install the ProActive Agent on each computer that you want to use for computation and configure it (double-click on agent tray icon and click on GUI Edit button), you can save the configuration and load it on another computer:
      1. In the General tab set the ProActive Location field to the Scheduler directory
      2. In the Connection tab select Resource Manager Registration as Enabled Connection
      3. Fill the Resource Manager URL field with the url of the Resource Manager like rmi://HOSTNAME_OR_IP_ADDRESS:1099/
      4. Fill the Node Name field with for example “workerNode”
      5. Use “demo” for both Username and Password fields
      6. Save the configuration, close it and start the agent
    • Second way is to start a computing node from command-line from Scheduler bin\windows directory by running the startNode.bat <NODE_NAME> then adding the url of the node to the Resource Manager through the Resource Manager GUI or the admin client.
  • Start the modified version of Sunflow from command-line by running the run_sunflow.bat <SCHEDULER_HOME> the parameter must be the absolute path of the Scheduler, from the Sunflow GUI select an example scene then in case of distributed rendering enter the URL of the Scheduler and use “demo” for both username and password.

Installation, Configuration and Execution for Linux Users

  • From the command-line go to the Scheduler bin\unix directory and run the startScheduler_newDB.sh script to start the Scheduler with an embedded Resource Manager
  • There are 2 ways to create computing nodes on remote computers:
    • Start a computing node from command-line by running $SCHEDULER_HOME/bin/unix/startNode.sh <NODE_NAME> then add the url of the node to the Resource Manager through the Resource Manager GUI
    • Deploy several computing nodes by using a GCM Deployement Descriptor

Example

rendered-julia

Figure 3: The rendered julia.sc scene

For example the distributed rendering of the julia.sc (see figure 3) scene using a completely heterogeneous infrastructure composed of 1 qual-core Windows XP (4 computing nodes) , 1 dual-core Windows Vista (2 computing nodes), 1 dual-core MacBook (2 computing nodes) and 1 dual-core Linux (2 computing nodes) is enough to significantly reduce the computing time.

rendering_times_classic_vs_distributed_-_julia_sc





No Comments »

No comments yet.

RSS feed for comments on this post. TrackBack URL

Leave a comment

IMPORTANT! To be able to proceed, you need to solve the following simple math (so we know that you are a human) :-)

What is 7 + 5 ?
Please leave these two fields as-is:
© 2012 ACTIVEEON S.A.S. All right reserved - Online Privacy Policy