wiki:Tutorial/OpenSG1/Clustering

Previous Chapter: Multithreading

Tutorial Overview

Next Chapter: Modelling


Clustering

At the very beginning I told you that OpenSG is really powerful when it comes to clustering. Now it is time we have a deeper look at that. Although clustering with OpenSG is easy and safe compared to other systems, it is still a bit more advanced than the other tutorials I presented so far.

Clustering in General and with OpenSG

The demand of computational power an application can have is nearly infinite - either it is because of bad programming or of high quality of the data and many power intensive features. Cutting edge applications need often the power of a supercomputer or a cluster of machines. Because supercomputers are in general a lot more expensive, clusters have become more and more interesting in the past years for industry as well as for universities.

You may have noticed that I like computer manufactured by a company having an apple as it's logo. Here is one interesting thing about them and clusters: The fastest computer on earth, at the time of this writing, is a supercomputer from NEC with a performance peak of 40 tera flops whereas the third fastest machine is a cluster of 1000 Apple G5's that score a peak of about 17 tera flops. Well, you might say that the NEC is more than twice as fast... yes, but it is also more than 10 times more expensive! That is why clusters get more and more interesting''

The following image shows a usual setup for a passive stereo application.

A simple cluster of three machines

If you want to drive a passive stereo application for example, you need three machines: the client is running the application (i.e. modifying the scenegraph, loading of files, handling user input, performing simulation etc.) and the two servers will only render. Each server would have a slightly different position of the camera to simulate human eyes and thus each server would render a different, yet similar, image.

The words "Client" and "Server" might be a bit misleading, so keep in mind, that the server does only rendering! One could say he serves the screen... or so. However, the X windowing system uses these words in an analogous way.

Of course this example was just one out of several possibilities on how to use a cluster with OpenSG. Another possibility is to drive a wall of screens (I mentioned the HEyeWall earlier) or to render just an output image, but by doing it with 48 machines for example.

We will start with a more simple example at the beginning - and don't worry if your cluster at home is out of order for the moment, all cluster-able applications can also be run on a single machine. In fact, OpenSG doesn't care about where the servers run, as long as they can be found on the local network.

One scene rendered in two independent windows

This image shows three windows: the small one is only for navigation interaction (i.e mouse movement) and is not necessarily required. Both other windows display one scene as if it were rendered into one window only. In this case, both windows run on the same computer, but this must not be the case as we can see very soon.

Example - A Multi Window Application

You might have guessed we need two different programs (client & server) and therefore we will need two .cpp files, too. This time I will present the whole program at once, because most of the code is not quite new at all. Here is the complete code for the client application - I tried to make it as short as possible

The Client

Here is the full code for the client application

#!cpp

// all needed include files

#include <OpenSG/OSGGLUT.h>

#include <OpenSG/OSGConfig.h>

#include <OpenSG/OSGSimpleGeometry.h>

#include <OpenSG/OSGGLUTWindow.h>

#include <OpenSG/OSGSimpleSceneManager.h>



#include <OpenSG/OSGMultiDisplayWindow.h>

#include <OpenSG/OSGSceneFileHandler.h>



OSG_USING_NAMESPACE

using namespace std;



SimpleSceneManager *mgr;

NodePtr scene;



int setupGLUT( int *argc, char *argv[] );



int main(int argc, char **argv)

{

#if OSG_MINOR_VERSION > 2

    ChangeList::setReadWriteDefault();

#endif

    osgInit(argc,argv);



    int winid = setupGLUT(&argc, argv);

	

    //this time we'll have not a GLUTWindow here, but this one

    MultiDisplayWindowPtr multiWindow = MultiDisplayWindow::create();



    //set some necessary values

    beginEditCP(multiWindow);

        // we connect via multicast

        multiWindow->setConnectionType("Multicast");

        // we want two rendering servers... 

        multiWindow->getServers().push_back("Server1");

        multiWindow->getServers().push_back("Server2");	

    endEditCP(multiWindow);



    //any scene here

    scene = makeTorus(.5, 2, 16, 16);

	

    // and the ssm as always

    mgr = new SimpleSceneManager;



    mgr->setWindow(multiWindow);

    mgr->setRoot  (scene);

    mgr->showAll();

    

    multiWindow->init();

    

    glutMainLoop();



    return 0;

}



void display(void)

{

    //redrawing as usual

    mgr->redraw();

	

    //the changelist should be cleared - else things

    //could be copied multiple times

    OSG::Thread::getCurrentChangeList()->clearAll();

	

    // to ensure a black navigation window

    glClear(GL_COLOR_BUFFER_BIT);

    glutSwapBuffers();

}



void reshape(int w, int h)

{

    glutPostRedisplay();

}



void mouse(int button, int state, int x, int y)

{

    if (state)

        mgr->mouseButtonRelease(button, x, y);

    else

        mgr->mouseButtonPress(button, x, y);

    glutPostRedisplay();

}



void motion(int x, int y)

{

    mgr->mouseMove(x, y);

    glutPostRedisplay();

}



int setupGLUT(int *argc, char *argv[])

{

    glutInit(argc, argv);

    glutInitDisplayMode(GLUT_RGB | GLUT_DEPTH | GLUT_DOUBLE);

    

    int winid = glutCreateWindow("OpenSG");

    

    glutReshapeFunc(reshape);

    glutDisplayFunc(display);

    glutMouseFunc(mouse);

    glutMotionFunc(motion);

	

    return winid;

}

Let us first have a closer look at the client application. Actually only the part of window creation is different, the rest is pretty much the same. We have the simple scene manager, we set up some callbacks and we create a scenegraph (which again is only a torus in this example).

Instead of an osg::GLUTWindow we now have a osg::MultiDisplayWindow which enables us to connect several independent windows. Let us have a look at the code

#!cpp

    MultiDisplayWindowPtr multiWindow = MultiDisplayWindow::create();



    beginEditCP(multiWindow);

        multiWindow->setConnectionType("Multicast");

        multiWindow->getServers().push_back("Server1");

        multiWindow->getServers().push_back("Server2");	

    endEditCP(multiWindow);

A multi window needs to know about how it should connect to the servers and it needs to know the "name" of every server that should be used. Servers are simply identified by a string. Not very creative at all, but I chose the names "Server1" and "Server2". Of course you can exchange these with the names of whatever you like.

That's all? Yes, at least for the client. Of course the servers are still missing. If you would run this application you won't see any window, because it will be waiting for a connection to the servers that will never come...

One word about the display() function: maybe you wonder why we have to clear the change list. We encountered the change list last chapter when I was talking about multithreading. The data exchange between the client and the servers works pretty much the same way, as clustering is some sort of multithreading on an arbitrary number of machines over a network. However, synchronization is handled automatically, but you still have to tell the system when to clear the change list.

The Server

Here is the full code of the server application

#!cpp

#include <OpenSG/OSGGLUT.h>

#include <OpenSG/OSGConfig.h>

#include <OpenSG/OSGClusterServer.h>

#include <OpenSG/OSGGLUTWindow.h>

#include <OpenSG/OSGRenderAction.h>



OSG_USING_NAMESPACE

using namespace std;



GLUTWindowPtr   window;

RenderAction   *ract;

ClusterServer  *server;



void display();

void reshape( int width, int height );



int main(int argc,char **argv)

{

    int             winid;



    // initialize Glut

    glutInit(&argc, argv);

    glutInitDisplayMode( GLUT_RGB |GLUT_DEPTH | GLUT_DOUBLE);



    if (!argv[1])

    {

        cout << "No name was given!" << endl;

        return -1;

    }



    // init OpenSG

#if OSG_MINOR_VERSION > 2

    ChangeList::setReadWriteDefault();

#endif

    osgInit(argc, argv);



    winid = glutCreateWindow(argv[1]);

    glutDisplayFunc(display);       

    glutIdleFunc(display);

    glutReshapeFunc(reshape);       

    glutSetCursor(GLUT_CURSOR_NONE);



    ract=RenderAction::create();



    window     = GLUTWindow::create();

    window->setId(winid);

    window->init();



    //create a new server that will be connected via multicast

    //argv[1] is the name of the server (at least it should be...)

    server     = new ClusterServer(window,argv[1],"Multicast","");

    server->start();



    glutMainLoop();



    return 0;

}



void display()

{

    //simply execute the render action

    server->render(ract);

    //and delete the change list

    OSG::Thread::getCurrentChangeList()->clearAll();

}



void reshape( int width, int height )

{

    window->resize( width, height );

}

The server is also quite similar to previous applications, although there are more differences compared to the client. First of all, it is very short - there are only three functions: main(), display() and reshape(). Well, the last one is obvious and the same as always, whereas the display() function differs slightly. Instead of calling the redraw() function of the simple scene manager we call the render command and passing a render action as argument. The reason for that is also very simple: there is no simple scene manager! Why should there be? As the name suggest, its task is to manage a (simple) scene, but that is already done by the client application. As the servers job is to render, we only need a render call.

Finally the main() function now features the registration of the callbacks and most importantly the creation of the server object

#!cpp

    server     = new ClusterServer(window, argv[1], "Multicast", "");

    server->start();

These two lines will start the server, which will take care of the connection to the client application. Multicast again is the connection type and argv[1] is the first argument from the command line that should specify the name of the server. The client is looking for "Server1" and "Server2" so these should be used as names.

Launching

The code can also be found in progs/14clustering_Client.cpp and progs/14clustering_Server.cpp.

This time the application is not started by a single executable, but by three. Open a terminal and if you are using Linux enter:

    ./14clustering_Server Server1 & ./14clustering_Server Server2 & ./14clustering_Client

If you are using Windows you have to open three command line shells and start one of the three programs in each shell. The order is not important.

After a very short connection time, you should see three overlapping windows - you have to rearrange them to see anything. If all windows are black, then you have to move the camera a bit, because the servers might have not started rendering.

Notice:

You should pay special attention to the names of the servers. If you make a typing error at that point the client will wait for a server that does not exist and your application will not start at all. Before you try to find any error deep within your application, you should check that first!

Connection Type

There are two different ways how servers can connect to the client, either via multicast or via a point to point connection. In the example above I used multicast as connection type. Have a look at these lines in the client

#!cpp

    multiWindow->setConnectionType("Multicast");

and in the server

#!cpp

    server = new ClusterServer(window,argv[1],"Multicast","");

which are responsible for setting up the connection. Multicast is straight forward to use, but if used in a large network (such as universities have, for example) you might get an administrator out of his cave, because network connection speed may suffer. In this case it is better to use StreamSock, which will establish a point to point connection, rather than scanning the whole network for potential servers. For big systems Multicast scales a lot better, as the data has to be put on the network only a single time, instead of once for each recipient.

When we developed the VR application, I mentioned previously, we had several people testing models and developing simultaneously and thus often started a client and two servers - but when more than one person started the client and servers at about the same time, it happened that one sever connected to the correct client, but the other to some other client... well, needless to say, that the immersion of the simulation was not that good ;)

After an angry administrator visited us (he finally found out why the network was so slow...) we finally had to switched from multicast to a point to point connection. So if you use Multicast, make sure you understand the implications.

Remote Aspects

From the last chapter you roughly know what aspects are. Remote aspects work in a very similar way. In a cluster environment remote aspects are used to propagate changes of field containers. The synchronization can take place automatically, but it can also be done manually with sendSync() and receiveSync(). Whether the synchronization is manually or automatically driven, it is important not to apply changes twice, so the change list has to be cleared always and also manually (like in the example). Under certain circumstances it is good to know when a field container is created or destroyed - the problem is that the change list carries only changes of existing field containers by default. When a new container is created, there is a bit of extra work to do. In this case it might come in very practical to have the possibility to define some callbacks for these events: registerCreated(), registerChanged() and registerDestroyed() will do what you are looking for.

Cluster Window

Before we begin with a more advanced tutorial, some more words about the clustering window. You have a window for rendering with GLUT, you have one for X and QT and so you also have a cluster window to render in a cluster. However, the cluster window is usually not used for rendering at all. You have to define a list of named servers, which will finally render - this is similar to the example

#!cpp

    ClusterWindow clusterWindow = ClusterWindow::create();

    clusterWindow->getServers().push_back("Server1_Name");

    clusterWindow->getServers().push_back("Server2_Name");

    //and so on

You can add any number of servers, but every one defined here must be available in the network (with exactly that name) in order to launch the application. If you have a look at the class diagram of osg::ClientWindow you will discover that two other windows are derived from cluster window, of which one we already know: the multi window from the example above. These two are special implementations which are adding more features.

Tutorial - A stereo application

As many users who want to use clustering have some sort of stereo application in mind, we will have a little tutorial here, which in the end, will be a fully functional stereo application. If you have two beamers and a screen around as well as at least three machines you can see the result in 3D - if you are not that lucky, you can still run the application on a single machine, but then you will miss the 3D effect.

The code comes in a few slices here... just paste them each below the previous one. We start right away with some includes and global variables

#!cpp

// all needed include files

#include <OpenSG/OSGGLUT.h>

#include <OpenSG/OSGConfig.h>

#include <OpenSG/OSGSimpleGeometry.h>

#include <OpenSG/OSGGLUTWindow.h>

#include <OpenSG/OSGSolidBackground.h>

#include <OpenSG/OSGDirectionalLight.h>

#include <OpenSG/OSGPerspectiveCamera.h>

#include <OpenSG/OSGTransform.h>

#include <OpenSG/OSGRenderAction.h>

#include <OpenSG/OSGViewport.h>

#include <OpenSG/OSGShearedStereoCameraDecorator.h>

#include <OpenSG/OSGMultiDisplayWindow.h>

#include <OpenSG/OSGSceneFileHandler.h>



OSG_USING_NAMESPACE

using namespace std;



NodePtr scene;



// one camera will be used only  - the

// decorators will distinguish between left and right

// eye position

PerspectiveCameraPtr camera;



// we need two viewports - one for the left

// and one for the right eye

ViewportPtr leftViewport;

ViewportPtr rightViewport;



MultiDisplayWindowPtr multiWindow;



NodePtr camBeacon, lightBeacon, lightNode;



RenderAction *renderAction;



int setupGLUT( int *argc, char *argv[] );

There is nothing special about that part, isn't it? The only new thing is the osg::ShearedStereoCameraDecorator. So let's go ahead with the createScenegraph() function.

#!cpp

NodePtr createScenegraph(char** argv)

{

    //create geometry - just a simple torus

    NodePtr n = SceneFileHandler::the().read(argv[1]);

    

    //we check the result

    if (n == NullFC)

    {

        n = makeTorus(1,5,8,16);

    }

	

    //create transformations & beacons for cameras & light

    camBeacon = Node::create();

    lightBeacon = Node::create();

    

    // the following style is a bit different than from before

    // this is only to remind you that beginEditCP()'s can also

    // be interleaved

    

    beginEditCP(camBeacon);

    beginEditCP(lightBeacon);

        //create Transformations

        TransformPtr camTrans, lightTrans;

        

        camTrans = Transform::create();

        lightTrans = Transform::create();

        

        beginEditCP(camTrans);

        beginEditCP(lightTrans);

            Matrix camM, lightM;

            camM.setTransform(Vec3f(0,1,10));

            lightM.setTransform(Vec3f(1,10,2));

            

            camTrans->setMatrix(camM);

            lightTrans->setMatrix(lightM);

                    

        endEditCP(camTrans);

        endEditCP(lightTrans);

            

        camBeacon->setCore(camTrans);

        lightBeacon->setCore(lightTrans);

    endEditCP(camBeacon);

    endEditCP(lightBeacon);

    // -- end of camera beacon creation

    

    //create the light source

    DirectionalLightPtr dLight = DirectionalLight::create();



    beginEditCP(dLight);

        dLight->setDirection(Vec3f(0,1,2));

            

        //color information

        dLight->setDiffuse(Color4f(1,1,1,1));

        dLight->setAmbient(Color4f(0.2,0.2,0.2,1));

        dLight->setSpecular(Color4f(1,1,1,1));

        

        //set the beacon

        dLight->setBeacon(lightBeacon);

    endEditCP  (dLight);

    

    // create the node that will contain the light source

    

    lightNode = Node::create();

    beginEditCP(lightNode);

        lightNode->setCore(dLight);

        lightNode->addChild(n);

    endEditCP(lightNode);

    

    // now create the root and add all children

    

    NodePtr root = Node::create();

    beginEditCP(root);

        root->setCore(Group::create());

        root->addChild(lightNode);

        root->addChild(camBeacon);

        root->addChild(lightBeacon);

    endEditCP(root);

    

    return root;

}

That is quite similar to the code from the last example - this time however, we have only one camera. That sounds not very smart as we intend to create a stereo application and two cameras would make more sense. Well, but that we have the decorators for. Watch the next section, the main() function

#!cpp

int main(int argc, char **argv)

{

#if OSG_MINOR_VERSION > 2

    ChangeList::setReadWriteDefault();

#endif

    osgInit(argc,argv);

        

    int winid = setupGLUT(&argc, argv);

	

    //create the main window for navigation

    GLUTWindowPtr navWindow = GLUTWindow::create();

    navWindow->setId(winid);

    navWindow->setSize(300,300);

    navWindow->init();

	

    scene =createScenegraph(argv);

	

    //we begin with creating our cameras

    camera = PerspectiveCamera::create();

	

    beginEditCP(camera);

        camera->setBeacon(camBeacon);

        camera->setFov(deg2rad(90));

        camera->setNear(0.1);

        camera->setFar(100);

    endEditCP(camera);

		

    //next we create the backgrounds

    

    SolidBackgroundPtr bkg = SolidBackground::create();

    beginEditCP(bkg);

        bkg->setColor(Color3f(0,0,0));

    endEditCP(bkg);

    

    leftViewport = Viewport::create();

    rightViewport = Viewport::create();

    

    //the decorator decorates the camera and will create the left eye

    ShearedStereoCameraDecoratorPtr cameraDecorator = ShearedStereoCameraDecorator::create();

    beginEditCP(cameraDecorator);

        cameraDecorator->setLeftEye(true);

        //unit length assume that one unit equals one meter

        cameraDecorator->setEyeSeparation(0.06);

        cameraDecorator->setDecoratee(camera);

        cameraDecorator->setZeroParallaxDistance(2);

        beginEditCP(leftViewport);

            leftViewport->setCamera    (cameraDecorator);

            leftViewport->setBackground(bkg);

            leftViewport->setRoot      (scene);

            leftViewport->setSize      (0,0,.5,1);

        endEditCP(leftViewport);

    endEditCP(cameraDecorator);

	

    //the right decorator for the right eye

    cameraDecorator=OSG::ShearedStereoCameraDecorator::create();

    beginEditCP(cameraDecorator);

        cameraDecorator->setLeftEye(false);

        cameraDecorator->setEyeSeparation(0.06);

        cameraDecorator->setDecoratee(camera);

        cameraDecorator->setZeroParallaxDistance(2);

        beginEditCP(rightViewport);

            rightViewport->setCamera    (cameraDecorator);

            rightViewport->setBackground(bkg);

            rightViewport->setRoot      (scene);

            rightViewport->setSize      (.5,0,1,1);

        endEditCP(rightViewport);

    endEditCP(cameraDecorator);



    multiWindow = MultiDisplayWindow::create();

    

    beginEditCP(multiWindow);

        multiWindow->setClientWindow(navWindow);

        multiWindow->setConnectionType("Multicast");

        multiWindow->getServers().push_back("Server1");

        multiWindow->getServers().push_back("Server2");

        multiWindow->setHServers(2);

        multiWindow->setVServers(1);

        multiWindow->addPort(leftViewport);

        multiWindow->addPort(rightViewport);

        multiWindow->init();

    endEditCP(multiWindow);

    

    // add a logo foreground to the right viwport

    

    //and the render action - more on that later

    renderAction = RenderAction::create();

    

    cout << "create scenegraph..." << endl;

	

    glutMainLoop();



    return 0;

}

Now have a closer look at the part concerning with the decorators. Each one decorates the same (and only) camera and they have both the same values for eye separation and distance to the zero parallax plane, but they set different values for the left eye field, indicating one is the left and one the right eye. Each decorator is assigned to its own viewport instead of the camera which we would have assigned normally. That is the reason why we only need a single camera - the decorators are translating two virtual cameras according to the given parameters of eye separation and distance to the zero parallax plane. The values provided are in model units, that means if you defined that one unit is equal to one meter, you should set the eye separation to about 0.06, smaller values will create no 3D effect and higher values will hurt your audience.

Finally we setup GLUT and define the usual callbacks

#!cpp

void reshape(int w, int h)

{

    multiWindow->resize(w, h);

    glutPostRedisplay();

}



void display(void)

{

    multiWindow->render(renderAction);

    

    //the changelist should be cleared - else things

    //could be copied multiple times

    OSG::Thread::getCurrentChangeList()->clearAll();

    

    // to ensure a black navigation window

    glClear(GL_COLOR_BUFFER_BIT);

    glutSwapBuffers();

}



void mouse(int button, int state, int x, int y)

{

    glutPostRedisplay();

}



void motion(int x, int y)

{

    glutPostRedisplay();

}



void keyboard(unsigned char k, int x, int y){

    switch(k){

        case 27:    {

            osgExit();

            exit(1);

        }

        break;

	}

}



int setupGLUT(int *argc, char *argv[])

{

    glutInit(argc, argv);

    glutInitDisplayMode(GLUT_RGB | GLUT_DEPTH | GLUT_DOUBLE);

    

    int winid = glutCreateWindow("OpenSG Navigation Window");

    

    glutDisplayFunc(display);

    glutMouseFunc(mouse);

    glutMotionFunc(motion);

    glutReshapeFunc(reshape);

    glutKeyboardFunc(keyboard);

    glutIdleFunc(display);

    return winid;

}

Now compile and execute the application. As this is a clustered application you have to start two servers called "Server1" and "Server2" as we did in the example

./14clustering_Server Server1 & 

./14clustering_Server Server2 & 

./15_stereo_client

As you can see, you can use the same servers as we used for the example. In nearly every case you can use this server application - it does not matter if you run a stereo or some other cluster application!

If you are running the application on a single machine it might be hard to notice the difference between the two viewports - in this case you can increase the eye separation, to increase the offset of both "eyes".

Of course this topic is a very huge one, and only a small part was covered here, but maybe this chapter will grow still a bit more in the future.


Previous Chapter: Multithreading

Tutorial Overview

Next Chapter: Modelling

Last modified 6 years ago Last modified on 01/17/10 01:11:44

Attachments (2)

Download all attachments as: .zip