Hacking VTK for Parallelisation (MPI and C++)

Have you ever wondered how to fully use [VTK](http://www.vtk.org/) (Visualisation Toolkit) for stuff you really want to accomplish?

In my case we have a code that uses VTK and we want to run it across several machines in parallel (Using OpenMPI). Kitware’s [ParaView](http://www.paraview.org/) is a nice piece of software using VTK for the visualization of data and as the name says enforces parallelisation. For every VTK file extension (vtu, vtp, vts, etc.) there exist a parallel version (pvtu, pvtp, pvts, etc.) which actually has more or less meta-information about your data, like your point and cell data tags but it also defines a set of multiples files which can be assigned, the so called piece files.

So if you use MPI and let run everything in parallel even writing the data into separate files (.vtu) and reference all of them using the parallel file (.pvtu) you have the advantage that you can open your case in paraview through that one file.

Unfortunately there is not that much of documentation about how to use this class [vtkXMLPUnstructuredGridWriter](http://www.vtk.org/doc/nightly/html/classvtkXMLPUnstructuredGridWriter.html).

Let’s define some vtu files. The following code is run in parallel with MPI:

std::string SPHEREDATAFNAME = „sphereData_“;

// Append process number
SPHEREDATAFNAME += std::to_string(gParallelMngr.getProcID());

SPHEREDATAFNAME += „.vtu“;

// … Your VTK Code here …

// Write file
auto writer = vtkSmartPointer<vtkXMLUnstructuredGridWriter>::New();
writer->SetFileName(SPHEREDATAFNAME.c_str());

writer->SetInputData(unstructuredGrid);

writer->Write();

Let’s say you have four parallel processes running. You are going to generate „`sphereData_0.vtu„`, „`sphereData_1.vtu„`, „`sphereData_2.vtu„` and „`sphereData_3.vtu„`. Now only one process is allowed to create the parallel file, let it be the master process (usually number 0):

if( gParallelMngr.isMaster() )
{
const std::string PIECESDATAFNAME = „sphereData.pvtu“;

auto pwriterObj = vtkSmartPointer<vtkXMLPUnstructuredGridWriter>::New();

pwriterObj->EncodeAppendedDataOff();
pwriterObj->SetFileName(PIECESDATAFNAME.c_str());
pwriterObj->SetNumberOfPieces( gParallelMngr.getNumProcs() );
pwriterObj->SetInputData(unstructuredGrid);
pwriterObj->Update();
}

There ya go! With „`pwriterObj->SetInputData(unstructuredGrid);„` the parallel file gets the metainformation and with „`SetFileName(…)„` and „`SetNumberOfPieces(…)„` it creates the reference names by that scheme.

So far this is very useful. I still have one problem, especially considering that the serial files are scattered across different machines. With this approach you can only use parallelisation on one local machine having the generated files in the same directory.

I asked myself, is it possible to set the parallel file paths manually? There is no VTK documentation in how to do that so far to my knowledge, but I found a way. Let me show you how.

Through the VTK classes there exist one virtual method where the attribute of theses references is set „`void WritePPieceAttributes(int index)„`. Because it is virtual you can create your own class and overload with your code. Let’s do this by creating a new class:

#include „vtkXMLPUnstructuredGridWriter.h“

#include „vtkObjectFactory.h“
#include „vtkSetGet.h“

class LDEMVTKXMLPUnstructuredDataWriter : public vtkXMLPUnstructuredGridWriter
{
public:

static LDEMVTKXMLPUnstructuredDataWriter* New();

void WritePPieceAttributes(int index)
{
std::string DATAFNAME = „ProcData“;
DATAFNAME += std::to_string(index);
DATAFNAME += „/spheredata_“;
DATAFNAME += std::to_string(index);
DATAFNAME += „.vtu“;

this->WriteStringAttribute(„Source“, DATAFNAME);
}

};
vtkStandardNewMacro(LDEMVTKXMLPUnstructuredDataWriter);

By overloading the „`WritePPieceAttributes()„` method you now can define your own rules.

Instead of creating a „`vtkXMLPUnstructuredGridWriter„`  you create a „`LDEMVTKXMLPUnstructuredDataWriter„` instance or however you called the class, but you call the same VTK Methods as before:

if( gParallelMngr.isMaster() )
{
const std::string PIECESDATAFNAME = „sphereData.pvtu“;

auto pwriterObj = vtkSmartPointer<LDEMVTKXMLPUnstructuredDataWriter>::New();

pwriterObj->EncodeAppendedDataOff();
pwriterObj->SetFileName(PIECESDATAFNAME.c_str());
pwriterObj->SetNumberOfPieces( gParallelMngr.getNumProcs() );
pwriterObj->SetInputData(unstructuredGrid);
pwriterObj->Update();
}

That’s it! In my case I put another directory for every file like „`ProcData1/spheredata_1.vtu„`, so I can mount them through SMB, NFS or similar and visuallize everything through the pvtu file.

Why write in parallel instead of collecting data and then create a big file? Because you can use the whole cpu nodes for everything and that the concept of MPI. If you visuallise the data with paraview you can collect the data you only require you will render what you only want to see and save important CPU ressources.

Github on your Server -> Gitlab rocks!

<b>Note</b>: This is an old article I had on my old wordpress server.

I really love Github. It’s just awesome, the stuff you can do.

I always envied that and tried to fill a gap by using redmine, which is okay. Nevertheless it reminds me of a combination of sourceforge webpages and trac.

I always felt the approach of Github was to focus on the user and not on the project.

In redmine, sourceforge, as others, you create a project and link users to it. On Github it is the opposite way. The user can create as many projects as they please and manage them. They can join to groups and work on something or fork. Reminds of Facebook. Why was it called social coding again?

Anyways, now let’s try Gitlab. Gitlab provides a solution that is like Github but it’s a piece of software you can put on your server running a nginx webserver. The code is very well maintained and with Ubuntu pretty easy to install.

The advantage is that you have fully control over your stuff and don’t need to pay for any commercial support. Here you can create your private repositories and protect them. You shouldn’t do that though if it is meant to be opensource. Also remember Github provides you space, so really it depends on your demands.

Taking a look at these instructions:

<a href=“https://about.gitlab.com/downloads/“ title=“https://about.gitlab.com/downloads/“>Gitlab downloads</a>

it is easy and nicely explained, but this should be done only if you have a server with nothing else on it. Apache, if installed or something similar might give you trouble you. So if you install this one I’d rather follow these instructions instead:

<a href=“https://www.digitalocean.com/community/tutorials/how-to-set-up-gitlab-as-your-very-own-private-github-clone“ title=“https://www.digitalocean.com/community/tutorials/how-to-set-up-gitlab-as-your-very-own-private-github-clone“>How to set up gitlab as your very own private github clone</a>

Yeah, no apache instructions on that site. Look, don’t use apache or if you need it for something else, tell nginx to use another port. That works very good for me. With links indicated below you can teach those both webservers to coexist peacefully:

<a href=“http://kbeezie.com/apache-with-nginx/“ title=“http://kbeezie.com/apache-with-nginx/“>Apache with Nginx</a>

So far I can tell with those problems, I really got my server working on some minutes. With redmine and the git plugins I needed much more time and configure a lot of more stuff.

I think redmine has more features, but I have to say I’m more interested in the power of git rather than Gantt Diagrams (Redmine had that feature). At work I didn’t see anyone using those thingys, but well it’s a way to manage yourself.

I feel gitlab like github as well are distraction free. So try it out and give the developers credit for their hard work!

Btw. Commander Genius source code is hosted on Github:

<a href=“https://github.com/gerstrong/Commander-Genius“ title=“https://github.com/gerstrong/Commander-Genius“>Commander Genius at Github</a>

If you like Games, if you platformers and jump&run, if you love Commander Keen, then what are you waiting for? Give it a try!

Using Commander Genius with LLVM/Clang

<b>Note</b>: This is an old article I had on my old wordpress server.

Since I got some time on my hands I decided to take a look at the hype of LLVM and wondered how does well would it work with Commander Genius…

Now I don’t want to go into details of LLVM or Clang. Tons of information about that technology can be found <a title=“here“ href=“https://en.wikipedia.org/wiki/Clang“>here</a> and anywhere else asking Dr. Google. I just want to see how well you can build Commander Genius using Clang and what other benefits you get. In this case I used Linux (Mageia 4.1 64-Bit)

<h4>CMake</h4>

Switching from GCC to Clang is very simple! If you don’t have any obscure defines in your CMakeLists files you use, you can tell cmake to switch to clang by setting:

<code>
export CC=/usr/bin/clang
export CXX=/usr/bin/clang++
cmake .
</code>

and there you go, just launch make and the build system does the rest. Remember to clean up caches if you had some.

<h4>QtCreator 3.2.1</h4>

Not as easy, but simple enough. I would launch those two exports in a terminal and then open QtCreator so its underlying build-system takes it correctly.

By using the clangmodel you get livefeedback without even compiling your sources. In some cases you get feedback while typing. You had that before, but now the messages come directly from the compiler, it matches to the whole build process

First of all you should enable some plugins, because by default they are deactivated. Go to Help->Plugins and enable ClangCodeModel. Don’t forget to restart Qt Creator.

<a href=“../../../../assets/2014/09/QtCreatorLaunch.png“><img src=“../../../../assets/2014/09/QtCreatorLaunch-300×168.png“ alt=“QtCreatorLaunch“ width=“300″ height=“168″ class=“alignnone size-medium wp-image-128″ /></a>

Now you should configure the Kit so Qt Creator knows what to use in order to build your app.

<a href=“../../../../assets/2014/09/KitClang.png“><img src=“../../../../assets/2014/09/KitClang-300×168.png“ alt=“KitClang“ width=“300″ height=“168″ class=“alignnone size-medium wp-image-127″ /></a>

Define a new Kit and viola, it will build using Clang. Nothing special so far except that you can do more from now on.

This is were Clang really shines in my opinion. First of it I feels like it builds a bit faster and links much faster than GCC.

Secondly, the warnings and errors are so much better explained.

<a href=“../../../../assets/2014/09/niceWarnings.png“><img src=“../../../../assets/2014/09/niceWarnings-300×168.png“ alt=“niceWarnings“ width=“300″ height=“168″ class=“alignnone size-medium wp-image-129″ /></a>

And last but not least the Clang Analyzer runs in the background and is able to give you some hints and warnings or even errors as you type. This is great because you don’t need to compile the code first in order to see what you did wrong.

<a href=“../../../../assets/2014/09/Analyzer.png“><img src=“../../../../assets/2014/09/Analyzer-300×182.png“ alt=“Analyzer“ width=“300″ height=“182″ class=“alignnone size-medium wp-image-126″ /></a>

The internal code analyzer used for GCC usually works nice, but since the messages do not come from the compiler, you are not sure, if these messages are real compiler issue. The advantage of that really is, that the analyzer actually uses Clang itself instead of guessing what GCC would say.

<h4>Binary sizes</h4>

I also did some binary size comparisons:

With GCC 4.8 (64-Bit) I get:
<code>
[gerstrong@localhost LINUX (dbfusion)]$ ls -l
-rwxrwxr-x 1 gerstrong gerstrong 3573043 Sep 22 17:50 CGeniusExe*
</code>

And with Clang 3.3 (64-Bit):
<code>
[gerstrong@localhost LINUX (dbfusion)]$ ls -l
-rwxr-xr-x 1 gerstrong gerstrong 3312886 Sep 22 18:00 CGeniusExe*
</code>

Building Commander Genius under Windows 7

Note: This is an old article I had on my old wordpress server.

There is a document around the internet (https://sourceforge.net/projects/clonekeenplus/files/CGenius/CompileCGunderWindows.pdf/download) I wrote some years ago in how to setup the source code of Commander Genius under Windows 7 the way you can compile it yourself and do changes wherever necessary. Yeah my english got better by then, but still very ugly I know. This guide below should be better and is more updated anyways…

Getting everything run on Windows is a bit more complex than with a linux distro providing some sort of software management, because the user has to download some dependencies by hand starting with a working compiler and all the required libs that get the sources built. As there are many options to download, I’m going to provide one that works well enough.

I hope with this article everything will be easier for those who want to build CG or create at no cost great and powerful programs using SDL or maybe even games. Of course this article might have some issues. If you encounter something or you don’t get the environment run, just leave a comment or let me know by dropping an email in my box. 🙂

Other opensource projects have the same requirements and often depend on more stuff than CG, but after going step by step with this tutorial you will have a nice opensource environment in which you can build a lot of cool programs and libs of course.

C++11 pushs the standard over the edges and boost the deployment of programs with even higher qualities. Well, you still can mess around if you want, but we have better tools not to do so.

So let’s start shall we?

 

Prerequisites

The requirements are:

  • Compatible compiler for source code (c++11)
  • CMake
  • SDL2, SDL2_image and SDL2_mixer
  • Boost
  • Curl
  • Python3 (Python3.5 or higher recommended)

What we also want, so we can develop and might improve the code or build our own projects from it:

  • Good IDE
  • Good Debugger
  • Git, hell yeah!

We are not focusing on Visual Studio here because I never build it on that platform. That has several reasons, but if someone wants to try that, well, he can do that I guess. All the stuff like libs have to run with that platform which can be tedious sometimes. Languagewise it shouldn’t be a problem because since M$ Visual Studio 2012 is out, there is some C++11 support so pretty sure enough so you can build it.

Ahh and of course your Windows XP, Vista, 7 or higher has to be 64 Bit, since those builds will only run on that.

For a 32-Bit system you can use the 32-bit compilers instead (QtCreator ships one), but all the libs need to be compatible with that platform. Personally, I would recommend 64-bit, it is about time…

Compatible compiler (GNU C++)

There are many packages you can download but I would recommend this package <a href=“http://sourceforge.net/projects/mingwbuilds/files/host-windows/releases/4.8.0/64-bit/threads-posix/seh/x64-4.8.0-release-posix-seh-rev2.7z/download“ title=“Mingw x64-4.8.0-release-posix-seh-rev2″></a>. It also has a debugger which does not solve all the STL goodness yet but with a strong IDE it provides some basic debugging nevertheless.

Btw. if you need a good unpacker that handles many compressed package formats try <a href=“http://peazip.sourceforge.net/“ title=“Peazip“>Peazip</a>

In the „Good Debbuger“ section of the second upcoming article I will explain how you can refine the debugger.

Just unpack the package at a place where you can access it easily. It should be a directory for development stuff, because more files from the libraries, like boost and SDL will be put there later on. I would recommend „C:\mingw64“ which we will use in the screenshots below for our configurations in the IDE.

CMake

That one is easy. Get it http://www.cmake.org/download/

You can download the Win32 Installer and start the setup, that’s all. You should remember the path where it is installed. I would think it is the best extracting the zip into C:\ming64, the way the „share“, „bin“ subdirectories are merged, but it’s not necessary.

CMake allows to cross compile programs. It is used by CG and its strength is that you can deploy project or makefiles for different systems. The IDE of my recommendation Qt Creator opens CMakeLists.txt files directly and acts as frontend for make. So you don’t have to worry about the sources that much when others want to have a project file for a different IDE.

CMake is able to generate project files for Eclipse, CodeBlocks and M$ Visual Studio or even Makefiles.

The Good IDE (QtCreator)

Also it should be clear that I use opensource software for building CG. It always has been that way.

I have tried a good number of IDEs to get CG work, because first I used Eclipse CDT for a long time, which was nice but always too clonky and overbloat in my opinion.

Later I changed to CodeLite, CodeBlocks, Anjuta, Kdevelop and also geany, but I never was happy with those. One day I tried QtCreator and instantley got hooked.

I’m going to recommend that one, but of course, any IDE in which you can define the compiler of choice will do.

Let’s go the website and download the Qt Online Installer: https://www.qt.io/download-qt-installer

You can also download a more recent version and use the online installer for better dependency solving. Just ensure it ships one mingw suite, otherwise you have get one separately. So if there is a newer version, take it. As you can see, it already ships a compiler you can use right away, but it’s only 32-bit. Yeah, 64-bit can runs 32-bit apps but we want to got further, let’s do that…

So we have to tell QtCreator to please use the 64-bit compiler and linker.

Open up Qt Creator and go to Tools->Options->Build & Run and go to the Debuggers Tab. You can clone the autodetected and edit the path of the clone so gdb for the 64-Bit compiler is set.

No go to the compilers tab, clone a profile and set the path where you find your compiler.

And finally create a kit (You can clone here as well), use the desktop profile setting and choose in the dropdown menus the compiler and debugger profiles you just created in the last steps.

Now, that you have everything set up you can open CMakeLists.txt as a project file, or create your own ones with a nice wizard. Using a new CMake C++ project under File->New it will create an extremely portable hello world, with CMakeLists.txt included.

If CMake was not detected, go to Tools->Options->Build & Run then go to the CMake tab and set the correct path of cmake.exe.

As you can see, staying with Windows and having a powerful build system at the same time with a fun IDE and a good compiler is possible without having to pay any money. Thanks to Richard Stallmann and all his community for all this and much more.

Btw. did you know, that with all that you can also build Fortran programs or even mixed C/C++ Fortran programs? GNU Fortran is shipped and Cmake also supports that. Just create a new CMake C++ project, open the CMakeLists.txt, add the line „enable_language (Fortran)“ before „add_exectuable(…)“ and put your for-files as parameter to that „add_exectuable“ command where the C++ file is already defined. Put your code into the for file and there you go! Fortran at no extra cost. Also you can extend Qt Creator to recognize the Fortran syntax.