Teaching kde how to share

First post seemed to have been well received, so I will try keep it up.

Part of the advantage of using KDE is all the libraries and framework available for your program. By using a few shared libraries, you get for free a ton of functionality. The shared libraries can be shared between multiple processes, so memory consumption also goes down.

... or so you would think.

Exmap can list memory consumption by shared library (what a wonderful tool), so I can quickly verify whether they are being efficiently shared by multiple processes.

So here it is:

The number one library that consumes more memory is libGLcore. As you can see from the screenshot, there are 4 processes using it. Each and every single one of them is making an almost full copy of the library in memory. The biggest section in the library ".text" is 6.3 MB big, and it is being completely duplicated between all 4 processes.

This does not seem to be limited with kde, I tried with mplayer and glxgears with same results. It does not share a single byte in the .text entry even with other similar processes.

If I had to guess, and that is all I can do since it is just a binary blob, could be wrong, but it is probably the way the library is compiled, my guess: not using PIC (position independent code). This causes all of the function calls to be relocated when the library is loaded, and make it impossible for the system to share the code since each copy has been modified by the relocation process.

This library is part of the nvidia binary drivers, so there is not much to do there. If the library was compiled in such a way that it could be shared between processes, it could save about 18 MB of ram by not duplicating the code.

I would love to hear from someone to try with intel drivers, do they present the same problem?

So... the score:

Change	Potential memory savings
Change phonon backend from xine to something else	22 MB
Avoid nvidia binary drivers	18 MB (guesstimate)

Finding bloat in KDE4

KDE 4.x is a terrific piece of code. Provided you speak C++, the code is easy to understand and clean.

However, as good as the code is, there are issues that cause kde 4 to eats up more memory that kde3. This blog is dedicated to finding some low hanging fruits in memory consumption.

My machine is an AMD Athlon(tm) 64 Processor 3400+, with 1GB ram. It is running Kubuntu 7.10, and I compiled kde from svn revision 795377 with debug enabled.

To start with, I ran kde4 and profiled system memory with exmap. Here is the result:

As you can see the memory consumption on my machine is about 169MB.

The single process that consumes more memory here is knotify4. Knotify is a kde daemon that sits there waiting for events to happen. Once an event happens, it receives a message via dbus, and may notify the user via a sound, a popup window or other mecanisms. It is the process that is responsible for playing a sound when you maximize, minimize windows, log in, log out, etc.

For such a simple task, it seems strange that it needs 30MB of memory, so I dug deeper.

Going to the details of knotify4:

Between the heap and anon pages (I am not sure what the difference is) it consumes about 25Mb.

So I ran knotify4 under a heap profiler, namely valgrind --tool=massif and the results are pretty self explanatory:

Playing sounds with xine seems to be eating up all the memory. I tested other xine based players, and I got the same results. It seems to me that xine library just sucks up a ton of memory for playing vorbis files.

I tried playing the same vorbis files with mplayer, and the result is very different, it uses 1MB for heap + anon.

To verify audio is the problem, I disabled sound notifications in the System Settings -> Notifications. This is what the memory consumption looks like with no audio notifications:


Now knotify4 uses 2MB effective memory. It is no longer a blip in the radar. And kde as a whole now uses about 22MB less memory.

It is worth noticing that phonon was designed so that back end can be replaced. Switching to a more efficient back end than xine can make a significant improvement in kde4 memory consumption, kudos to the phonon team to make such transition easy. Of course, the other way to fix the issue is fixing xine library itself. Being written in C and looking at the code, it is clear that is something I do not want to debug.

I will be posting more articules pinpointing low hanging fruits in kde memory consumption, so I will keep the score in a table as I find stuff:

Change	Potential memory savings (aprox)
Change phonon backend from xine to something else	22 MB