Saturday, August 6, 2016

Danny Hillis: The Pattern on the Stone

"The greatest achievement of our technology may well be the creation of tools that allow us to go beyond engineering - that allow us to create more than we can understand."

Tuesday, January 26, 2016

Unsere Art zu leben - Eine ehrliche Gesellschaftskritik

Es gibt Menschen, die die freien Gesellschaften mit Gewalt bekämpfen. Und es gibt Menschen, denen diese freien Gesellschaften Gewalt antun - ohne dass dies hier groß auffallen würde.
Immer, wenn irgendwo auf der Welt Terroristen gemordet haben, müssen Regierungschefs vor Kameras treten und einige Sätze dazu erklären. Eine traurige Routine ist das geworden; Angela Merkel bewältigt sie mit Variationen ein und derselben Bemerkung. Die Terroristen zielten auf "unser freies Leben in freien Gesellschaften", sagte sie nach dem Anschlag von Istanbul. Sie griffen "unsere Art zu leben" an, sagte sie nach dem Horror von Paris, vor zwei Monaten. Es sind Worte des Trotzes und des Beharrens - was in dem Moment auch angemessen ist. Doch in ihnen verbirgt sich viel mehr, als das Floskelhaftige in ihnen zum Ausdruck zu bringen scheint.
Dass Gesellschaften sich von Terroristen nicht intellektuell herausfordern (oder gar einschränken) lassen, ist eine Binsenweisheit, die nur Terroristen nie kapieren. Wer andere in die Luft sprengt oder erschießt, mit dem diskutiert man nicht, den bekämpft man. Dennoch steckt in einer Formulierung wie "unsere Art zu leben" nicht allein eine Kampfansage an Mörder. Denn zum einen mag sie die Freiheit beschreiben, nach Istanbul zu reisen und dort die Hagia Sophia zu besichtigen; oder den Besuch von Kneipen, Restaurants und Konzerten in Paris, München, Kopenhagen. Sie mag zum Ausdruck bringen, dass in freien Gesellschaften jeder nach seiner Façon leben darf; Männer, Frauen, Schwule, Heteros, Fleischesser, Vegetarier, Gläubige, Atheisten.
Das ist das eine, das von dem Begriff umfasst wird. Das andere ist das, worauf ausgerechnet Angela Merkel einmal hingewiesen hat, und zwar ganz und gar nicht im Zusammenhang mit Terror. Es ist einige Jahre her, es war die Neujahrsansprache kurz nach dem gescheiterten Weltklimagipfel von 2009. Da dachte sie laut darüber nach, "wie wir unseren Wohlstand erhalten" - nämlich "indem wir unsere Art zu leben und zu wirtschaften ändern".
Es wäre geboten, die Lebensart in den reichen, freien Gesellschaften als beides wahrzunehmen: als Errungenschaft und als gigantisches Problem. Gelebt wird hier ein Freiheitsverständnis, das absolut ist. Nichts ist den Menschen hier fremder als Beschränkung im persönlichen Alltag. Freiheit ist erstens der Kneipenbesuch und zweitens, dass der Wirt Heizpilze auf den Gehsteig stellt, damit man auch im Januar den Wein und den Barsch draußen genießen kann. Freiheit ist, dass Amerikaner 6,6 Milliarden Kilowattstunden Strom allein für Weihnachtsbeleuchtung aufwenden, mehr als Tansania im gesamten Jahr verbraucht. Freiheit ist, ein Auto zu bauen (und zu kaufen), das pro Kilometer 224 Gramm Kohlendioxid ausstößt. Benedikt XVI. war ein Papst, der nicht mit Beispielen erklärte, sondern der lieber grundsätzlich formulierte. So gelang ihm der Satz, "dass Materie nicht nur Material für unser Machen ist" - den Menschen des 21. Jahrhunderts muss man an eine solche Banalität erinnern. Denn der macht und macht; und wer denkt schon über Terrorismus, Klimawandel und Flüchtlinge nach, wenn er unterm Heizpilz sitzt.
Das ist in Wahrheit die größte Herausforderung: die Auseinandersetzung mit einem so alltäglich gewordenen Lebensstil. Das Nachdenken darüber, ob zum Beispiel aus Westafrika auch deshalb so viele Flüchtlinge kommen, weil europäische Kutter den Einheimischen dort die Barsche weggefischt haben? Wie viele ökonomische und kulturelle Konflikte künftig allein deshalb drohen, weil Menschen ihre Heimat für unbewohnbar erklären und nach Europa aufbrechen; mal der Armut wegen, mal um einem Krieg zu entkommen, immer aus Perspektivlosigkeit? "Wir haben unseren Wohlstand auf dem Rücken der Entwicklungsländer aufgebaut. Das wird nicht mehr lange gut gehen. Diese Spannungen entladen sich." Hört sich nach Brot-für-die-Welt-Sätzen an. Ihr Urheber kommt aber aus der CSU, es ist der Entwicklungsminister Gerd Müller. Er fügt noch hinzu, dass man sich nur nichts von Obergrenzen für Flüchtlinge versprechen soll: "Die Menschen werden uns nicht fragen, ob sie kommen können."
Die Klima-Apokalypse wird zwar nicht dadurch verhindert, dass Politiker im Dezember in Paris eine Vereinbarung dazu getroffen haben. Doch ohne eine solche Vereinbarung würde sie ganz gewiss eines Tages zur Realität. So wie der Vereinbarung erst noch Taten folgen müssen, so werden Menschen zu ebenjenen Taten nur bereit sein, wenn es eine Instanz gibt, die sie orchestriert. Niemand gibt auch nur Teile seiner Lebensweise auf, also von Freiheit, solange er mit Recht sagen kann: "Nützt ja doch nichts." Solange bleibt die Menschheit ein Gewusel aus Räuberbanden - und in dem trachtet jeder nach Beute, solange es noch Beute gibt. In diesem Gewusel wird es immer einen Unterschied geben zwischen abstrakten Einsichten und konkretem Handeln. Ausgerechnet ein katholischer Bischof lebt diesen Unterschied auf unübertreffliche Art vor. Erst schrieb er neulich einen Aufsatz, in dem er "die Industrienationen" aufforderte, die Emissionen zu reduzieren. Dann schaffte er als Dienstwagen einen VW Phaeton an, das Auto, das 224 Gramm Kohlendioxid pro Kilometer ausstößt. Typisch Bischof? Typisch Mensch.
Die Frage des 21. Jahrhunderts ist, ob der Mensch lernen wird, dass Wohlstand nur durch Verzicht zu sichern ist - siehe Merkels Neujahrsansprache von 2009 -, und ob er bereit ist, sich diesen Verzicht organisieren zu lassen. Jeder Gemeinschaft geht es schlecht, wenn der Einzelne alle Freiheiten haben darf; sei es eine Nation oder eine Weltgemeinschaft. Jeder Staat beschneidet - zum Beispiel durch Steuern - die Freiheit seiner Bürger. Wenn die Einzelnen weniger haben, haben alle zusammen mehr. Das ist der Gedanke dabei. Lässt sich dieses Prinzip auf die Welt als Ganzes übertragen?
Es gibt Menschen, die "unsere Art zu leben" mit Gewalt bekämpfen, und es gibt Menschen, denen diese Lebensart ihrerseits Gewalt antut. Auf einen "sozialen Klimawandel" stimmt der Essener Bischof Franz-Josef Overbeck die Deutschen deshalb ein (der mit dem Phaeton, übrigens). Er meint damit, dass Zuwanderung kein vorübergehendes Phänomen sein wird. Nur welches Ausmaß sie bekommen wird, das können die Menschen in den reichen Ländern mitbestimmen - noch.

Wednesday, October 7, 2015

Building a car (from the perspective of a pragmatic software engineer)

The ideas for building a car from the perspective of a pragmatic software engineer came up during the series development of an Internet of Things (IoT) platform and control unit for a German automaker.
The automotive industry is currently undergoing a disruptive technology change mainly driven by software. The self-driving (electric) car will not only change the automotive industry but also society and urban life.
To enable this technology change, the automotive industry finally needs sustainable software platforms that allow for building reliable and scalable software systems in a very short time. (And I am definitely not talking about AUTOSAR here.)
Hence, the automotive industry needs to build software platforms including software development kits (SDK), great APIs, documentation and tools. The automakers must design for simplicity, modularity and security. This requires them to look at the car as a (classical) distributed system of computers and to find good abstractions to easily manage each system and the whole system of systems.
Today, I often see a lot of bad abstractions or no abstractions at all in automotive software systems. That results in slow development cycles, way to much (unreadable) code which is hard to maintain, and at worst, unhappy developers.
Having some good fundamental abstractions would allow for building applications from a set of reusable components that form the basic building blocks of this new software platforms. (And no, I don't have AUTOSAR Software Components in mind.)

So what kinds of software platforms do you need to build a car?
  • Deeply embedded (body) domain (engine, powertrain, light, etc.)
  • Infotainment domain (Head units, dashboards and head-up displays)
  • Internet of Things domain (secure cloud connectivity, eCall, WiFi hotspot, car sharing, over-the-air updates)
  • Driver assistance systems for self-driving cars (highly reliable, real-time number crunchers)
  • Private cloud (machine learning platforms to teach your self-driving cars, data mining to analyse customer behavior, online services like roadside assistance, etc.)
Deeply embedded (body) domain
Automotive real-time software systems of this domain run on embedded controllers with little CPU power and few memory. Therefore, application frameworks and libraries for these kinds of systems avoid or lack dynamic memory allocation and many other well-known abstractions. We have built our own application framework for this kind of deeply embedded systems, called Mindroid.ecpp.

Infotainment domain
Systems in this category are often the same as today's high-end smartphones. So why not use today's smartphone platforms for these kinds of system, or at least parts of it.

Internet of Things domain
Systems in this category are often the same size as today's mid-range smartphones. So why not use today's IoT application frameworks and smartphone platforms for these kinds of systems.

Driver assistance systems (for self-driving cars)
Hardware and software systems in this category are really big computing beasts. Due to its reliability and safety (ASIL) requirements, these are the most challenging and interesting automotive software platforms of the future.

Private clouds
Well, private clouds. Let's do it with Erlang technologies! (Or with any other well-known open source infrastructure.)
To be honest, building a machine learning and data mining platform really is some work to do. Today, only IT giants like Google, Facebook, Amazon, Apple, Microsoft, etc. own and operate such platforms. Hopefully, this will change.

Now let's bring the various car domains (yes, also the private cloud) together. As we learned from Alan Kay, the key in making great and growable systems is much more to design how its modules communicate rather than what their internal properties and behaviors should be. Hence, the connected car needs a simple, clean and robust messaging paradigm.
Choosing form the classical messaging paradigms of distributed systems, I think topic-based publish-subscribe messaging fits best for most use-cases. It allows us to build loosely coupled, flexible, scalable and easy to test (asynchronous) software systems. And it avoids dependencies, something that is really important when evolving systems. Furthermore, a publish-subscribe messaging infrastructure should support quality of service levels for at-most-once, at-least-once and exactly-once delivery to simplify the overall software architecture.
One can also add synchronous messaging systems, like Remote procedure calls. I always try to keep synchronous remote procedure calls to a minimum in order to avoid some of the dependencies (and shared state) they often introduce into distributed systems.
What data to exchange with the publish-subscribe messaging between the domains? Well, let's define a domain model for representing real-world (domain-specific) objects, processes and rules.
E.g. speed, mileage, door states, window states, sequences to safely open the car by a smartphone app, and so on.
Applications must only know about the application framework, the publish-subscribe mechanism and the domain model. They should definitely not know about which protocol to use for domain model exchange, serialization, model binding and many other things. This has to happen within the application framework and a publish-subscribe messaging service.
To evolve the domain model over time, protocols like Google's protocol buffers that allow for changing message formats without breaking backwards-compatibility provide great flexibility.
Using a publish-subscribe messaging mechanism abstracts away all the quirks of automotive bus systems and protocols like CAN, LIN, FlexRay, MOST or even Ethernet.
And of course you never have the "one size fits all" abstraction. E.g. besides publish-subscribe messaging you still need H.264 video streaming over RTP to connect your cameras to a driver assistance system, and others.

To sum up, the automotive industry should build application frameworks with a few simple building blocks for each domain and provide a variety of system services on top of it, e.g. logging, publish-subscribe messaging, power management (using wake locks and leases), etc. - just like Google did with the Open Handset Alliance.

Advice for ambitious system architects and application framework engineers
  • The most dangerous enemy of a better solution is an existing codebase that is just good enough - Eric S. Raymond.
  • Seriously think about threading and message-passing. Get the damn threading right by the application framework for the application developers. You are developing an application framework for developers, not against them!
  • Modularity is there for reusability of components and to ease refactorings within software systems while trying to minimize dependencies between the components. Try to minimize the number of control units within a car!
  • Keep an eye out for beauty and aesthetics in software systems. This has to do with software architecture and naming. Naming is so important!
Links

This is my personal opinion about software engineering in the automotive sector, not that of the company I am working for, nor of anybody else in that company.

Friday, May 2, 2014

RISC design principles for project management

When David A. Patterson and Carlo H. Sequin designed the first RISC processors in the beginning of the 80s they soon realized that designing instructions that could be issued (started) quickly was the key to good performance. While the initial emphasis was on simple instructions that could be executed quickly, they learned, that how long an instruction actually took mattered less than how many could be started per second.
I learned that the same principles apply to good project management in the last few years. Thanks RISC designers!

Tuesday, October 1, 2013

Mindroid - Android everywhere

Mindroid is an application framework that lets you create applications using a set of reusable components - just like Android. The name Mindroid has two different meanings. On one hand Mindroid is a minimal set of core Android classes and on the other hand these classes also form Android's mind (at least in my opinion). There are three versions of Mindroid focusing on different domains.

The feature-richest version is Mindroid.java. It is a complete application framework intended for machine-to-machine (M2M) communication projects and the Internet of Things. It is almost completely Android API-compliant with minor exceptions since Mindroid even runs on Java v1.4 (Java SE and Jave ME CDC) platforms like IBM's J9 Java virtual machine on Qualcomm's modem chipsets running an L4 microkernel OS. Mindroid.java also includes an application framework documentation. For the future, I plan to make Mindroid run on Java ME 8.

Another version of Mindroid is Mindroid.cpp. It is written in C++ using reference counting and other nice gimmicks. Currently, Mindroid.cpp is not really a complete application framework but focuses on Android's messaging and concurrency features. Nevertheless, it provides all messaging abstractions for implementing a lot of applications - the Android Transporter Player is one such example. At the moment Mindroid.cpp runs on POSIX operating systems like Linux, Android or QNX Neutrino RTOS and it uses dynamic memory allocations.

The third version is Mindroid.ecpp. This deeply embedded version of Mindroid is similar to Mindroid.cpp but it runs on tiny embedded operating systems like CMSIS-RTOS for ARM Cortex-M processors and it also runs on AUTOSAR-OS for various embedded processors. Such deeply embedded environments often lack dynamic memory allocation which is also not required by Mindroid.ecpp.

Why?

One of the main reasons for building Mindroid is the fantastic Actor implementation of Google's Android operating system. This actor model is the core building block of Android's whole software architecture. In all versions of Mindroid (and Android) an Actor is implemented by the Looper and Handler classes. A Looper has a thread and a message queue. The threads blocks on the message queue waiting for something to do. To put something into a message queue there is the Handler class. Each Handler also has a handleMessage method for processing messages. An example of this scenario is given in the documentation of the Looper class. Each Handler is attached to exactly one message queue (and thus to one Looper) at creation time. A message object always refers to its Handler that will process it later. Now think of software components in terms of Handlers. Software components (Handlers) talk to each other by passing messages. For example, you can run each Handler on its own Looper (thread) if you like. But you are also allowed to put multiple Handlers on the same Looper. (Android is also able to put multiple Handlers in different operating system processes that communicate with each other using the Binder IPC.) This behavior (software architecture) is changeable at will without a need to change the inner workings of a software component.

How?

Here are "Hello World" examples (using two Handlers) for all versions of Mindroid. One Handler prints "Hello " and the other one "World!" once in a second resulting in "Hello World!" outputs.

Mindroid.java
public class HelloWorld extends Service {
  private static final String LOG_TAG = "HelloWorld";
  private static final int SAY_HELLO = 0;
  private static final int SAY_WORLD = 1;
 
  private Handler mHelloHandler = new Handler() {
    public void handleMessage(Message message) {
      switch (message.what) {
      case SAY_HELLO:
        System.out.print("Hello ");
        mWorldHandler.obtainMessage(SAY_WORLD)
          .sendToTarget();
        break;
      }
    }
  };
 
  private Handler mWorldHandler = new Handler() {
    public void handleMessage(Message message) {
      switch (message.what) {
      case SAY_WORLD:
        System.out.println("World!");
        Message hello = mHelloHandler
          .obtainMessage(SAY_HELLO);
        mHelloHandler.sendMessageDelayed(hello, 1000);
        break;
      }
    }
  };

  public void onCreate() {
    mHelloHandler.obtainMessage(SAY_HELLO)
      .sendToTarget();
  }

  public void onDestroy() {
    mHelloHandler.removeMessages(SAY_HELLO);
    mWorldHandler.removeMessages(SAY_WORLD);
  }

  public IBinder onBind(Intent intent) {
    return null;
  }
}
Mindroid.cpp
static const int SAY_HELLO = 0;
static const int SAY_WORLD = 1;

class HelloHandler : public Handler {
public:
  HelloHandler(const sp<Handler>& worldHandler) :
      mWorldHandler(worldHandler) {
  }

  virtual void handleMessage(const sp<Message>& msg) {
    switch (msg->what) {
    case SAY_HELLO:
      printf("Hello ");
      sp<Message> message = mWorldHandler
        ->obtainMessage(SAY_WORLD);
      message->metaData()->putObject("Handler", this);
      message->sendToTarget();
      break;
    }
  }

private:
  sp<Handler> mWorldHandler;
};

class WorldHandler : public Handler {
public:
  WorldHandler() {
  }

  virtual void handleMessage(const sp<Message>& msg) {
    switch (msg->what) {
    case SAY_WORLD:
      printf("World!\n");
      sp<Handler> helloHandler = msg->metaData()
        ->getObject<Handler>("Handler");
      sp<Message> message = helloHandler
        ->obtainMessage(SAY_HELLO);
      helloHandler->sendMessageDelayed(message, 1000);
      break;
    }
  }
};

int main() {
  Looper::prepare();
  sp<Handler> worldHandler = new WorldHandler();
  sp<Handler> helloHandler = 
    new HelloHandler(worldHandler);
  helloHandler->obtainMessage(SAY_HELLO)
    ->sendToTarget();
  Looper::loop();

  return 0;
}
Mindroid.ecpp
static const int SAY_HELLO = 0;
static const int SAY_WORLD = 1;

class HelloHandler : public Handler {
public:
  HelloHandler(Handler& worldHandler) :
      mWorldHandler(worldHandler) {
  }

  virtual void handleMessage(const Message& msg) {
    switch (msg.what) {
    case SAY_HELLO:
      printf("Hello ");
      mWorldHandler.obtainMessage(mMessage,
          SAY_WORLD);
      mMessage.obj = this;
      mMessage.sendToTarget();
      break;
    }
  }

private:
  Handler& mWorldHandler;
  Message mMessage;
};

class WorldHandler : public Handler {
public:
  WorldHandler() {
  }

  virtual void handleMessage(const Message& msg) {
    switch (msg.what) {
    case SAY_WORLD:
      printf("World!\n");
      Handler* helloHandler = (Handler*) msg.obj;
      helloHandler->obtainMessage(mMessage,
          SAY_HELLO);
      helloHandler->sendMessageDelayed(mMessage,
          1000);
      break;
    }
  }

private:
  Message mMessage;
};

static uint8_t sHelloHandler[sizeof(HelloHandler)];
static uint8_t sWorldHandler[sizeof(WorldHandler)];

int main() {
  Message message;

  Looper::prepare();
  Handler* worldHandler =
    new (sHelloHandler) WorldHandler();
  Handler* helloHandler =
    new (sWorldHandler) HelloHandler(*worldHandler);
  helloHandler->obtainMessage(message, SAY_HELLO);
  message.sendToTarget();
  Looper::loop();

  return 0;
}

For more information on Mindroid please consult the Mindroid.java documentation and the examples provided with the application frameworks on GitHub.

Thursday, April 18, 2013

Concurrency: A single human body cell versus the internet

This week I asked Alan Kay (the inventor of OOP, Smalltalk, the Dynabook, and much more) if there are more concurrent activities going on inside a single human body cell than on the whole internet? Since he is a computer scientist as well as a molecular biologist he seemed to be the right person to answer this question. Here is a summary of the short conversation:

Hi Daniel

Good question -- and we have to work on the question some more in order to get a reasonable comparison.

Since matter is concurrently active at the subatomic level, the more mass we have the more concurrent activities -- so the Internet wins here because it has much more mass.

Things get a lot more tricky at the "cell mechanics" level. Typical human body cells have about 80 billion large molecules of about 10,000 types (the other 70% is water and other small molecules), and these are concurrently active. 
There are several billion computers on the Internet (plus many nuts and bolts computers such as routers) and we can guess that most of these have a few hundred processes each (and each with some number of threads) -- all of which are pseudo-concurrent (it's really how many CPU type things are operating).

So at the hardware level with real CPU like things, there are likely ~ 10 in each computer, and now more with multiples cores, so that gives us an estimate of around 20 billion real concurrent processes not counting routers and other infrastructure.

If we count pseudo-processes (which are logically concurrent), then I think in the Internet we've exceeded a single cell at the active large molecule as a process level of concurrency.

We have about 10 Trillion cells in our body that have our DNA (and about 90 Trillion that are microorganisms of many different kinds).

I think we are safe to say that the concurrent activities in one human body completely dominate the Internet.

However, I have not looked below the CPU level to processes at the hardware level .... perhaps you'd like to take a shot at estimating that!

Cheers,
Alan


Hi Alan,

thanks for your detailed answer. I really appreciate that you took the time to think about this. I came up with this question when thinking about concurrency (maybe also parallelism and message passing) as measurement unit for complex systems. There is no measurement unit for concurrency, I think. But its quite hard to define one, just like comparing the incomparable :-).
...
I also tried to google the answer to this question but since I am a software engineer and only a hobbyist in biology, I wasn't quite sure if I can count all proteins as concurrently active entities or if I should only count ribosomes, motor proteins etc. as concurrently active. On the internet I found that there are about 15.000 ribosomes in one single cell. If only counting ribosomes (which I think is not correct) would reduce the concurrency within a human body cell enormously. Hard to compare...

Thank you very much and best regards,
Daniel


Hi Daniel

You have to at least count proteins that are acting as enzymes -- they are continuously active. Also, you will be interested (and amazed) if you look at the dynamics of biochemistry (i.e. why does it work? How do the molecules "find" each other). These processes are happening concurrently.

I think it is hard to compare -- but mainly for "level" not for estimating how many entities are actually "in process" concurrently.

Cheers,
Alan


Really impressive, isn't it? And we software engineers think that today's multi-core computers provide concurrency and parallelism. |-O

Sunday, November 4, 2012

Concurrent Hello World in Go, Erlang and C++

This week I started taking a deeper look at the Go programming language since I am somewhat addicted to scalability, reliability, concurrency and message passing :-). The first thing I always do when playing around with a new software platform is to write a concurrent "Hello World" program. The program works as follows: One active entity (e.g. thread, Erlang process, Goroutine) has to print "Hello " and another one "World!\n" with the two active entities synchronizing with each other so that the output always is "Hello World!\n". Here is the concurrent Hello World program in Go, Erlang and in C++ using the Mindroid framework.

Go
package main

import "fmt"

func main() {
    sayHello := make(chan string)
    sayWorld := make(chan string)
    quitter := make(chan string)

    go func() {
        for i := 0; i < 1000; i++ {
            fmt.Print("Hello ")
            sayWorld <- "Do it"
            <-sayHello
        }
        sayWorld <- "Quit"
    }()

    go func() {
        for {
            var reply string
            reply = <- sayWorld
            if (reply == "Quit") {
                break
            }
            fmt.Print("World!\n")
            sayHello <- "Do it"
        }
        quitter <- "Done"
    }()
 
    <-quitter
}

Erlang
-module(hello_world).

-export([start/0, say_hello/2, say_world/0]).

say_hello(0, WorldPid) ->
    WorldPid ! quit;

say_hello(N, WorldPid) ->
    io:format("Hello ", []),
    WorldPid ! {sayWorld, self()},
    receive
        sayHello ->
            say_hello(N - 1, WorldPid)
    end.

say_world() ->
    receive
        quit ->
            quit;
        {sayWorld, HelloPid} ->
            io:format("World!~n", []),
            HelloPid ! sayHello,
            say_world()
    end.

start() ->
    WorldPid = spawn(hello_world, say_world, []),
    spawn(hello_world, say_hello, [1000, WorldPid]).

C++ (Mindroid)
#include <stdio.h>
#include "mindroid/os/Message.h"
#include "mindroid/os/Handler.h"
#include "mindroid/os/LooperThread.h"

using namespace mindroid;

static const int SAY_HELLO = 0;
static const int SAY_WORLD = 1;
static const int QUIT = 2;

class HelloHandler : public Handler
{
public:
  HelloHandler(const sp<Handler>& worldHandler) : 
    mWorldHandler(worldHandler), mCounter(0) {}

  virtual void handleMessage(const sp<Message>& msg) {
    switch (msg->what) {
      case SAY_HELLO:
        mCounter++;
        if (mCounter <= 1000) {
          printf("Hello ");
          sp<Message> sayWorld =
            mWorldHandler->obtainMessage(SAY_WORLD);
          sayWorld->metaData()->putObject("SayHello",
            obtainMessage(SAY_HELLO));
          sayWorld->sendToTarget();
        } else {
          mWorldHandler->obtainMessage(QUIT)
            ->sendToTarget();
          Looper::myLooper()->quit();
        }
        break;
    }
  }

private:
  sp<Handler> mWorldHandler;
  int mCounter;
};

class WorldHandler : public Handler
{
public:
  virtual void handleMessage(const sp<Message>& msg) {
    switch (msg->what) {
      case SAY_WORLD:
        printf("World!\n");
        msg->metaData()
          ->getObject<Message>("SayHello")
          ->sendToTarget();
        break;
      case QUIT:
        Looper::myLooper()->quit();
        break;
    }
  }
};

int main() {
  sp< LooperThread<WorldHandler> > wlt =
    new LooperThread<WorldHandler>();
  wlt->start();

  Looper::prepare();
  sp<Handler> hh =
      new HelloHandler(wlt->getHandler());
  hh->obtainMessage(SAY_HELLO)->sendToTarget();
  Looper::loop();

  return 0;
}

I think I really like Go (although Erlang is a bit more crafty :-)). Go eases developing scalable and reliable distributed systems while being a good to read language. Since you can also handle much more concurrent activities with one machine using Goroutines (compared to plain operating system threads) it also helps in saving energy.

Links

Thursday, June 21, 2012

Android Transporter on the Raspberry Pi

The Android Transporter which we have developed at E.S.R.Labs now also runs on the Raspberry Pi. It allows you to easily share and display the contents of an Android gadget wirelessly with a television set or a beamer.

The demo video shows how the Android Transporter turns your smartphone or tablet together with the Raspberry Pi into a powerful media hub to watch movies, play games, or do slideshows.
This way, the Raspberry Pi becomes the cheapest gaming console or set-top box :-).

The Android Transporter for the Raspberry Pi is based on my Android messaging and concurrency (a.k.a. Mindroid) C++ library. The source code of this library is available on GitHub.

Tuesday, May 8, 2012

Android Transporter WiFi Display



The video above shows the Android Transporter, a realtime WiFi Display for Android which I and some other guys have implemented during the last few days. We have done this for our new software startup company called E.S.R.Labs, which is short for Embedded Software Research Labs.
The WiFi Display mirror mode beams the display contents of your smartphone or tablet onto other gadgets, TVs, beamers, or other devices. Soon, there will also be a dual screen mode with an accompanying SDK.
I think WiFi Displays will find their firm place for interconnecting devices to make the world around you a bit smarter :-).

Saturday, March 31, 2012

Android's C++ reference counting

#include <stdio.h>
#include "android/os/Ref.h"

using namespace android::os;

class RefTest : public Ref {
public:
    RefTest(int32_t id) : mID(id) {
        printf("RefTest ctor: %d\n", mID);
    }

    virtual ~RefTest() {
        printf("RefTest dtor: %d\n", mID);
    }

    int32_t id() const {
        return mID;
    }

private:
    int32_t mID;
};

int main() {
    sp<RefTest> ref1 = new RefTest(1);

    {
    sp<RefTest> ref2 = new RefTest(2);
    }

    wp<RefTest> ref3 = ref1;
    sp<RefTest> ref4 = ref3.toStrongRef();
    if (ref4 == NULL) {
        printf("RefTest object is destroyed\n");
    } else {
        printf("RefTest object %d is still around\n",
            ref4->id());
    }
    ref4 = NULL;

    ref1 = NULL;
    ref4 = ref3.toStrongRef();
    if (ref4 == NULL) {
        printf("RefTest object is destroyed\n");
    } else {
        printf("RefTest object %d is still around\n",
            ref4->id());
    }

    return 0;
}

To use the reference counting mechanism for (non Android) C++ projects check out the Ref class of my native Android messaging and concurrency framework. It contains Google's C++ reference counting class from the Android project along with some fixes and refactorings for code readability.

To get started you have to inherit (virtual) public from the Ref base class so that each object of a particular class that derives from Ref contains reference counters and the necessary management methods to increment and decrement them.
Afterwards you are able to create instances of types like RefTest as in the example above and assign them to strong (smart) pointers of type sp<RefTest>. There also is a wp<> template class for weak pointers when you have to get around circles of strong references.

Each Ref object has a WeakRefImpl object that keeps two reference counters, one for weak references and one for strong references. Whenever the strong reference counter is incremented or decremented then the weak reference counter also gets incremented or decremented. But when you create a wp<> to some object only the object's weak reference counter is incremented. Therefore, managed objects live as long as there is some strong pointer referring to it. However, the managed object's WeakRefImpl object may live longer than the object itself because it lives as long as there is some weak pointer referring to the object. If that is the case and the object itself is already destroyed the weak pointer's toStrongRef method returns NULL to indicate that the object is not available anymore.

The Ref base class is thread-safe without needing expensive locks. It completely synchronizes the access to the reference counters using atomic operations. The sp<> and wp<> template classes are not thread-safe. So always make sure you copy the strong and weak pointers when you share managed objects with different threads.

The C++0x shared_ptr class is similar to Android's C++ reference counting class but there are also some differences. E.g. while C++0x keeps the reference counter outside of the managed object in the shared_ptr class Android's reference counting mechanism keeps the counter in the object itself.

Saturday, February 4, 2012

The Erlang Design Pattern

Over the last couple of weeks I did an OO programming experiment. I call it the Erlang design pattern. It is based on the Actor model but goes some steps further. At its core just like the Actor model there are active entities (objects) that have a thread and a message queue with the thread waiting on the message queue to do some stuff.

The Erlang design pattern extends the Actor model by first dividing the software program into active (actors, that have their own thread) and passive (objects, whose methods are called by other actors) objects.
Second, I enforced that all methods of an active or passive object are always called by the same thread context (e.g. actor). So for each object there is no shared state between different thread contexts anymore. Two different objects of the same class could as a matter of course be used by two different actors.
Third, if two or more actors want to share data they have to explicitly do this via message passing. To avoid a lot of copying I shared data using smart pointers in C++ and references in Java and C#. When one actor sends such a data structure to another actor it loses its reference to that data structure to make sure that it is only owned by one actor at a time (see also Microsoft Singularity's exchange heap).

Building software systems using the Erlang design pattern was very interesting because it helped to write concurrent, modular, scalable and readable code :-). Maybe you will try it at your next code retreat.
Of course there also were some quirks. E.g. I couldn't use my C++ closures (which I really like) anymore because they delegate methods of an object from one thread context to another.
To sum this up, the Erlang design pattern was an interesting experiment which I will most probably not use every day in its pure form. But it helps to better understand how ideas of functional programming result in modular, scalable and readable (OO) software systems.

For C++, I used my Android messaging and concurrency (a.k.a. Mindroid) framework to play around with the Erlang design pattern and for Java I just used the Android SDK.

Of Joe Armstrong's Erlang design principles, the Actor model "solves" the first and third point. The Erlang Design pattern also "solves" the second point and goes a step further on the third one.
  • The world is concurrent.
  • Things in the world don't share data.
  • Things communicate with messages.
  • Things fail.
To achieve to same degree of concurrency as Erlang does with its lightweight processes one definitely needs to build its own virtual machine, OS thread abstractions and libraries.
But for smartphone and tablet software development one will most probably not need that degree of concurrency and is able to stick with OS threads :-).

See also:

Sunday, January 1, 2012

How do you read source code?

If software is eating the world as Marc Andreessen and I think, how do you [r]ea(d|t) source code?
Well, let's first answer why you should be good at reading source code at all.
First it's always great fun to figure out how things work. By reading source code one is exactly doing that to learn about interesting software systems and projects.
Another reason for reading source code may be to get better (and faster) at reading and writing software by learning from others and sometimes also from their mistakes.
If you join a new software company or an open source project you are probably going to work on a huge existing codebase and therefore you should be able to get into it quickly, e.g. to implement tests and features or to fix bugs.
The primary goal of reading source code always is to be able to think and reason about all aspects of a software system's codebase. In this article I put together some advise and patterns for reading source code which made my life as software engineer a lot easier :-).

Now the big question is: How do you read source code?
Before you begin to dive deep into the source code of a software project you should make sure to have enough domain specific knowledge to understand the particular piece of software. Hence, you should start to get the big picture by reading documentation and computer science literature about the software platform/product or field of computer science (e.g. Windows apps, Mac OS X and iOS apps, Android apps, operating systems, computer networks, browsers, search engines, databases, etc.).
You don't have to know everything about the topic, but you have to understand the core abstractions and the basic building blocks of the software platform/product. E.g. you should know what processes, threads, semaphores, etc. are before you write your own scheduling algorithm for Linux (see Modern Operating Systems by Andrew S. Tanenbaum). You should also know about Linux specific process management before doing this (see Linux Kernel Development by Robert Love and Linux Kernel Architecture by Wolfgang Mauerer).
But most probably you have already done this before investigating a particular piece of software. So let's get started...

For starters, all software systems or at least all subsystems of huge software systems have some basic building blocks and core abstractions that you will notice all over the place. These components (e.g. classes, modules, actors, data structures, etc.) are also known as hubs. The hubs are simultaneously part of various aspects or subsystems of the whole codebase. Therefore the hubs interlink the subsystems and yet make huge codebases look like small worlds.
Hubs form the contexts around which software engineers build the software architecture. They also implement quite a lot of the core features and functionality. As software systems grow, more and more other components will depend on the hubs. Therefore look for the hubs first and learn about their responsibilities. Usually even huge software systems only have a relatively small number of hubs. Hence, you don't have to fear millions of lines of source code because the hubs will guide you through the codebase. E.g. if we take a look at Google's Android OS, I would say that the following classes (active objects and processes) are the hubs: Zygote, ActivityManagerService, WindowManagerService, PackageManagerService, ConnectivityService and the SurfaceFlinger. You see, just 6 components :-).
You can also repeat the game at a smaller scale, e.g. for Android's widget framework where the View, ViewGroup and ViewRoot classes are the hubs upon which a lot of other UI components build.
This reductionist approach also works for other software systems such as operating systems, filesystems, networking stacks, web backend platforms, etc.
For more details on hubs and network theory I suggest Albert-Laszlo Barabasi's book Linked.

Next, after identifying the hubs you should try to understand the interaction patterns between the hubs. The interactions may rely on different mechanisms like pure API calls or message passing (e.g. message queues or IPC calls). To get the idea of how hubs depend on each other I suggest to just draw some pictures of the hubs, their dependencies and their interactions.
As an example just take a look at one of my previous blog posts about Andoid Architecture Patterns.
On the 7th slide there is a picture about how Android starts activities, services and content providers within their own Linux OS processes. It does so by several interactions between the ActivityManagerService, the Zygote process and the app's process.

As you see, getting the big picture is done by identifying the hubs and understanding their interactions using a top-down approach. To dig deep into specific parts or aspects of software systems we have to change our source code reading patterns. Therefore we will switch to a bottom-up approach to inspect modules, classes, data structures, methods, functions, etc. Later we are able to combine both code reading approches. This strategy of summarizing the findings of the top-down and the bottom-up approach is called downward causation.

I think the bottom-up approach works best by starting with the active entities (threads, actors, processes) that breathe life into the hubs. This is because to be able to think and reason about some piece of source code you really need to understand the environment in which the hubs run.
So always make sure which active entities run which parts of a system's source code and try to understand how and when they interact with each other. This will help you to achieve the primary goal of reading source code, that is to be able to think and reason about all aspects of a software system's codebase (solely with your brain and without the help of external tools like a debugger :-)).
Getting into the details of some piece of source code always starts with trying things out. I do that by adding some logging code or by making assumptions about the code's behavior which I verify with tests afterwards. Another method is to do modifications to the source code just to check how the code behaves under the new circumstances. Breaking or damaging the code may also help you to learn about it ;-).
While reading source code always ask yourself: "How does it work?" and "Why have the developers done it that way?". This will most probably cause you some sleepless nights but it will also make you a better software engineer and software architect.
Everything you do to get better at thinking and reasoning about the source code will help you to develop stronger debugging and analytical skills which in turn enable you to implement new features, fix bugs or do refactorings.

By thinking and reflecting about the source code you are reading you will learn a lot about how to write software systems and platforms. Besides, from bad software you will also learn what to avoid when developing software systems.
Furthermore, there are two great articles about how to write great source code and software systems. Rich Hickey's talk about "Simple made easy" at InfoQ and Erlang's programming rules and conventions. These two guides are outstanding no matter which programming language you use.
So, reading code really is fun. Maybe next time instead of reading another software engineering book just read some source code. (GitHub is really great for that.)

Since you need some staying power to get into a huge codebase I suggest to pick a software project that provides some fun and purpose along the way :-). Maybe the list below contains an interesting software project for you...

Software projects