Lessons Learned from a Rube Goldberg Software written in Java, Python and C++

BY MARKUS SPRUNCK

A Rube Goldberg Software is a deliberately over-engineered software to perform a simple task in a complicated fashion. The term is inspired by the so called Rube Goldberg Machines. Some weeks ago, I saw an awesome video (link) of a Rube Goldberg Machine and I decided to write a Rube Goldberg software with three programming general purpose languages.

In the case you like this software, you may add an implementation in your favorite language. Just fork the code on GitHub.

By Rube Goldberg [Public domain], via Wikimedia Commons


Design, Execution and Expected Output

Three programs in different languages, i.e. Java, Python and C++ and with identical functionality create and call each other in a cyclic manner. The weird thing is, that each program creates, compiles and executes the next from the scratch without loading the sources from the filesystem. So, every program must know the code of all the others and start them in a asynchronous way.

A Starter application creates the file HelloWorld.java in the working directory, compiles it and executes it with two command line arguments. This Java program creates and executes a Python program. The Python program creates, compiles and executes a C++ program. Then the C++ program creates, compiles and executes the initial Java program.

This cycle is executed several times and all three programs write into a file called HelloWorld.log.

In the first command line argument the source code of Java, Python and C++ is encoded. In the second argument is defined how often the Java program should be executed.

Stopping the application after a finite number of runs and delaying the execution is important. Because when you have processes which live just some milli seconds and start then the next process are difficult to kill the processes.

Expected output

2016-06-04T23:12:40.097 - JavaSE - round 3

2016-06-04T23:12:40.111 - JavaSE - Hello, World!

2016-06-04T23:12:40.119 - JavaSE - delete HelloWorld.py succeeded=false

2016-06-04T23:12:40.119 - JavaSE - create HelloWorld.py

2016-06-04T23:12:40.123 - JavaSE - execute HelloWorld.py

2016-06-04T23:12:40.124 - JavaSE - end

2016-06-04T23:12:40.124 - JavaSE - exit

2016-06-04T23:12:43.170 - Python - round 2

2016-06-04T23:12:43.170 - Python - Hello, World!

2016-06-04T23:12:43.171 - Python - delete HelloWorld.cpp succeeded=true

2016-06-04T23:12:43.171 - Python - create HelloWorld.cpp

2016-06-04T23:12:43.172 - Python - execute

2016-06-04T23:12:44.003 - Python - end

2016-06-04T23:12:44.003 - Python - exit

2016-06-04T23:12:47.009 - cpp - round 2

2016-06-04T23:12:47.009 - cpp - Hello, World!

2016-06-04T23:12:47.010 - cpp - delete HelloWorld.java succeeded=true

2016-06-04T23:12:47.010 - cpp - create HelloWorld.java

2016-06-04T23:12:47.012 - cpp - execute HelloWorld.java

2016-06-04T23:12:47.759 - cpp - end

2016-06-04T23:12:47.759 - cpp - exit

2016-06-04T23:12:50.931 - JavaSE - round 2

2016-06-04T23:12:50.940 - JavaSE - Hello, World!

2016-06-04T23:12:50.946 - JavaSE - delete HelloWorld.py succeeded=true

2016-06-04T23:12:50.946 - JavaSE - create HelloWorld.py

2016-06-04T23:12:50.949 - JavaSE - execute HelloWorld.py

2016-06-04T23:12:50.950 - JavaSE - end

2016-06-04T23:12:50.950 - JavaSE - exit

2016-06-04T23:12:53.988 - Python - round 1

2016-06-04T23:12:53.988 - Python - Hello, World!

2016-06-04T23:12:53.990 - Python - delete HelloWorld.cpp succeeded=true

2016-06-04T23:12:53.990 - Python - create HelloWorld.cpp

2016-06-04T23:12:53.990 - Python - execute

2016-06-04T23:12:54.384 - Python - end

2016-06-04T23:12:54.384 - Python - exit

2016-06-04T23:12:57.391 - cpp - round 1

2016-06-04T23:12:57.393 - cpp - Hello, World!

2016-06-04T23:12:57.394 - cpp - delete HelloWorld.java succeeded=true

2016-06-04T23:12:57.394 - cpp - create HelloWorld.java

2016-06-04T23:12:57.397 - cpp - execute HelloWorld.java

2016-06-04T23:12:58.101 - cpp - end

2016-06-04T23:12:58.101 - cpp - exit

2016-06-04T23:13:01.279 - JavaSE - round 1

2016-06-04T23:13:01.290 - JavaSE - Hello, World!

2016-06-04T23:13:01.296 - JavaSE - delete HelloWorld.py succeeded=true

2016-06-04T23:13:01.296 - JavaSE - create HelloWorld.py

2016-06-04T23:13:01.299 - JavaSE - execute HelloWorld.py

2016-06-04T23:13:01.300 - JavaSE - end

2016-06-04T23:13:01.301 - JavaSE - exit

2016-06-04T23:13:04.342 - Python - round 0

2016-06-04T23:13:04.343 - Python - stopped

During development, when the first base functionality worked, I have had to reboot my development machine - because I had problems to stop the circle.

Lessons Learned

The source code size is quite different a) Python 70 lines of code, b) Java has 91 lines of code and c) C++ 154 lines of code. My expectation was that all the programs have almost the same size.

Two reasons - why the C++ code is so large - can be found in the facts that C++ has missing support of base64 decoding and the need to start a new thread to execute a process asynchronously.

Additionally the C++ code is not platform independent. Starting on Mac OS X and Windows differs and the code is not platform independent. Here I decided not to code for Windows, but this would result some additional lines.

The key problem with C++ is the missing support of base functionality in the standard libraries. Python and Java are here the clear winners.

Some empirical investigations indicate that Java and C++ are in the same range in the category source code size (see https://page.mi.fu-berlin.de/prechelt/Biblio/jccpprt_computer2000.pdf) but for this example this is not true.

Even if the expressiveness of Java and C++ should be almost the same, the standard library support decides how much work you will have to develop real world software. And this is quite often ignored when just the expressiveness of software is compared.


Code of the Java SE Starter

The Starter program is not necessarily needed to start, but it is more convenient to calculate the arguments automatically. As already mentioned each program must know all the source code and the best is to provide this information is with a command line argument. The source code is encoded as base64 string and concatenated with a space and then encoded again as base64 string. This gives a single argument with all the needed source code information.

The second argument is the number of rounds and the default value is 3. The argument is the same for all programs. They decode the source code for the next process and create a file.

import java.io.IOException;

import java.nio.file.Files;

import java.nio.file.Paths;

import java.util.Base64;

import static java.nio.file.StandardCopyOption.*;


public class Starter {

private static final String CPP_SRC_PATH = "../cpp/src/HelloWorld.cpp";

private static final String PYTHON_SRC_PATH = "../python/src/HelloWorld.py";

private static final String JAVA_SE_SRC_PATH = "../java-se/src/HelloWorld.java";

public static void main(String[] args) throws IOException, InterruptedException {

// read all sources and encode

final String contentJava = new String(Files.readAllBytes(Paths.get(JAVA_SE_SRC_PATH)));

final String contentPython = new String(Files.readAllBytes(Paths.get(PYTHON_SRC_PATH)));

final String contentCpp = new String(Files.readAllBytes(Paths.get(CPP_SRC_PATH)));

final String contentJavaEncoded = Base64.getEncoder().encodeToString(contentJava.getBytes());

final String contentPythonEncoded = Base64.getEncoder().encodeToString(contentPython.getBytes());

final String contentCppEncoded = Base64.getEncoder().encodeToString(contentCpp.getBytes());

final String contentAll =

String.format("%s %s %s", contentJavaEncoded, contentPythonEncoded, contentCppEncoded);

final String contentAllEncoded = Base64.getEncoder().encodeToString(contentAll.getBytes());

System.out.print(contentAllEncoded);

// cleanup

Files.deleteIfExists(Paths.get("HelloWorld.log"));

Files.deleteIfExists(Paths.get("HelloWorld.java"));

Files.deleteIfExists(Paths.get("HelloWorld.class"));

Files.deleteIfExists(Paths.get("HelloWorld$1.class"));

Files.deleteIfExists(Paths.get("HelloWorld.cpp"));

Files.deleteIfExists(Paths.get("HelloWorld"));

Files.deleteIfExists(Paths.get("HelloWorld.py"));

Thread.sleep(1000);

// start

Files.copy(Paths.get(JAVA_SE_SRC_PATH), Paths.get(System.getProperty("user.dir")

+ "/HelloWorld.java"), REPLACE_EXISTING);

new ProcessBuilder("javac", "HelloWorld.java").start().waitFor();

new ProcessBuilder("java", "HelloWorld", contentAllEncoded, "3").start();

}

}

Additionally the starter cleans all intermediate files from the execution directory.

Code of the JavaSE HelloWorld

This code is strait forward and the only special thing is the Thread (line 27) which has to be created to start the Python program asynchronously.

import java.io.BufferedWriter;

import java.io.FileWriter;

import java.io.IOException;

import java.io.PrintWriter;

import java.nio.file.Files;

import java.nio.file.Paths;

import java.time.LocalDateTime;

import java.util.Base64;


public class HelloWorld {

static final String LOG_FILE = "HelloWorld.log";

private static final String RESULT_FILE = "HelloWorld.py";

private final String contentAllEncoded;

private final Integer numberOfRounds;

public HelloWorld(String contentAllEncoded, Integer numberOfRounds) {

this.contentAllEncoded = contentAllEncoded;

this.numberOfRounds = numberOfRounds;

}

private void executeProgram() {

appendMessage(LOG_FILE, " - JavaSE - execute " + RESULT_FILE, true);

new Thread() {

public void run() {

Integer remainingNumberOfRounds = (numberOfRounds - 1);

ProcessBuilder pb = new ProcessBuilder("python", RESULT_FILE,

contentAllEncoded, remainingNumberOfRounds.toString());

try {

pb.start();

} catch (IOException e) {

e.printStackTrace();

}

}

}.start();

}

private void createProgram() throws IOException {

boolean succeeded = Files.deleteIfExists(Paths.get(RESULT_FILE));

appendMessage(LOG_FILE,

" - JavaSE - delete " + RESULT_FILE + " succeeded=" + succeeded,

true);

appendMessage(LOG_FILE, " - JavaSE - create " + RESULT_FILE, true);

String contentPythonEncoded =

new String(Base64.getDecoder().decode(contentAllEncoded),

"UTF-8").split(" ")[1];

String contentPython =

new String(Base64.getDecoder().decode(contentPythonEncoded), "UTF-8");

appendMessage(RESULT_FILE, contentPython, false);

}

private void appendMessage(String file, String message, boolean addTimeStamp) {

try (PrintWriter out =

new PrintWriter(new BufferedWriter(new FileWriter(file, true)))) {

if (addTimeStamp) {

LocalDateTime time = LocalDateTime.now();

out.print(time.toString());

}

out.println(message);

out.flush();

} catch (IOException e) {

System.err.println(e.getMessage());

}

}

private void run() throws IOException {

appendMessage(LOG_FILE, " - JavaSE - Hello, World!", true);

createProgram();

executeProgram();

appendMessage(LOG_FILE, " - JavaSE - end", true);

}

public static void main(String[] args) throws IOException, InterruptedException {

// wait

Thread.sleep(3000);

final String contentAll = args[0];

final Integer numberOfRounds = Integer.parseInt(args[1]);

HelloWorld helloWorld = new HelloWorld(contentAll, numberOfRounds);

helloWorld.appendMessage(LOG_FILE, " - JavaSE - round " + numberOfRounds, true);

if (numberOfRounds > 0) {

helloWorld.run();

helloWorld.appendMessage(LOG_FILE, " - JavaSE - exit", true);

} else {

helloWorld.appendMessage(LOG_FILE, " - JavaSE - stopped", true);

}

}

}

Code of the Python HelloWorld

In Python no extra Thread has to be created.

import base64

import os

import sys

import time

import datetime

import subprocess

from subprocess import Popen


class HelloWorld(object):

LOG_FILE = "HelloWorld.log";

RESULT_FILE = "HelloWorld.cpp";

def __init__(self, contentAll):

self.contentAll = contentAll

self.numberOfRounds = numberOfRounds

def executeProgram(self):

self.appendMessage(self.LOG_FILE, " - Python - execute", True);

args = ['c++', self.RESULT_FILE, '-o', 'HelloWorld']

subprocess.call(args)

args = ['./HelloWorld', self.contentAll, self.numberOfRounds]

Popen(args)

def createProgram(self):

if os.path.exists(self.RESULT_FILE):

os.remove(self.RESULT_FILE)

os.remove(self.RESULT_FILE[:-4])

os.remove(self.RESULT_FILE[:-4] + ".class")

os.remove(self.RESULT_FILE[:-4] + "$1.class")

self.appendMessage(self.LOG_FILE, " - Python - delete " + self.RESULT_FILE

+ " succeeded="

+ str(not os.path.exists(self.RESULT_FILE)).lower(), True);

self.appendMessage(self.LOG_FILE, " - Python - create " + self.RESULT_FILE, True);

contentCppEncoded = base64.b64decode(self.contentAll).split(' ')[2]

contentCpp = base64.b64decode(contentCppEncoded)

self.appendMessage(self.RESULT_FILE, contentCpp, False);

def appendMessage(self, file, message, addTimeStamp):

f = open(file, 'a+')

if (addTimeStamp):

time = datetime.datetime.now()

f.write(time.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-3])

f.write(message)

f.write('\n')

f.flush()

f.close()

def run(self):

self.appendMessage(self.LOG_FILE, " - Python - Hello, World!", True);

self.createProgram()

self.executeProgram()

self.appendMessage(self.LOG_FILE, " - Python - end", True);


if __name__ == '__main__':

# wait

time.sleep(3)

contentAll = sys.argv[1]

numberOfRounds = sys.argv[2]

helloWorld = HelloWorld(contentAll)

helloWorld.appendMessage(helloWorld.LOG_FILE, " - Python - round " + str(sys.argv[2]), True);

if int(numberOfRounds) > 0:

helloWorld.run()

helloWorld.appendMessage(helloWorld.LOG_FILE, " - Python - exit", True);

else:

helloWorld.appendMessage(helloWorld.LOG_FILE, " - Python - stopped", True);

Code of the C++ HelloWorld

In C++ a lot of boiler plate code is needed, e.g. base64 encoding, proper formatting of time stamps.

#include <stdio.h>

#include <sys/time.h>

#include <unistd.h>

#include <cstdlib>

#include <ctime>

#include <fstream>

#include <iostream>

#include <iterator>

#include <sstream>

#include <string>

#include <vector>


using namespace std;


class HelloWorld {


public:

static const string LOG_FILE;


static const string RESULT_FILE;


static const string BASE64_CHAR;


private:


string base64_decode(string const& encoded_string) {

size_t in_len = encoded_string.size();

size_t i = 0;

size_t j = 0;

int in_ = 0;

unsigned char char_array_4[4], char_array_3[3];

std::string ret;


while (in_len-- && (encoded_string[in_] != '=')) {

char_array_4[i++] = encoded_string[in_];

in_++;

if (i == 4) {

for (i = 0; i < 4; i++)

char_array_4[i] = static_cast<unsigned char>(BASE64_CHAR.find(char_array_4[i]));


char_array_3[0] = (char_array_4[0] << 2) + ((char_array_4[1] & 0x30) >> 4);

char_array_3[1] = ((char_array_4[1] & 0xf) << 4) + ((char_array_4[2] & 0x3c) >> 2);

char_array_3[2] = ((char_array_4[2] & 0x3) << 6) + char_array_4[3];


for (i = 0; (i < 3); i++)

ret += char_array_3[i];

i = 0;

}

}


if (i) {

for (j = i; j < 4; j++)

char_array_4[j] = 0;


for (j = 0; j < 4; j++)

char_array_4[j] =

static_cast<unsigned char>(BASE64_CHAR.find(char_array_4[j]));


char_array_3[0] = (char_array_4[0] << 2) + ((char_array_4[1] & 0x30) >> 4);

char_array_3[1] = ((char_array_4[1] & 0xf) << 4) + ((char_array_4[2] & 0x3c) >> 2);

char_array_3[2] = ((char_array_4[2] & 0x3) << 6) + char_array_4[3];


for (j = 0; (j < i - 1); j++)

ret += char_array_3[j];

}

return (ret);

}


private:

string contentAllEncoded;

string numberOfRounds;


public:

HelloWorld(string contentAllEncoded, string numberOfRounds) {

this->contentAllEncoded = contentAllEncoded;

this->numberOfRounds = numberOfRounds;

}


void executeProgram() {

appendMessage(LOG_FILE, " - cpp - execute " + RESULT_FILE, true);

std::system("javac HelloWorld.java");

string command = "java HelloWorld " + this->contentAllEncoded + " " + this->numberOfRounds

+ " &";

std::system(command.c_str());

}


void createProgram() {

remove(RESULT_FILE.c_str());

appendMessage(LOG_FILE,

" - cpp - delete " + RESULT_FILE + " succeeded="

+ ((!std::ifstream(RESULT_FILE.c_str())) ? "true" : "false"), true);


appendMessage(LOG_FILE, " - cpp - create " + RESULT_FILE, true);

string contentAll = base64_decode(contentAllEncoded);

istringstream buf(contentAll);

istream_iterator<string> beg(buf), end;

vector<string> tokens(beg, end);

string contentCppEncoded = tokens.at(0);

string contentCpp = base64_decode(contentCppEncoded);

appendMessage(RESULT_FILE, contentCpp, false);

}


void appendMessage(std::string file, std::string message, bool addTimeStamp) {

ofstream outfile;

outfile.open(file, ios_base::app);

if (addTimeStamp) {

time_t now;

time(&now);

timeval curTime;

gettimeofday(&curTime, NULL);

int milli = curTime.tv_usec / 1000;

char buf[sizeof "2011-10-08T07:07:090"];

strftime(buf, sizeof buf, "%FT%T", localtime(&now));

sprintf(buf, "%s.%03d", buf, milli);

outfile << buf;

}

outfile << message << endl;

outfile.flush();

outfile.close();

}


void run() {

appendMessage(LOG_FILE, " - cpp - Hello, World!", true);

createProgram();

executeProgram();

appendMessage(LOG_FILE, " - cpp - end", true);

}


};


const string HelloWorld::LOG_FILE = "HelloWorld.log";


const string HelloWorld::RESULT_FILE = "HelloWorld.java";


const string HelloWorld::BASE64_CHAR =

"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";


int main(int argc, char *argv[]) {

// wait

usleep(3000000u);


string contentAll = argv[1];

string numberOfRounds = argv[2];

HelloWorld* helloWorld = new HelloWorld(contentAll, numberOfRounds);

helloWorld->appendMessage(HelloWorld::LOG_FILE, string(" - cpp - round ") + numberOfRounds,

true);

if (stoi(numberOfRounds) > 0) {

helloWorld->run();

helloWorld->appendMessage(HelloWorld::LOG_FILE, " - cpp - exit", true);

} else {

helloWorld->appendMessage(HelloWorld::LOG_FILE, " - cpp - stopped", true);

}

return (0);

}

Setup Development Environment (Mac OS X)

Figure 1: Project structure in Eclipse