More‎ > ‎

Lessons Learned from a Rube Goldberg Software written in Java, Python and C++

BY MARKUS SPRUNCK


By Rube Goldberg [Public domain], via Wikimedia Commons
A Rube Goldberg Software is a deliberately over-engineered software to perform a simple task in a complicated fashion. 
The term is inspired by the so called Rube Goldberg Machines. Some weeks ago, I saw an awesome video (link) of a Rube Goldberg Machine and I decided to write a Rube Goldberg software with three programming general purpose languages. 

In the case you like this software, you may add an implementation in your favorite language. Just fork the code on GitHub.

Design, Execution and Expected Output

Three programs in different languages, i.e. Java, Python and C++ and with identical functionality create and call each other in a cyclic manner. The weird thing is, that each program creates, compiles and executes the next from the scratch without loading the sources from the filesystem. So, every program must know the code of all the others and start them in a asynchronous way.

A Starter application creates the file HelloWorld.java in the working directory, compiles it and executes it with two command line arguments. This Java program creates and executes a Python program. The Python program creates, compiles and executes a C++ program. Then the C++ program creates, compiles and executes the initial Java program. 

This cycle is executed several times and all three programs write into a file called HelloWorld.log. 

In the first command line argument the source code of Java, Python and C++ is encoded. In the second argument is defined how often the Java program should be executed.

Stopping the application after a finite number of runs and delaying the execution is important. Because when you have processes which live just some milli seconds and start then the next process are difficult to kill the processes. 

Expected output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
2016-06-04T23:12:40.097 - JavaSE - round 3
2016-06-04T23:12:40.111 - JavaSE - Hello, World!
2016-06-04T23:12:40.119 - JavaSE - delete  HelloWorld.py    succeeded=false
2016-06-04T23:12:40.119 - JavaSE - create  HelloWorld.py
2016-06-04T23:12:40.123 - JavaSE - execute HelloWorld.py
2016-06-04T23:12:40.124 - JavaSE - end
2016-06-04T23:12:40.124 - JavaSE - exit
2016-06-04T23:12:43.170 - Python - round 2
2016-06-04T23:12:43.170 - Python - Hello, World!
2016-06-04T23:12:43.171 - Python - delete HelloWorld.cpp    succeeded=true
2016-06-04T23:12:43.171 - Python - create HelloWorld.cpp
2016-06-04T23:12:43.172 - Python - execute
2016-06-04T23:12:44.003 - Python - end
2016-06-04T23:12:44.003 - Python - exit
2016-06-04T23:12:47.009 - cpp    - round 2
2016-06-04T23:12:47.009 - cpp    - Hello, World!
2016-06-04T23:12:47.010 - cpp    - delete  HelloWorld.java  succeeded=true
2016-06-04T23:12:47.010 - cpp    - create  HelloWorld.java
2016-06-04T23:12:47.012 - cpp    - execute HelloWorld.java
2016-06-04T23:12:47.759 - cpp    - end
2016-06-04T23:12:47.759 - cpp    - exit
2016-06-04T23:12:50.931 - JavaSE - round 2
2016-06-04T23:12:50.940 - JavaSE - Hello, World!
2016-06-04T23:12:50.946 - JavaSE - delete  HelloWorld.py    succeeded=true
2016-06-04T23:12:50.946 - JavaSE - create  HelloWorld.py
2016-06-04T23:12:50.949 - JavaSE - execute HelloWorld.py
2016-06-04T23:12:50.950 - JavaSE - end
2016-06-04T23:12:50.950 - JavaSE - exit
2016-06-04T23:12:53.988 - Python - round 1
2016-06-04T23:12:53.988 - Python - Hello, World!
2016-06-04T23:12:53.990 - Python - delete HelloWorld.cpp    succeeded=true
2016-06-04T23:12:53.990 - Python - create HelloWorld.cpp
2016-06-04T23:12:53.990 - Python - execute
2016-06-04T23:12:54.384 - Python - end
2016-06-04T23:12:54.384 - Python - exit
2016-06-04T23:12:57.391 - cpp    - round 1
2016-06-04T23:12:57.393 - cpp    - Hello, World!
2016-06-04T23:12:57.394 - cpp    - delete  HelloWorld.java  succeeded=true
2016-06-04T23:12:57.394 - cpp    - create  HelloWorld.java
2016-06-04T23:12:57.397 - cpp    - execute HelloWorld.java
2016-06-04T23:12:58.101 - cpp    - end
2016-06-04T23:12:58.101 - cpp    - exit
2016-06-04T23:13:01.279 - JavaSE - round 1
2016-06-04T23:13:01.290 - JavaSE - Hello, World!
2016-06-04T23:13:01.296 - JavaSE - delete  HelloWorld.py    succeeded=true
2016-06-04T23:13:01.296 - JavaSE - create  HelloWorld.py
2016-06-04T23:13:01.299 - JavaSE - execute HelloWorld.py
2016-06-04T23:13:01.300 - JavaSE - end
2016-06-04T23:13:01.301 - JavaSE - exit
2016-06-04T23:13:04.342 - Python - round 0
2016-06-04T23:13:04.343 - Python - stopped  

During development, when the first base functionality worked, 
I have had to reboot my development machine - because I had problems to stop the circle.

Lessons Learned

The source code size is quite different a) Python 70 lines of code, b) Java has 91 lines of code and c) C++ 154 lines of code. My expectation was that all the programs have almost the same size. 

Two reasons - why the C++ code is so large - can be found in the facts that C++ has missing support of base64 decoding and the need to start a new thread to execute a process asynchronously.

Additionally the C++ code is not platform independent. Starting on Mac OS X and Windows differs and the code is not platform independent. Here I decided not to code for Windows, but this would result some additional lines. 

The key problem with C++ is the missing support of base functionality in the standard libraries. Python and Java are here the clear winners.

Some empirical investigations indicate that Java and C++ are in the same range in the category source code size (see https://page.mi.fu-berlin.de/prechelt/Biblio/jccpprt_computer2000.pdf) but for this example this is not true. 

Even if the expressiveness of Java and C++ should be almost the same, the standard library support decides how much work you will have to develop real world software. And this is quite often ignored when just the expressiveness of software is compared.

Code of the JavaSE Starter

The Starter program is not necessarily needed to start, but it is more convenient to calculate the arguments automatically. As already mentioned each program must know all the source code and the best is to provide this information is with a command line argument. The source code is encoded as base64 string and concatenated with a space and then encoded again as base64 string. This gives a single argument with all the needed source code information. 

The second argument is the number of rounds and the default value is 3. The argument is the same for all programs. They decode the source code for the next process and create a file.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Base64;
import static java.nio.file.StandardCopyOption.*;

public class Starter {
    
    private static final String CPP_SRC_PATH = "../cpp/src/HelloWorld.cpp";
    private static final String PYTHON_SRC_PATH = "../python/src/HelloWorld.py";
    private static final String JAVA_SE_SRC_PATH = "../java-se/src/HelloWorld.java";
    
    public static void main(String[] args) throws IOException, InterruptedException {
        
        // read all sources and encode
        final String contentJava = new String(Files.readAllBytes(Paths.get(JAVA_SE_SRC_PATH)));
        final String contentPython = new String(Files.readAllBytes(Paths.get(PYTHON_SRC_PATH)));
        final String contentCpp = new String(Files.readAllBytes(Paths.get(CPP_SRC_PATH)));
        final String contentJavaEncoded = Base64.getEncoder().encodeToString(contentJava.getBytes());
        final String contentPythonEncoded = Base64.getEncoder().encodeToString(contentPython.getBytes());
        final String contentCppEncoded = Base64.getEncoder().encodeToString(contentCpp.getBytes());
        final String contentAll = 
                    String.format("%s %s %s", contentJavaEncoded, contentPythonEncoded, contentCppEncoded);
        final String contentAllEncoded = Base64.getEncoder().encodeToString(contentAll.getBytes());
        System.out.print(contentAllEncoded);
        
        // cleanup
        Files.deleteIfExists(Paths.get("HelloWorld.log"));
        Files.deleteIfExists(Paths.get("HelloWorld.java"));
        Files.deleteIfExists(Paths.get("HelloWorld.class"));
        Files.deleteIfExists(Paths.get("HelloWorld$1.class"));
        Files.deleteIfExists(Paths.get("HelloWorld.cpp"));
        Files.deleteIfExists(Paths.get("HelloWorld"));
        Files.deleteIfExists(Paths.get("HelloWorld.py"));
        Thread.sleep(1000);
        
        // start
        Files.copy(Paths.get(JAVA_SE_SRC_PATH), Paths.get(System.getProperty("user.dir") 
            + "/HelloWorld.java"), REPLACE_EXISTING);
        new ProcessBuilder("javac", "HelloWorld.java").start().waitFor();
        new ProcessBuilder("java", "HelloWorld", contentAllEncoded, "3").start();
    }
}

Additionally the starter cleans all intermediate files from the execution directory.

Code of the JavaSE HelloWorld

This code is strait forward and the only special thing is the Thread (line 27) which has to be created to start the Python program asynchronously.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.IOException;
import java.io.PrintWriter;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.time.LocalDateTime;
import java.util.Base64;

public class HelloWorld {
    
    static final String LOG_FILE = "HelloWorld.log";
    
    private static final String RESULT_FILE = "HelloWorld.py";
    
    private final String contentAllEncoded;
    
    private final Integer numberOfRounds;
    
    public HelloWorld(String contentAllEncoded, Integer numberOfRounds) {
        this.contentAllEncoded = contentAllEncoded;
        this.numberOfRounds = numberOfRounds;
    }
    
    private void executeProgram() {
        appendMessage(LOG_FILE, " - JavaSE - execute " + RESULT_FILE, true);
        new Thread() {
            public void run() {
                Integer remainingNumberOfRounds = (numberOfRounds - 1);
                ProcessBuilder pb = new ProcessBuilder("python", RESULT_FILE, 
                        contentAllEncoded, remainingNumberOfRounds.toString());
                try {
                    pb.start();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }.start();
    }
    
    private void createProgram() throws IOException {
        boolean succeeded = Files.deleteIfExists(Paths.get(RESULT_FILE));
        appendMessage(LOG_FILE,
                " - JavaSE - delete  " + RESULT_FILE + "    succeeded=" + succeeded, 
               true);
        appendMessage(LOG_FILE, " - JavaSE - create  " + RESULT_FILE, true);
        String contentPythonEncoded = 
                new String(Base64.getDecoder().decode(contentAllEncoded),
                "UTF-8").split(" ")[1];
        String contentPython = 
                new String(Base64.getDecoder().decode(contentPythonEncoded), "UTF-8");
        appendMessage(RESULT_FILE, contentPython, false);
    }
    
    private void appendMessage(String file, String message, boolean addTimeStamp) {
        try (PrintWriter out = 
        new PrintWriter(new BufferedWriter(new FileWriter(file, true)))) {
            if (addTimeStamp) {
                LocalDateTime time = LocalDateTime.now();
                out.print(time.toString());
            }
            out.println(message);
            out.flush();
        } catch (IOException e) {
            System.err.println(e.getMessage());
        }
    }
    
    private void run() throws IOException {
        appendMessage(LOG_FILE, " - JavaSE - Hello, World!", true);
        createProgram();
        executeProgram();
        appendMessage(LOG_FILE, " - JavaSE - end", true);
    }
    
    public static void main(String[] args) throws IOException, InterruptedException {
        // wait
        Thread.sleep(3000);
        
        final String contentAll = args[0];
        final Integer numberOfRounds = Integer.parseInt(args[1]);
        HelloWorld helloWorld = new HelloWorld(contentAll, numberOfRounds);
        helloWorld.appendMessage(LOG_FILE, " - JavaSE - round " + numberOfRounds, true);
        if (numberOfRounds > 0) {
            helloWorld.run();
            helloWorld.appendMessage(LOG_FILE, " - JavaSE - exit", true);
        } else {
            helloWorld.appendMessage(LOG_FILE, " - JavaSE - stopped", true);
        }
    }
}

Code of the Python HelloWorld

In Python no extra Thread has to be created. 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
import base64
import os
import sys
import time
import datetime
import subprocess
from subprocess import Popen

class HelloWorld(object):
    
    LOG_FILE = "HelloWorld.log";
    
    RESULT_FILE = "HelloWorld.cpp";
    
    def __init__(self, contentAll):
        self.contentAll = contentAll
        self.numberOfRounds = numberOfRounds
    
    def executeProgram(self):
        self.appendMessage(self.LOG_FILE, " - Python - execute", True);
  
        args = ['c++', self.RESULT_FILE, '-o', 'HelloWorld']
        subprocess.call(args) 
        args = ['./HelloWorld', self.contentAll, self.numberOfRounds]
        Popen(args) 
  
    def createProgram(self):
        if os.path.exists(self.RESULT_FILE):
            os.remove(self.RESULT_FILE)
            os.remove(self.RESULT_FILE[:-4])
            os.remove(self.RESULT_FILE[:-4] + ".class")
            os.remove(self.RESULT_FILE[:-4] + "$1.class")
        self.appendMessage(self.LOG_FILE, " - Python - delete " + self.RESULT_FILE 
            + "    succeeded=" 
            + str(not os.path.exists(self.RESULT_FILE)).lower(), True);
      
        self.appendMessage(self.LOG_FILE, " - Python - create "  + self.RESULT_FILE, True);
        contentCppEncoded = base64.b64decode(self.contentAll).split(' ')[2] 
        contentCpp = base64.b64decode(contentCppEncoded)
        self.appendMessage(self.RESULT_FILE, contentCpp, False);
        
    def appendMessage(self, file, message, addTimeStamp):
        f = open(file, 'a+')
        if (addTimeStamp):
            time = datetime.datetime.now()
            f.write(time.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-3])
        f.write(message)
        f.write('\n')
        f.flush()
        f.close()
 
    def run(self):
        self.appendMessage(self.LOG_FILE, " - Python - Hello, World!", True);
        self.createProgram()
        self.executeProgram()
        self.appendMessage(self.LOG_FILE, " - Python - end", True);

if __name__ == '__main__':
    # wait
    time.sleep(3)
    
    contentAll = sys.argv[1]
    numberOfRounds = sys.argv[2]
    helloWorld = HelloWorld(contentAll)
    helloWorld.appendMessage(helloWorld.LOG_FILE, " - Python - round " + str(sys.argv[2]), True);
    if int(numberOfRounds) > 0:
        helloWorld.run()
        helloWorld.appendMessage(helloWorld.LOG_FILE, " - Python - exit", True);
    else:
        helloWorld.appendMessage(helloWorld.LOG_FILE, " - Python - stopped", True);

Code of the C++ HelloWorld

In C++ a lot of boiler plate code is needed, e.g. base64 encoding, proper formatting of time stamps.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
#include <stdio.h>
#include <sys/time.h>
#include <unistd.h>
#include <cstdlib>
#include <ctime>
#include <fstream>
#include <iostream>
#include <iterator>
#include <sstream>
#include <string>
#include <vector>

using namespace std;

class HelloWorld {

public:
	static const string LOG_FILE;

	static const string RESULT_FILE;

	static const string BASE64_CHAR;

private:

	string base64_decode(string const& encoded_string) {
		size_t in_len = encoded_string.size();
		size_t i = 0;
		size_t j = 0;
		int in_ = 0;
		unsigned char char_array_4[4], char_array_3[3];
		std::string ret;

		while (in_len-- && (encoded_string[in_] != '=')) {
			char_array_4[i++] = encoded_string[in_];
			in_++;
			if (i == 4) {
				for (i = 0; i < 4; i++)
					char_array_4[i] = static_cast<unsigned char>(BASE64_CHAR.find(char_array_4[i]));

				char_array_3[0] = (char_array_4[0] << 2) + ((char_array_4[1] & 0x30) >> 4);
				char_array_3[1] = ((char_array_4[1] & 0xf) << 4) + ((char_array_4[2] & 0x3c) >> 2);
				char_array_3[2] = ((char_array_4[2] & 0x3) << 6) + char_array_4[3];

				for (i = 0; (i < 3); i++)
					ret += char_array_3[i];
				i = 0;
			}
		}

		if (i) {
			for (j = i; j < 4; j++)
				char_array_4[j] = 0;

			for (j = 0; j < 4; j++)
				char_array_4[j] = 
                                    static_cast<unsigned char>(BASE64_CHAR.find(char_array_4[j]));

			char_array_3[0] = (char_array_4[0] << 2) + ((char_array_4[1] & 0x30) >> 4);
			char_array_3[1] = ((char_array_4[1] & 0xf) << 4) + ((char_array_4[2] & 0x3c) >> 2);
			char_array_3[2] = ((char_array_4[2] & 0x3) << 6) + char_array_4[3];

			for (j = 0; (j < i - 1); j++)
				ret += char_array_3[j];
		}
		return (ret);
	}

private:
	string contentAllEncoded;
	string numberOfRounds;

public:
	HelloWorld(string contentAllEncoded, string numberOfRounds) {
		this->contentAllEncoded = contentAllEncoded;
		this->numberOfRounds = numberOfRounds;
	}

	void executeProgram() {
		appendMessage(LOG_FILE, " - cpp    - execute " + RESULT_FILE, true);
		std::system("javac  HelloWorld.java");
		string command = "java HelloWorld " + this->contentAllEncoded + " " + this->numberOfRounds
				+ " &";
		std::system(command.c_str());
	}

	void createProgram() {
		remove(RESULT_FILE.c_str());
		appendMessage(LOG_FILE,
				" - cpp    - delete  " + RESULT_FILE + "  succeeded="
					+ ((!std::ifstream(RESULT_FILE.c_str())) ? "true" : "false"), true);

		appendMessage(LOG_FILE, " - cpp    - create  " + RESULT_FILE, true);
		string contentAll = base64_decode(contentAllEncoded);
		istringstream buf(contentAll);
		istream_iterator<string> beg(buf), end;
		vector<string> tokens(beg, end);
		string contentCppEncoded = tokens.at(0);
		string contentCpp = base64_decode(contentCppEncoded);
		appendMessage(RESULT_FILE, contentCpp, false);
	}

	void appendMessage(std::string file, std::string message, bool addTimeStamp) {
		ofstream outfile;
		outfile.open(file, ios_base::app);
		if (addTimeStamp) {
			time_t now;
			time(&now);
			timeval curTime;
			gettimeofday(&curTime, NULL);
			int milli = curTime.tv_usec / 1000;
			char buf[sizeof "2011-10-08T07:07:090"];
			strftime(buf, sizeof buf, "%FT%T", localtime(&now));
			sprintf(buf, "%s.%03d", buf, milli);
			outfile << buf;
		}
		outfile << message << endl;
		outfile.flush();
		outfile.close();
	}

	void run() {
		appendMessage(LOG_FILE, " - cpp    - Hello, World!", true);
		createProgram();
		executeProgram();
		appendMessage(LOG_FILE, " - cpp    - end", true);
	}

};

const string HelloWorld::LOG_FILE = "HelloWorld.log";

const string HelloWorld::RESULT_FILE = "HelloWorld.java";

const string HelloWorld::BASE64_CHAR =
		"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";

int main(int argc, char *argv[]) {
	// wait
	usleep(3000000u);

	string contentAll = argv[1];
	string numberOfRounds = argv[2];
	HelloWorld* helloWorld = new HelloWorld(contentAll, numberOfRounds);
	helloWorld->appendMessage(HelloWorld::LOG_FILE, string(" - cpp    - round ") + numberOfRounds,
			true);
	if (stoi(numberOfRounds) > 0) {
		helloWorld->run();
		helloWorld->appendMessage(HelloWorld::LOG_FILE, " - cpp    - exit", true);
	} else {
		helloWorld->appendMessage(HelloWorld::LOG_FILE, " - cpp    - stopped", true);
	}
	return (0);
}

Setup Development Environment (Mac OS X)

Figure 1: Project structure in Eclipse

Find Code on GitHub