I've been developing on the JVM this test runner called Jumi, and out of curiosity I've been screencasting its development since day one [1]. One of my listeners made a comment, whose point I understood to be that with functional programming I could have avoided a bunch of problems which I now need to tackle (i.e. thread safety).
Jumi is fundamental software and thus it has some special requirements regarding reliability, compatibility and dependencies. That limits what solutions I can use, and causes extra work for things which would be free in another language (for example episodes 75-102 would not been necessary if I could have used Scala's case classes and pattern matching). But I can survive without those nice-to-haves.
tl;dr
Using leading-edge programming languages can give some nice safety guarantees, but I can cope without them. Sometimes it's more important to use old and proven tools instead of the latest and greatest.
A Craftsman's Programming Language
Java's development started in 1991 [2] and Java became one of the dominant languages some time around 1997-2000 [3]. Compared to that, Scala's design started in 2001, Clojure's design started in 2007 and neither of them even shows up yet in the top 50 of TIOBE index.
In 2001 Pete McBreen wrote in Software Craftsmanship (pages 88-89) that COBOL is a craftsman's language because of its long history, whereas back then Java was too new a language. Nowadays Java is considered to be in the same position as COBOL was back then [4].
That's why even though I prefer writing things in Scala, Clojure or some other modern language, when there are high requirements for stability and long life I prefer a language which has been in mainstream use for over 10 years, has a proved history about its stability, and which can be trusted to be still in use after 10 or even 20 years.
It will take still many years for functional programming languages to enter the mainstream, and then it will take many more years to see how reliable they are. Java has a long history of backward compatibility, whereas Scala has a short history of backward incompatibility (not even minor releases are binary compatible). Lisp has a long history, but it has never entered mainstream, and Clojure is still very new. It will take at least 10 more years to see whether they will succeed.
Referential Transparency and Other Language Features
Some arguments in the original post were that tests can be a maintenance burden and by making the system more simple there is need for less tests. He also implied that higher-level abstractions and referential transparency would solve my problems much simpler.
I'm already trying to make my systems as simple as I can make them, given all the restrictions imposed on them. And I'm already using higher-level abstractions in Jumi - I'm using the actor model for concurrency management, so most of the code is single-threaded. I've been bitten by shared-state concurrency enough many times to have learned to avoid it.
What I don't agree is that referential transparency would solve my problems. For one, the JVM is not referentially transparent. It would be perilous to try to ignore the existence of side-effects on an impure platform, when creating fundamental software which does not have control over the code it executes (i.e. testing frameworks and test code). Even when using the actor model [5], I could not rely on all the single-threaded objects being used only from one thread, so I started writing the thread-safety checker which the commenter objected to.
As a counter example to the claim that referential transparency would cause simplicity, let's consider Scala's testing frameworks. The most popular testing frameworks on Scala (specs, specs2, ScalaTest) are implemented mostly in functional style. Each of their core - the part which controls how the tests are executed - is a couple thousand lines of code (in total with all matchers and test runners they are over 10k lines).
In contrast Specsy is written in imperative-style Scala and its core is only about 300 lines of code (with adapters for JUnit it's in total about 600 lines, because of some impedance mismatch, but after Jumi is ready that superfluous code can be removed). But with those 300 lines it provides more expressiveness, better isolation of side-effects and less bugs than any of those other testing frameworks. Quite soon I'll convert Specsy from Scala into Java, because of compatibility and dependency reasons, which should increase its size by only about 20% because I'm not using much of Scala's features.
The programming languages and paradigms are insignificant factors in achieving simplicity - the way that they are used is much more important.
Amount of Tests
If circumstances require so, I don't mind missing some nice language features, because I can to some extent get the same safety benefits by writing some more tests, and the lack of some syntactic sugar just causes more typing - and typing is not the bottleneck. As is said regarding the debate of dynamic vs. static typing, "the compiler is a unit test." The compiler gives some guarantees of correctness, but even if it doesn't, I can cover the same problem areas with a little bit more tests.
Regarding the number of tests, I agree on what Kent Beck said in an interview: "I don't think people really know what too much really is. -- I've never achieved it [having too many tests], and I've tried." Kent also gave an inspiring anecdote: "The guy I learned the most about testing from was a compiler writer, and he had five lines of test for every line of compiler that he wrote, and he was the most productive guy that I've ever seen."
When in episodes 103-123 of Let's Code Jumi I write a bytecode enhancer which makes sure that an object annotated with @NotThreadSafe is only accessed from one thread during its lifetime (a stronger requirement than no concurrent access, but easier to check), it actually finds one concurrency bug, so my sense of insecurity was warranted. I don't think that those 10 hours were wasted - I've spent much longer times hunting weird bugs - and in the future it lets me worry less that from which thread an object might be called from. I can then better focus on only those few places that need to be thread-safe, even writing a proof of correctness if necessary. Of course my default is to make everything immutable, but sometimes mutability produces simpler systems (simple in design, not easy to implement).
Also the build tests which the commenter was complaining about, the ones that also have tests which operate on just the test data of other tests, are one of the tests which have failed many times throughout this project, almost every time that I add a new dependency to the project. They are some the most valuable tests in reminding me about things that I forget. When I write each of the system's requirements as a test, then I don't anymore need to keep that requirement in my mind at all times, and I can focus on just the problem at hand.
Though "every test is a point of coupling in a system we want as decoupled as possible," I don't think that all coupling is bad. I try to keep my tests as much as possible coupled to the requirements of the system (see Three Styles of Naming Tests), and decoupled from the implementation, to the extent reasonable (fully decoupled code does nothing).
[1] I would have wanted to see something similar from how JUnit was developed, but unfortunately Kent Beck and Erich Gamma did not use a screen recorder in 1997 when they opened their laptops on a plane going to Uppsala.
[2] Java was first known as Oak: The Java History Timeline, The Prehistory of Java, Oak (Wikipedia)
[3] In 1997 JavaOne became the world's largest developer conference and in 2000 Java was 5th in the TIOBE index.
[4] Quote 1: "If I remember correctly, McBreen's book mentions something about COBOL being the craftsman's language, I guess Java would be the modern equivalent."
Quote 2: "Java is the new COBOL"
[5] In this application I wrote the actor library myself because it's a core part of the system, and because of my reliability requirements I want to know exactly how it works. And since I'm writing fundamental software, I need to minimize external dependencies. Why use a 1MB Akka Actors library when I can do just what I need in 20KB?
No comments:
Post a Comment