My current hobby project, Dimdwarf Application Server, will be a scalable high-availability application server and a distributed object database. It lets the application programmer to write single-threaded event-driven code, which the application server will then execute multi-threadedly. The concurrency issues are hidden from the application programmer using STM and DSM. The programming model is the same as what Project Darkstar has (being involved with Darkstar is where I got the idea), but the architecture of the implementation has some differences. As for other similar application servers, there is for example Terracotta, but other than that I don't know similar systems - mostly just distributed caches and databases. My primary motivation for creating Dimdwarf is intellectual challenge, as it will be the most complicated application I have written this far.
Background
In January 2008 I got involved in Project Darkstar, which is an open source application server designed for the needs of MMO games and is developed by Sun Microsystems. I liked its simple programming model, how the objects are automatically persisted and executed transactionally, so that the programmer can concentrate more on the application logic than the concurrency issues. There were some things that I felt could be improved about Darkstar, so I invented transparent references and implemented them in Darkstar (they should be included in the main codebase in near future). There were also some other utilities that I wrote.
Then in June 2008 I got the idea for Dimdwarf and send a mail about it to a couple of other Project Darkstar community members, Emanuel Greisen and Martin Eisengardt, with whom I had been discussing about making development on Darkstar easier. My initial goal was to solve the GPL license and testability issues that Darkstar has: Since Darkstar Server is GPL'd, you can not embed it in a commercial game, for example to make a single-player mode for a multiplayer game, or to distribute the server side of your application without publishing it under GPL. Testing Darkstar applications was hard and you had to use use MockSGS for running unit tests, because Darkstar could not be easily decoupled from the code that uses it. Also debugging Darkstar applications was hard, because you would have to deal with transaction timeouts and multiple threads.
My idea with Dimdwarf was to create a light version of Darkstar Server, one that uses an in-memory database, is single-threaded (at least initially), doesn't use timeouts and has no clustering support (thus making it simple), but you could anyways use a Dimdwarf-to-Darkstar adapter library to run Dimdwarf applications on Darkstar (thus getting the scalability benefits without being infected by GPL, as Dimdwarf uses the BSD license). Even Dimdwarf's name reflects this goal: dim = not smart / synonym for dark, dwarf = small / one kind of a star. Dimdwarf would be light, unintrusive and testing friendly, so that Dimdwarf applications are decoupled from Dimdwarf and you won't need an extensive testing environment and mocking framework to test the applications.
In August 2008 I opened a project page for Dimdwarf at SourceForge and begun writing some code. I was able to reuse the code that I wrote for transparent references, but otherwise I started it from scratch.
Extended Project Goals
Originally I was aiming to keep Dimdwarf as simple as possible and to not make it a scalable high-availability application server. But in January 2009 I read a paper called The End of an Architectural Era and it gave me some ideas about the distributed database design for Darkstar, so I started a thread about it on Darkstar forums. After thinking about it for a couple of days, making a scalable high-availability database did not anymore seem too hard. Extending Dimdwarf to be a high-availability solution started to feel like being within my reach, so I added it to Darkstar's long-term goals.
The high-availability version of Dimdwarf will go under the name Dimdwarf-HA and I've thinking about it passively now for a couple of months. I will first finish the single-node version of Dimdwarf, after which I'll expand it to a clustered multi-node version. The same architecture can be used for both the embedded stand-alone version of Dimdwarf and the clustered Dimdwarf-HA - in fact the new architecture will be much simpler and testable than Dimdwarf's current development version, because it will have less concurrency-aware code.
A following article will discuss Dimdwarf-HA's architecture in more detail.
No comments:
Post a Comment