Per Spilling's Blog

Software Craftsmanship, DDD, Software Architecture and Agile Methods

Report From QCon London 2012

After wanting to go to the QCon London conference for a long time I finally made it this year.

My favorite presentations

My favorite presentations where the following:

Simple Made Easy, Rich Hickey (slides). This was the keynote on the second day of the conference. Hickey’s presentation is a philosophical talk where he analyses the root cause of complexity when constructing software. He challenges the way software is being built today with object oriented programming languages and complex frameworks. His answer, of course, is to construct software in a functional manner as one does with the language he has designed, Clojure. A very thought provoking presentation. I really need to learn Clojure after this! :-) Btw. he gave this presentation also last year at the Strange Loop conference.

How Github Works, Zack Holman (slides). He talked about how the company culture at Github. This was a very entertaining and good presentation with beautiful slides. Github is a radical company that gives its employees extreme freedom:

  • There are no managers, at least not in the traditional sense. The company founders give some general directions on what they want to achieve, and then the rest is up to the employees.
  • Developers propose their own ideas for what they want to develop, and if enough colleagues like the idea then they go ahead and do it
  • There are no fixed working hours. Employees can work at what hours are best for themselves, like at night if they prefer that.
  • No meetings (well, almost :-))
  • Almost all communication is done via various asynchronous collaboration software (chat and forums). This mode of communication is encouraged in order to allow people to work concentrated and stay “in the zone” without unnecessary disturbances from colleagues.

Very interesting ideas that many companies could learn from, although Github of course is somewhat of a special case as the developers make tools that they use themselves, i.e. they are their own customer. This makes it easier to have such a free and and self-organizing way of working. Also being a small startup makes things a bit special. It will be interesting to see how the Github is doing after a couple of years with strong growth in number of employees.

Breaking the Monolith, Stefan Tilkov (slides). He gave an overview of how large software applications typically evolve over time, and what we can do to avoid ending up with large monolithic applications that are extremely difficult to maintain and change. He gave some good advice such as making clear distinction between strategic, external design decisions, and tactical, more implementation-specific design decisions. The strategic decisions are those that one will have to live with for the whole lifespan of the application. Those should be carefully thought out, and are things like communication protocols, API design, and in general the external borders of the system. The tactical design decisions should be left more open and allow room for flexibility in implementation by the individual teams. He also argued that things like programming language should be an internal implementation detail, and that systems should be designed to be loosely coupled modules that could be implemented in different programming languages depending on the context and the development team.

Other good presentations

Concurrent Programming using the Disruptor, Trisha Gee (slides). Disruptor is a core part of the architecture at the LMAX, a financial exchange in London. She held a lively presentation using lots of handwritten drawings in order to illustrate how the design works. The Disruptor architecture, which is designed by Martin Thompson btw, is based on old software design techniques used for instance in the Tandem NonStop system. The design avoids locking and other concurrency issues, keeping it simple, robust and super fast. Read more about it here.

Lock-free Algorithms for Ultimate Performance, Martin Thompson & Michael Barker (slides). This was an extremely hard-core presentation on how to achieve maximum performance in Java applications. The techniques they presented have been used to create some of the worlds fastest high frequency trading systems, where trades are done in a few microseconds. Key principles are single writer principle and shared nothing designs. In addition they use techniques which they call “Mechanical Sympathy”, which are very low-level techniques for using the memory caches (L1, L2 and L3) in the CPU in the most optimal way. And btw; Martin Thompson is collaborating with Doug Lea, which means that some of these techniques will hopefully be part of standard Java in the future.

Resilient Response In Complex Systems, John Allspaw (slides). This was the keynote on the last day of the conference. He talked about the importance of designing systems that are easy to operate and monitor, qualities that are often given a lower priority when the systems are designed. He also stressed that short time to roll back, or roll out a new version is an important quality when production issues need to be fixed. Another interesting topic he covered was how to build organizations that are good at handling crisis situations, such as when a large site has a serious production problem. He said that we, as a software industry, can learn a lot from the experience gained in this area in other sectors, such as the military. He specifically mentioned aircraft carriers as organizations that are extremely good at handling stressful situations, and also gave several tips regarding good books in this field.

Scala: simplifying development at, Graham Tackley (slides). He talked about the reasons for, and the experiences done, redesigning Guardian’s website from a Java/Spring/Hiberate monolith to a modular Scala/Play2/MongoDB architecture. In many ways they have followed the same principles as outlined by Stefan Tilkov in his talk. Very impressive how they have managed to continually redesign and improve the architecture. Along the way they have experimented with Guice, Python/Django, Scalatra, Scala/Lift, Guice, and more. He also discussed the criticism that Scala has gotten lately with respect to to being to complex and difficult. Their experience was quite contrary; the complexity has been greatly reduced compared to the previous Java-stack, and the code is now much more readable and maintainable. He did say, however, that they value Clarity over Cleverness, and thus avoid “clever” Scala constructs that make the code less easy to understand. The main issue that have with Scala is the SBT build tool. In my opinion this is a schoolbook-example of how to design and evolve complex software systems. Kudos to the team at Guardian! Read more about it here:

Netflix’ Open Source Scalable Java PaaS, Adrian Cockcroft (slides). This presentation was actually not part of the official conference program, but was given at the London Java Community evening event in conjunction with QCon. Adrian Cockcroft is the technical director at Netflix, and displayed an impressive knowledge of the Netflix architecture for a person in such a position. The Netflix architecture is extremely scalable and is 100% cloud-based, deployed on AWS and using Cassandra as the datastore. Being 100% cloud-based is fairly new, and was driven by a growth in usage that was so large that they couldn’t build new data centers fast enough. Between Jan 2010 and Jan 2011 they had 37X growth in usage, growing to more that 20 billion requests per month. They are now the biggest AWS customer. The architecture is very modular and service oriented (no ESB though!) with strong focus on automation, fault tolerance and scalability. They have also open-sourced several of the frameworks they have developed. It seems like that is the trend now with all the cool innovative companies; I like :-) Read more about the Netflix architecture on the Netflix Tech Blog.

Overall impression

I had a great time. Kudos to the organizers for a well-organized conference! My only gripe was that the conference was too big for the venue, The Queen Elizabeth II Conference Centre. Many rooms where much too small for the big crowds, which meant that many people where not able to get in. The building itself is also not very well suited for a conference of this size, as it felt a bit like a labyrinth sometimes, and the rooms are spread out over 3 floors. I hope that the organizers are able to find a better venue next year.