Category Archives: Scala

Early Impressions of Some New Choices

My team and I were recently tasked with a pretty monumental project. We are basically building the services that will power a whole host of extremely high traffic websites that can do everything from paying a bill to streaming video content to an iPad. The complexity of this type of project is very appealing and I’m really happy be a part of something this ambitions.

As part of this endeavor we were given the freedom to choose just about whatever tools and technologies we wanted to build the most scalable, maintainable and bullet-proof system we could. After about a week spikes on different languages, frameworks and tooling we came to the initial conclusion that we would adopt:

  • Git for version control
  • Spring for DI and other stuff (which we since removed, see below)
  • Scala as our primary development language
  • Gradle for our builds
  • MongoDB for persistance

This blog is one of many to report on my impressions of these choices and the tradeoffs they are (or aren’t) causing us to make versus a standard J2EE/Spring Java project that just about everyone on my team has done in the past successfully.

First, Git.

Git. Is. Awesome. Seriously, it is amazing. I’ve been using Git for about two years but, before now, only in very small projects. I thought it was good before, but I’ve really grown to love just how much flexibility and speed it buys me personally not counting how much it buys our team in terms of ease of branching and an easy (now) to understand code history. If you haven’t yet adopted Git as your SCM system, you absolutely should.

Secondly, Spring.
We’ve all used Spring pretty heavily in the past and have a good track record of success with it. However, as we began coding our DI, we came to a very startling conclusion: we probably don’t need it anymore.

Relying on our experience with DI in general and our understanding of what Spring does “under the covers” as it wires objects together, we were able to create a native Scala DI solution that is statically typed, easy to understand (read: no XML or annotations) and, given that it’s simply part of the language, extremely quick to bootstrap. There is no ApplicationContext to wait for or mock for testing. It does its work when the classes are executed (and slightly later if we’re using the lazy modifier which we usually do) and that’s that. While I’m sure we’ll be refining this solution over time, the fact that we own it and it works as well as it does for us makes me suspect that we, as a team, have grown “beyond” the need for Spring. Spring helped us (and me in particular) understand the need for things like DI and AOP and caused me to think of my code in a very decoupled manner. Thanks Spring, but we’ve grown up!

Next, Scala.
I’m not sure what I think of Scala yet. It’s a daily thing. Coming from a lot of Groovy and JavaScript work, I have grown to like the free love devil-may-care type system. Some of the recent code I’ve written in those two languages is the best work I think I’ve ever done. However, I do see the value in a stricter type system and there are many things about Scala I love.

The biggest concern I have is how ridiculously complex Scala can be if you let it. The community has yet to (as far as I can tell) establish a concise set of patterns (something like Effective Java) and many teams (including ours) spend a lot of time trying to figure out what esoteric features to take advantage of and which to ignore. The exciting thing about this is that we are building our own Scala “way” of doing things internally in our team. That’s also the scary part. However, so far, it’s been relatively smooth aside from those times when I think “my god this was so much better in [Groovy|Java]” which are becoming less frequent as I continue to learn the language.

I had high hopes for Gradle when we started the project. We moved from a combination of Gant and Maven and, at first, we were loving the combination of convention and extendability. It really allowed us to customize our build and do some pretty neat stuff with very little effort.

My biggest complaint, which is starting to sour my whole opinion of the tool, is how slow it is. Running gradle tasks in our project on a quad-core Xeon takes 17 seconds! Running a build from scratch takes almost 30 minutes. And, since there isn’t yet great IntelliJ support for it, running a gradle idea (which I do several times a day to refresh the .iml dependencies) takes 4 minutes.

These are all unacceptable timeframes in my opinion. Granted: much of the 30 minutes I mentioned above is spent downloading dependencies (perhaps the majority of it), but the other tasks download nothing and still take far too long.

As gross as it sounds, I’m actually missing Maven. Not necessarily for configuration and dependency management (which were and still are a pain in many cases) or its contrived complexity and “xml hell” but more because it was faster and had first-class IDE support. These things are all missing right now with Gradle and I feel like it might be causing us to lose velocity rather than gain it as the time saved setting up the project does not match the time we all spend waiting for tasks to complete on a daily basis.

We may be doing something wrong and I’m open to doing what it takes to modify our setup, but I feel like what we’ve done is extremely reasonable and a relatively “common” case in terms of project structure and dependencies. To verify that assumption, I ran the same types of commands using another complex project: Gradle itself. I had similar results to those I see in our project. I’m hoping things improve in this regard after a smart colleague of mine attends Gradle training next week.

Finally, MongoDB.
I don’t have much of an opinion on this one as we haven’t integrated to it just yet. I’m interested to see how it compares to CouchDB as that’s where my previous NoSQL experience lies.

Again, I’ll be writing more as we get farther into the project. I feel (mostly) pretty good about the decisions we’ve made and the team I’m on. It’s been, and will continue to be, an awesome ride!