Close
Glad You're Ready. Let's Get Started!

Let us know how we can contact you.

Thank you!

We'll respond shortly.

LABS
Taking a break from Rspec

On my next new project, I think I’m going back to Test::Unit. I’ve lost patience with Rspec, and it seems like I’m not alone. But I’ve spent more time praising and lobbying for it than some of its other current detractors, so I feel an explanation of my reversal is in order. First, I’ll start off with what I like about Rspec, what got me to spend the time and energy switching to it to begin with.

I’ve always stressed that Rspec brings nothing fundamentally new to the table. For **’s sake, it’s a testing framework. Setup. Teardown. Mocking. There’s not a hell of a lot more to see. And that’s okay. What’s great about Rspec is that it lets you use real strings to label your test fixtures and cases. When I learned about it, I was struggling to name my test cases like sentences.

test_that_foo_does_bar_when_it_has_been_bazzed

Luckily, I’m a Dvorak typist so the underbar is close at hand, but not being able to freely compose descriptions of what I’m testing in a natural way can be very limiting. Writing helps me gather my thoughts, so the act of labelling the test can really help me. I take special pride in well written “it” strings. The describe block strings are also helpful, but not as big a deal to me. The clever assertion hacks are also cute and fun to write. The built in mocking is nice, but I’m not a big mockofascist, so I don’t get too excited about it.

And… that’s basically it. My appreciation for Rspec can be broken down as follows:

  • 70%: String test fixture and case names
  • 20%: .should be_valid etc
  • 10%: Mocking

Now, what sucks about Rspec snuck up on me. It boils down to 2 things:

  1. The framework is too f-ing complicated in its implementation.
  2. The framework is too presumptive about how I wish to organize my tests.

The second is an aesthetic gripe, which I insist is fair game, since the framework’s merits are mainly aesthetic anyways. The first is a much deeper issue. The semantics of a Test::Unit test fixture are straightforward. The test fixture is a class. It contains methods beginning with the word test. A seperate instance of the class is created, in which each test runs. A setup and teardown method run before and after each method invocation.

And that is more or less all I need to know. There’s some stuff I’d like to have, like a global setup / teardown akin to before(:all), but I can what’s there without really understanding anything about the framework’s implementation. It rides on the semantics of Ruby, and I understand Ruby because I use it every day.

Rspec, on the other hand, sends me down a labyrinthine path full of Ruby meta-object-protocol tricks to accomplish even the simplest of tasks. And trust me, I have a pretty solid grasp of the MOP. I love it, use it, and cringe when people refer to it as “magic” (it’s like assembly programmers calling a for loop magic or something). But there’s use and then there’s overuse. I myself can be accused of both. The eval family of methods is great. Therein lies the power to implement DSL’s with their own semantics that can diverge quite dramatically from Ruby. Therein also lies the problem. I like classes. I like inheritance. I like the object oriented model. So if software written in an object oriented language can get away with employing the basic object oriented tools to accomplish its mission, well then it by all means should. I don’t have time to dig around the BehaviorEvalModule, or whatever else I’ve looked at in myriad diversions to get Rspec to do something of medium hardness.

So I guess that’s it. Rspec does the basics really well. Ridiculously beautiful stuff. But venture beyond the limited tracks they’ve laid and you’re in the jungle. So anyway. I’m not ready to write the whole thing off yet, but I am going to revisit Test::Unit for a while, see if I might not give myself what I miss from Rspec atop its simpler implementation. We’ll see if I come back.

Comments
  1. Tim Case says:

    This describes pretty accurately my experience with rspec, and I’ll be doing the same thing on my next project which is to return to test unit and see how that looks now that my eyes have been bdd’d. I’ve found that the pain of mocking actually helped me to redesign my code to have less couple/more cohesion in certain instances, and then make me curse my lungs out in other cases. It may have even pushed me far enough to sell my soul to fixtures (have mercy on me). All in all I’m glad I decided to take the rspec train for a ride, and the jury is not out on which I like better. I’m thinking next round sees me back with test unit, with the addition of a new sidekick called mocha/stubba

  2. evan says:

    Make sure you try Chris Neukirchen’s test/spec before giving up on BDD. It’s lightweight and Test::Unit compatible.

  3. Nathan Sobo says:

    I took a look at it, but it seems like a reimplementation of a lot more than I’d like. I took Test::Unit and welded Rspec like behaviors on top of it that are syntactic sugar for the typical usage. I basically take describes and its and generate test cases and methods with them, correctly formatted and uniqueified of course. There are probably going to be edge cases that suck with this technique, but it’s fully backward compatible with Test::Unit when the fanciness doesn’t work out. It’s going pretty well. I’m testing Treetop, my parsing framework, and need to test some sophisticated metaprogramming behavior. It’s nice to have simpler test framework semantics when dealing with complex semantics of my own.

  4. Chad Woolley says:

    I’d be interested to hear what caused you to venture beyond the limited paths they’ve laid, and what problems you had.

    I agree it’s more complicated under the covers than necessary, and this is the crux of your only non-aesthetic gripe. For example, it would be nice to write specs in the context of a class, not the mysterious anonymous eval’d context that currenly exists for specs (I know Brian would like to improve this, too).

    However, in practice, this really hasn’t affected me – the default syntax and implementation works fine for me in most cases (I can’t remember any problems I’ve had recently), and I don’t find the need to force the framework to do anything else. I can still add cross-describe common behavior by hacking the configuration in my spec helper. For example, I set up a shared describe in the Spec::Runner.configure block in my common spechelper, and then use itshouldbehavelike(“my common describe”) wherever I want that shared behavior. Worked fine when I did it, although it might not scale well.

    Then again, I’m not hardcore like you, Sobo. So, it would probably be very enlightening to see examples of where the complexity has caused you problems.

    Thx,

    — Chad

  5. Alex C (in email) says:

    Coincidentally, I was talking with David Chelimsky about just this gripe at Agile2007 yesterday. He was sympathetic to the complaint and is open to allowing an rspec file to contain a class, which would make it clearer that describe and it just define methods. Rather than put words in his mouth, I’ll let him read your post and see what he thinks…

  6. Brian Takita says:

    I’ve been able to refactor the internals of Rspec to make it simpler. There has already been some good progress, such as getting rid of BehaviourEval. I still have a little more work in simplifying the Example objects, but its very close.

    If anybody is interested, see:
    svn://rubyforge.org/var/svn/rspec/branches/20070804behaviourrefactoring

    The big challenge is writing a DSL. One must be very careful to create a good object model when creating a DSL. I’ve yet to see a library that employs a DSL do this.

    For example Rake also suffers from the internal complexity issue.

    I hope introduce a decent Object Model with the Rspec refactoring, and would love your input.

    I will be writing a blog post about this issue.

  7. Tammer Saleh says:

    These are pretty much the same gripes I had with rSpec (blogged about on giantrobots). I looked at test/spec and others, but eventually decided that I needed to make my own framework (Shoulda). Give it a looksee and let me know what you think.

  8. Alex C says:

    Brian –

    You gotta escape underscores in markdown (with a backslash, like this: _). Here’s a copyable link for your refactoring branch:

    svn://rubyforge.org/var/svn/rspec/branches/20070804_behaviour_refactoring

  9. Alex C says:

    This deserves another blog post, but this gripe — that the semantics are unclear and difficult to extend even for people who are experts in OO — also applies to many constructs in Rails. Why can’t I construct a view and call its render method? Why do controllers and views share instance variables? Why can’t I construct a Routes or an Application and ask _it_ what’s the path for a given controller+action? It would make tests so much easier. Mixins are great but they seem to have been used to the exclusion of other, more appropriate patterns like inheritance, delegation and composition…

  10. Brian Takita says:

    Tammer, the way to extend the context (now behaviour) was to add methods to extend BehaviourEval and not Object.

    I understand that it is very confusing and easy to miss, and hopefully the refactoring will make it easier to grok.

    Your experience highlights missing documentation on extending rspec.

  11. Brian Takita says:

    Also, to set the record straight, I introduced ContextEval pattern as my first patch to rspec, so you could say its on me.

    The motivation was to reuse Rail’s existing testing infrastructure, because it was and still is tightly coupled with Test::Unit. Since there was the tight coupling, Rspec on Rails used to reimplement a bunch of methods that Rails just dumps into Test::Unit.

    The MOP introduced was a first attempt to stay DRY. The infrastructure was put into place because I figured that we would want to inherit from Test::Unit::TestCase in the case of Rails to get the reuse.

    It turns out that composition is more effective and simpler in this case. From this, it should be easier to extend rspec.

  12. Chad Woolley says:

    Good job, Brian. Way to own up AND set things straight. Merge that stuff…

  13. Nathan Sobo says:

    Hey Tammer, looks like what you’ve done is exactly what I did on Friday. But probably a lot more polished. Does it work independently of Rails?

  14. Tammer Saleh says:

    Hey Nathan,

    The basic context/should stuff works fine outside of rails. The other half (ActiveRecord and ActionController specific macros) are of course rails dependent.

    I would have released these as separate plugins, but then the rails helpers would have depended on the shoulda stuff, and that felt wrong. Anyway, the rails specific stuff has been the greatest part of the plugin in terms of time savings.

    rock on,
    Tammer

Post a Comment

Your Information (Name required. Email address will not be displayed with comment.)

* Copy This Password *

* Type Or Paste Password Here *