Glad You're Ready. Let's Get Started!

Let us know how we can contact you.

Thank you!

We'll respond shortly.

Ben Moss

Posts By

Means, Ends, Mocks, Stubs

When I was first learning Test Driven Development, I recall being pretty confused by the distinction between mocks and stubs. I remember reading explanations that were basically saying that stubs allow you to override the way an object handles a message, while mocks are basically the same thing but which also set an expectation that the overridden message is actually received in the course of the test.

While this made sense on a basic level, I struggled with understanding why you would use one versus the other. Mostly I failed to understand why you would use a stub when mocks can act as this "super stub" that not only gives you control over the executing code, but also act as verification that things are working as you're expecting them to.

I can't recall what I read, heard, or saw that finally made this make sense for me, but I wanted to try and explain my thinking as it stands now for others who may be struggling with the distinction.

One metaphor that occurred to me today is of the difference between means and ends. If we step away from worrying about code for a minute and think in terms of normal human interactions, this difference is pretty easy to explain.

When I am at the bar, and I say to the bartender "I'll have a Manhattan," the purpose of my statement is to get the bartender to make me a Manhattan. So, simple enough, we would say that the ends of my statement are to request that to the bartender.

What does that leave for the means? Well, the fact that I used my vocal chords instead of a sign language would be one, or choice of words that I used to express myself. These facts are incidental to me getting my drink, they belong to the context (belonging to a species and culture that communicates vocally, having a certain vocabulary and mannerisms in a certain language) I find myself in, but we can draw a distinction between my desire for a drink and the things I have to do to get one. Put me in a jungle in Mozambique and you'd probably observe quite a different range of behaviors in order for me to accomplish the same goals.

What does any of this have to do with mocking and stubbing, you may be wondering? Well, if we were to model the way I behave in this interaction in RSpec, we might come up with something like this:

it "can order a drink from a bartender" do
  speech = MeansOfCommunication[:speech]
  manhattan = Cocktails[:manhattan]

  ben =
  ben.desires << manhattan
  ben.primary_means_of_communication = speech

  bartender =
  speech.stub(:phrase_search).with(bartender, manhattan).
    and_return("I'll have a Manhattan")

  bartender.should_receive(:order).with("I'll have a Manhattan")


A slightly complicated example, but I wanted to illustrate how the various incidental concerns we identified above are translated as distinct from the essential ones. The essential part of the interaction, that an order was placed expressing my desire, is set up with a mock expectation. The merely incidental part, the means by which I expressed my desire to the bartender, is represented with the stub on the speech object. We don't really care in this situation how I was able to translate my desire through the medium of speech, what we care about is that expressed my desire to the bartender.

Another way I like to think of the difference between mocks and stubs is something called the Command-Query separation. The rule of thumb is that you should mock commands, while stubbing queries. If we think of this in relation to the example above, it maps directly to our distinction between means and ends. When we think of how we're going to order the Manhattan, we query our brain, thinking to ourselves "what do I have to say to get this jerk to make me a decent Manhattan?", and we come up with the right phrase. We use that phrase to make our command to the bartender, which is the essential part of the interaction.

We can understand the prohibition against mocking queries by trying to imagine what it would mean if we wanted to test how we interact with our speech center. Just thinking of phrases all day without saying them to anyone would be purposeless, precisely because it lacks any outward goal. Once we attach a goal to the querying, like coming up with our drink order, we end up implicitly testing our speech abilities through the ends that they are used for.

Debugging Travis builds

(Due credit to Trung Lê's article on which all this is based)

We recently moved our project's CI from a TeamCity server onto Travis CI's new private CI-as-a-service program. We compared several other hosted CI services and found Travis to be the easiest to use, and with the help of Trung's article also the easiest to debug.

Travis provides all their build worker images as Vagrant boxes available for download. I'd never worked with Vagrant before, but getting it set up is pretty simple. Follow the guide on Vagrant's page and download the latest version of Vagrant. I've found that the boxes that Travis provide right now don't work with VirtualBox 4.2, and so I'd recommend installing VirtualBox with 4.1 from their 'older builds' page.

Once you're done installing both, you'll want to install the worker box. The Ruby box is installable via

vagrant box add travis-ruby

This will download and install the VM, which will probably take a few minutes. After that's complete, initialize the box with

vagrant init travis-ruby

which will create a Vagrantfile for you where you can configure various settings for how Vagrant hosts the VM on your machine. I've found it necessary to add

config.ssh.username = "travis"

to get SSH to work properly. After that,

vagrant up
vagrant ssh

will connect you to the box, and verify that things are working properly. If you are prompted for a password upon sshing, it should be travis.

Now you'll want to either scp your Github SSH key or create a new one and clone your project down to the box. After you have your project, all that remains is to get Travis running.

I haven't yet figured out how Travis starts from .travis.yml files, so for now we have just created a shell script in which we specify all our Travis setup tasks, put it as the 'script' key in our .travis.yml, and then just run it directly on our local box. You can find more info about doing this on the Travis docs page. Aside from working around the .travis.yml, we haven't seen any other gotchas in simulating the Travis worker process.

Once you've successfully gotten your build to run inside the Vagrant box, you can follow the instructions on Vagrant's site to repackage your customized Travis worker and share it with the rest of your team.

Running JSHint from within Jasmine

I often find myself wasting a lot of time debugging a mysteriously failing Jasmine spec, only to find the root cause being some missing semicolon, the accidental use of double-equals equality or some similar Javascript eccentricity hiding inside my code. On my current project we had been using JSHint through the jslint_on_rails gem to lint our Javascript as part of our test suite, but the unfortunate part of that is that it's outside of our normal TDD cycle, functioning more like a style enforcer than something that can actually help you find bugs.

To help with getting more instant feedback, I wanted to see if we could get JSHint to run against our code from within the Jasmine test suite itself. I came across this blog post from Brandon Keepers describing how he does exactly that.

One gotcha I ran into was that I had a number of global objects defined in my specs both by Jasmine and several libraries we were using. JSHint defines some options for whitelisting common globals in its docs under "Environments", but does a poor job of explaining how to whitelist additional variables. I discovered that by passing an object literal of globals as a third argument to the JSHINT function, I could exempt these from definition warnings.

A partial list of ones I've so far found handy:

var globals = {
   _: false,
   _V_: false,
   afterEach: false,
   beforeEach: false,
   confirm: false,
   context: false,
   describe: false,
   expect: false,
   it: false,
   jasmine: false,
   JSHINT: false,
   mostRecentAjaxRequest: false,
   qq: false,
   runs: false,
   spyOn: false,
   spyOnEvent: false,
   waitsFor: false,
   xdescribe: false