Glad You're Ready. Let's Get Started!

Let us know how we can contact you.

Thank you!

We'll respond shortly.

absolute dates make tests brittle

Yesterday I modified an old app to turn off a feature for records that were more than a certain number of days old. This is an old enough app that the test data is still yaml fixture files, and in the fixtures I see records with attributes like so:

published_at: 2006-08-25 00:00:00 -07:00
created_at:   2006-08-24 00:00:00 -07:00
updated_at:   2006-08-25 00:00:00 -07:00

And of course, the tests code also makes assumptions about absolute dates…

get :show, :year => '2006', :month => '08', :day => '24'
assert_not_nil assigns[:article]

That makes adding tests that need to check for dates relative to the current date very difficult to integrate into the test suite. It also means that over time, your fixture data gets “stale” and you don’t have any data that appears to be recent, which may or may not matter to your application.

One way to deal with this is to mock time in your test code, so that tests always run at the same effective time, so the hardwired absolute dates in fixtures are always relatively the same age. I think that’s a good Plan B, but I prefer Plan A.

Plan A is to create your test data with dates and times relative to when the tests are being run. You can do that in yaml fixtures by embedding ruby in the yaml:

published_at: <%= 2.days.ago.to_s(:db) %>
created_at:   <%= 3.days.ago.to_s(:db) %>
updated_at:   <%= 2.days.ago.to_s(:db) %>

And in your test code, be flexible too:

pub = article.published_at.to_date
get :show, :year => pub.year.to_s, :month => pub.month.to_s, :day => pub.mday.to_s
assert_not_nil assigns[:article]

Keep your test dates relative, and a year from now you’ll thank yourself (or somebody else will).

  1. The problem with your plan A is that it can induce a different type of brittleness: tests that fail intermittently as time passes. This is especially a problem with projects like billing, accounting, and insurance systems, where date boundaries really matter, and things like leap years and varying month length change desired behavior.

    Personally, I’m a big fan of making a clock a first-class citizen, rather than relying on calls to global system primitives. It’s easy enough to do if you start out with it. And as often is true, replacing an overloaded general mechanism with a limited, specific interface has design payoffs down the road.

  2. The TimeCop gem has been my favorite answer to this problem.

    TimeCop.freeze( # Freeze time # Travel through time (when frozen!)

    TimeCop.return # Return to

  3. Josh Susser says:

    @William: Obviously there’s a trade-off when you have conflicting requirements. We have a common pattern here where we use instead of for getting the time, and that makes it very easy to mock in tests. I think Plan B is valid for when you care about things like month boundaries and actual absolute dates (holidays, leap days, etc.). But putting absolute dates in your fixtures with no flexibility at all is just a big bucket of fail.

  4. Mark Wilden says:

    Another area where I’ve been bitten by relative times is when a DST boundary is crossed.

    About fixtures, ’nuff said.

  5. I’m with William. I’ve seen enough tests with relative dates fail on month boundaries, or on leap years, or when run near midnight (in a particular time zone), or when run in a different timezone (including DST related timezone changes), that I always specify the time in my tests. The midnight bugs are the worst, because the automated build fails at night, but then developers can’t reproduce (and they falsely blame the build server). Using absolute vs relative times makes it much easier to then make those edge cases *explicit* in your tests.

    TimeCop is wonderful, and the approach is also good. I personally prefer to pass in or set a clock object rather than use system globals. Of course, this approach might require changes to pre-existing code. If that doesn’t work for you, that’s where TimeCop comes in. :)

    The flip side of using absolute dates is that the original developers may not consider all of these edge cases when they initially write the tests, so if using relative dates causes the build to fail on March 1st, it can identify a real bug. The unfortunate case (in my experience) is that developers who forget or don’t know about testing these date/time related edge-cases often blame the test suite rather than realize they have a bug in their production code.

  6. Josh, seems you can’t quite avoid me these days, huh?

    Yeah, the primary motivation behind Timecop was that I found myself adding conditional arguments for functions that represented ‘now’ or ‘today’ in an effort to make my code testable. It felt very dirty having to change my method signatures to support testing.

    And as such Timecop was born. The true value it brings to the table is that it doesn’t matter who wrote the code and which libraries they are using for date logic (Date, Time, DateTime). Timecop mocks them all and allows you to write your time-sensitive tests without worrying about how the functionality was actually implemented.

    Regarding relative dates, I’m not sure I agree. Here’s a real example I recall in the pre-Timecop days. My app included a class for generating monthly calendars. When using relative dates, it forces you to write code that answers “when is the next february with 29 days” or “when is the next time that DST will change over”. This code is itself difficult to write accurately. By settling on specific dates in the past, I could just look at a calendar one time and pull out a leap year.

    Anyways yeah, I agree that this stuff is all really tricky and the edge cases are easy to overlook for novices.

Post a Comment

Your Information (Name required. Email address will not be displayed with comment.)

* Copy This Password *

* Type Or Paste Password Here *