Close
Glad You're Ready. Let's Get Started!

Let us know how we can contact you.

Thank you!

We'll respond shortly.

LABS
Rake, Set, Match!

A few days ago I finally discovered why rake db:migrate:redo consistently angers me nearly as much as watching Paula Dean deep fry the vegetable kingdom. As any devoted connoisseur of the db rake tasks in Rails knows, db:migrate:redo always leaves your schema.rb file in the wrong state. The reason, as mentioned in our standup blog, is that rake will only invoke a given task once in a particular run.

To trivially test this try running a single task twice:

rake db:rollback db:rollback

You’ll find that your database only rolls back one migration. Now, you can set the STEP environment variable when calling db:rollback, but this is, as I said, a trivial example. It gets worse.

Take a look at the implementation of the db:migrate:redo task. The part we’re interested in looks like this:

namespace :migrate do
  task :redo => :environment do
    ...
    Rake::Task["db:rollback"].invoke
    Rake::Task["db:migrate"].invoke
  end
end

That looks fine; db:migrate:redo just verifies that your new migration will properly run down and up without blowing up. Sweet.

But, here’s what db:migrate looks like:

  task :migrate => :environment do
    # Do migratey stuff
    Rake::Task["db:schema:dump"].invoke if ActiveRecord::Base.schema_format == :ruby
  end

And rollback:

  task :rollback => :environment do
    # Do rollbacky stuff
    Rake::Task["db:schema:dump"].invoke if ActiveRecord::Base.schema_format == :ruby
  end

Both db:migrate and db:rollback dump the schema after they run, as they should. If you were to migrate or rollback your database and not dump the schema, then your schema would be in an invalid state. So, of course you can see where this is going, when you run db:migrate:redo the task performs the rollback, dumps the schema, performs the migrate, and then doesn’t dump the schema, because that task has already run. Boom, your schema is one migration behind, db:test:prepare loads the invalid schema into your test database, and all your tests fail (or, worse, pass inappropriately)

Now, I assumed this was a bug in Rake, and so I went on a little investigatory safari through the jungles of the Rake code to find it and kill it. I found the culprit, but invoking each task at most one time is, somewhat surprisingly, the expected behavior; it’s tested and everything. Now I can only wonder why. Why prevent invocation of a task more than once in a given rake run? The code contains unrelated guards against circular task dependencies, so that’s not it. Is this an example of overly-speculative defensive coding, or is there an actual use case for which this behavior is desirable? I’d like to hear from anyone who has written tasks that depend on this behavior, as well as anyone who (like me) considers this behavior unexpected and has run into problems because of it.

Assuming no one steps forward with a compelling reason that Rake should behave this way, I’d suggest that this be changed. I could see the value of it (perhaps as a performance optimization?) if rake tasks were guaranteed to not change the state of anything they operate on, or even were guaranteed to be idempotent; but neither is the case. This behavior severely limits the composability of tasks, since a task writer has to know which atomic tasks have run, and avoid any task that might try to run them again.

In the meantime, Rake provides a way to explicitly re-enable tasks that have run once, but it doesn’t seem to work. The db:schema:dump definition looks like this:

namespace :schema do
  task :dump => :environment do
    # Do dumpy stuff
    Rake::Task["db:schema:dump"].reenable
  end
end

That #reenable call is meant to tell the task “hey, task, you can run again.” I tried calling #reenable on the db:schema:dump task inside the db:migrate and db:rollback tasks as well, but without any luck.

Fellow Pivot David Stevenson would likely put it this way: Khaaaaaaaaaaaaaaann!

Comments
  1. Luke Bayes says:

    Hey Adam,

    Thanks for the post!

    I’ve been using Rake for a few years now to compile, run and debug complex ActionScript and Flex projects. The compiler is unusually slow, and some tasks involve preprocessing hundreds of files. For me, it has proven to be a benefit that Rake only executes a particular task once in an execution cycle.

    With that said, I do agree that (in spite of the fact that it doesn’t seem to be a word) the ‘reenable’ method (or some such feature) should be available and it should work as expected.

  2. James Adam says:

    Rake works this way because it was originally designed to be a replacement for Make. So, when you are are saying

    rake :my_task => :another_task

    you are actually saying”‘before `my_task` runs, ensure that `another_task` has been run”, which is subtly different from “before `my_task` runs, run `another_task`”. It’s the definition of a prerequisite.

    Typically, in a `make` situation, this would be something like “before you link the binary, ensure that any changed source files have been compiled”. If all the compiled files are already up-to-date, you don’t need to run the prerequisite task, saving a bunch of time.

    The problem here is really that Rake is being used in a way that it wasn’t designed for, but for the most part this behaviour suits us well (we only ever want to load the environment once, for example). To change this characteristic of Rake would be to completely change it’s nature as a tool.

    I completely sympathise with your frustration though – after months of minor annoyance, I finally got around to writing a patch for this, only to discover that it’s already fixed in the master branch of Rails.

  3. James Adam says:

    FYI, here’s the [commit](http://github.com/rails/rails/commit/ba146a84d0ed8a886fdc6b6794ce99a9d37c0190), and the [ticket](https://rails.lighthouseapp.com/projects/8994/tickets/1412-dbmigrateredo-does-not-dump-the-schema-after-migrating-back-up), which looks to be along the same lines are your solution. If it definitely doesn’t work for you, that’s the place to add feedback.

  4. James Adam says:

    Last comment, I promise. Here’s my test Rakefile:

    task :redo => [:rollback, :migrate]

    task :rollback do
    puts “rollback”
    Rake::Task[“dump”].invoke
    end

    task :migrate do
    puts “migrate”
    Rake::Task[“dump”].invoke
    end

    task :dump do
    puts “dumping”
    Rake::Task[“dump”].reenable
    end

    and here’s the output of `rake –trace`

    $ rake –trace
    (in /Users/james/Code/experiments)
    ** Invoke default (first_time)
    ** Invoke rollback (first_time)
    ** Execute rollback
    rollback
    ** Invoke dump (first_time)
    ** Execute dump
    dumping
    ** Invoke migrate (first_time)
    ** Execute migrate
    migrate
    ** Invoke dump (first_time)
    ** Execute dump
    dumping
    ** Execute default

    As you can see, the `dump` task is successfully invoked the second time (you can see that Rake thinks that it’s never been invoked). So, `reenable` seems to work, at least in this simple situation.

  5. Jeremy says:

    What about calling .execute instead of .invoke on the rake tasks? Then they get run every time.

  6. Adam Milligan says:

    James, you provided the little mental nudge I needed in order to make this fit in my brain. It’s been a long time since I used make with any frequency, and I’ll admit I always avoided it as much as possible.

    Based on your example, and the fact that the Rails patch you referenced quite clearly doesn’t work, I wonder if the problem has to do with namespacing. I’ll have to play with that.

  7. Will Bryant says:

    What version of Rake are you using?

  8. Alex Chaffee says:

    James is right. Check out Yehuda Katz’ project called Thor: “rake and sake needed to be replaced for scripts, not as a replacement for make”. Basically it’s an interface for scripts, and most modern builds are better modeled as scripts, not as dependency trees.

    http://yehudakatz.com/2008/05/12/by-thors-hammer/

    I haven’t used it much yet so I don’t know if it supports invoking the same task multiple times, but, you know, if it doesn’t, it probably should…

    Steve C raves, “It’s like Erector for Rake!”

  9. Adam Milligan says:

    I’m happy to think about switching to Thor, but I’m not, so far, able to think about it for the entire Rails community. For better or for worse the current state of affairs is that Rails uses Rake.

    Thor also has to overcome its potentially insurmountable lack of alliteration with Rails.

Post a Comment

Your Information (Name required. Email address will not be displayed with comment.)

* Copy This Password *

* Type Or Paste Password Here *