Close
Glad You're Ready. Let's Get Started!

Let us know how we can contact you.

Thank you!

We'll respond shortly.

LABS
leave your migrations in your Rails engines

If you are using Rails engines to break up a single app into modular pieces, migrations (as they are currently implemented in Rails 3.2.13) become clumsy.

There are three options for migrations within an engine (spoiler: #3 is the best):

1) You can use the your_engine_name:install:migrations rake task, which copies the migrations out of the engine and into the wrapping Rails app where they can be run normally. This works fine if your migrations in your engine never change, but if you’re actively developing your engine you need to run this rake task each time you add a migration.

2) You can put all your migrations in your wrapping Rails app. This works if you’re using your engines as a way to break up your app, but it doesn’t feel right. If your models, views, and controllers all live within the engine (and depend on migrations), shouldn’t your migrations live within the engine as well? If your migrations live in the wrapper Rails app, you actually create a weird upward dependency where the engine is actually dependent on the wrapper app. This is bad.

3) You can monkey patch Rails so all of your engine’s migrations automatically get run in the wrapper Rails app. Everything just works, and migrations live where they should: in the engine. If you’re breaking up your large Rails app into engines, this is the way to go. Here’s how you do it….

Within your Rails engine, there should be a file called engine.rb here’s an example of it for an engine I called EngineWithMigrations:


module EngineWithMigrations
  class Engine < ::Rails::Engine
    isolate_namespace EngineWithMigrations
  end
end

All you need to do is tell Rails to add your engine’s migration directory to its list of places it looks for migrations (note: see the update at the bottom of the post if you are using Rails 4). Like so:


module EngineWithMigrations
  class Engine < ::Rails::Engine
    isolate_namespace EngineWithMigrations

    initializer :append_migrations do |app|
      unless app.root.to_s.match root.to_s
        app.config.paths["db/migrate"] += config.paths["db/migrate"].expanded
      end
    end
  end
end

app.config is the config of your wrapper Rails app, config is the config of your engine. The above line adds the engine's migration directory to the wrapper Rails app's migration directory list. The unless wrapping it is to keep your migrations from running twice in your testing dummy app (which already runs migrations fine). Now when you run rake db:migrate from your wrapper app, your engine's migrations just work!

Rails 4 Update:

In order to get your migrations to work with Rails 4, the initializer needs to change slightly:


module EngineWithMigrations
  class Engine < ::Rails::Engine
    isolate_namespace EngineWithMigrations

    initializer :append_migrations do |app|
      unless app.root.to_s.match root.to_s
        config.paths["db/migrate"].expanded.each do |expanded_path|
          app.config.paths["db/migrate"] << expanded_path
        end
      end
    end
  end
end

Thanks Systho for pointing this out!

Comments
  1. Hi,

    Thanks for this tip. I had to fix the code to this:

    “`
    initializer :append_migrations do |app|
    unless app.root.to_s == root.to_s
    app.config.paths[“db/migrate”] += config.paths[“db/migrate”].expanded
    end
    end
    “`

    because I had a case where the wrapping app was called:

    “`
    wrapping app : ../foobar_customer
    engine app : ../foobar
    “`

    and this found a match from this wrapper app.

  2. Systho says:

    Update for rails 4 : paths are now stored in a special object instead of an array, therefore the intializer should look like :

    initializer :append_migrations do |app|
    unless app.root.to_s.match root.to_s
    config.paths[“db/migrate”].expanded.each do |expanded_path|
    app.config.paths[“db/migrate”] << expanded_path
    end
    end
    end

  3. Ben Smith says:

    @Systho thanks for the Rails 4 tip! I just tried your initializer and it’s spot on. I’ll update the post.

  4. Jez says:

    Is this technique appropriate if the engine is used in more than one application? Creating the engine’s schema in a new application from its migration history seems wrong in the same way it is not a good idea to recreate an application’s database from its migrations.

    Engines released publicly often provide an “install” generator which generates a single installation migration for the application, but it seems in this case that would not work as the migrations would conflict.

    Perhaps this isn’t a big deal if the engine is only to be used privately in a small number of applications, but I’m interested to hear if anyone has any experience of doing this.

  5. Ben Smith says:

    @Jez good question. My gut tells me that if you’re gemifying and distributing your engine, you should use the default rake task to copy the migrations from the engine into the wrapping Rails app. The main reason being the case where you remove the dependency on the engine at some point in the future… this would leave your database in a strange state.

    However if you are using unbuilt engines as a technique to break your Rails app into smaller, more manageable pieces, then I think this technique is much better.

    Even if you are developing an engine to be used by multiple Rails apps, you can start with the migrations within the engine (ie while you are actively developing the engine), then switch to doing the default copy-migrations-into-wrapping-app once your engine has solidified and is ready to be distributed and used in multiple Rails apps.

  6. Brooks says:

    Thanks for the rails 4 update! Works great.

  7. Joey Lorich says:

    This is definitely a convenient technique, but it seems to fail when chaining rake db commands.

    On it’s own “rake db:migrate” will pull in the changes just fine, however if you chain multiple commands like “rake db:drop db:create db:migrate” it does not include migrations from the engine.

    I’m still trying to figure out why specifically it’s happening, but it seems to be that rake db:create doesn’t run any engine initializers and then the chained commands run in the same context so the ‘db/migrate’ paths are never added in.

  8. Mike C says:

    @Joey Lorich,

    I am seeing the same behavior you are for Rails 4 engines. Specifically I was using the technique to handle testing engines that depend on other engines. Does anybody have a reason that if I chain rake commands db:drop db:create db:migrate that only the local db:migrate gets called? I can see from puts statements that at some point in the rake process the append to function is being hit and the paths are correct. If I run db:migrate alone it all works as expected.

    Other than this little snag, this is excellent test use case for teams with multiple engines for a given app.

    Thanks for the great writeup.

    -Mike

  9. Mike C says:

    One more quick comment: If you force the :environment task to be called the problem @Joey Lorich mentions is resolved. There are other implications to what :environment is doing, for our use case this was ok. (high level understanding of :environment http://stackoverflow.com/questions/7044714/whats-the-environment-task-in-rake)

    We created the following rake tasks as a standard part of a gem included in each engine:

    desc “reset the database, drop, create, & migrate”
    task :db_reset => [:environment, “db:drop”, “db:create”, “db:migrate”]

    desc “run any missing migrations and run all specs under test ENV”
    task :engine_spec => [:must_be_test_env, :environment, :spec]

    desc “reset the test database and run all specs”
    task :db_reset_and_test => [:must_be_test_env, :db_reset, :engine_spec]

    desc “check for RAILS_ENV == test”
    task :must_be_test_env do
    raise “PLEASE re-run with RAILS_ENV=test” unless (ENV[‘RAILS_ENV’] == ‘test’ rescue false)
    end

    desc “Run all specs in spec directory (excluding plugin specs)”
    RSpec::Core::RakeTask.new(:spec => ‘db:migrate’)

    Naming is a little weak at the moment because reset is used to mean something different else where, so rename as you see fit.

    -Mike

  10. Mikael Henriksson says:

    For the Rails 4 version I would recommend people do the following instead.

    initializer :append_migrations do |app|
    unless app.root.to_s.match root.to_s
    app.config.paths[“db/migrate”].concat config.paths[“db/migrate”].expanded
    end
    end

    concat is faster!

  11. Damien says:

    We had a problem with db:setup not being able to find engine migrations and so not including them in the schema_migrations table.

    I fixed it by adding setting updating those migration_paths after setting them in the application config:

    if ActiveRecord::Tasks::DatabaseTasks.migrations_paths.nil?
    ActiveRecord::Tasks::DatabaseTasks.migrations_paths = [expanded_path]
    else
    ActiveRecord::Tasks::DatabaseTasks.migrations_paths << expanded_path
    end

    I don't entirely understand how this works, so I hope it doesn't break anything else.

  12. Tim Mertens says:

    @Joey Lorich && @Mike C

    I found that the :environment rake task does not work as expected when run after db:drop. More specifically, when it is run after db:load_config, which is a prerequisite to db:drop.

    So the following works:

    $$$$
    $ bundle exec rake environment db:drop db:create db:migrate
    $$$$

    While this does not:

    $$$$
    $ bundle exec rake db:drop environment db:create db:migrate
    $$$$

    TO RESOLVE, I added the :environment task as a prerequisite to the db:load_config task:

    ##### lib/tasks/db.rake

    Rake::Task[“db:load_config”].enhance [:environment]

    #####

  13. Martin Streicher says:

    For Rails 3.2.19, I had to do the following to make this work:

    initializer :append_migrations do |app|
    unless app.root.to_s.match root
    app.config.paths[“db/migrate”] << File.join(root, 'db/migrate')
    end
    end

    where root is defined in the engine as the top-level directory of the gem.

  14. mentero says:

    I also noticed that tasks like `rake db:rollback` and `rake db:migration:redo` stopped working.

  15. Stuart says:

    I’m still having a problem with matching root names

    I have

    engine – user_manager (web engine) depends on
    engine – user (model engine)

    thus

    /container_app/components/user_manager/spec/dummy is matched by
    /container_app/components/user

    UserManager won’t run the User migrations because of the match. I’ve only just found this so I am either going to have to fix it somehow or rename the User engine to Account

  16. Small code, big trick!

    Amazing tip :)

  17. Kevin Ross says:

    @Damien –

    DatabaseTasks.migration_paths is set before the engine has a chance to alter the paths.

    Code reference:
    https://github.com/rails/rails/blob/master/activerecord/lib/active_record/tasks/database_tasks.rb#L59

    I’m looking to see if we can affect, alter, or reset it at the right time.

  18. Dimitris says:

    Hello,

    I apologize but i am new in ruby.

    – I build an empty project in order to test the below functionality
    – created an engine with name countries_management
    – generate a model inside the engine
    – inside the main application in Gemfile the engine has been added
    (gem ‘countries_management’,path: “vendor/engines/countries_management”)
    – inside the routes of main application the engine mount has been added
    (mount CountriesManagement::Engine, at: “/countries_management”)
    – Also the above code for rails 4 has been added in engine file

    But when i run rake db:migrate on the main project nothing happens why ? what i am doing wrong ?
    Do i need separate database.yml file inside the engine ???

  19. This is very cool. One note to Joey Lorich, though:

    On it’s own “rake db:migrate” will pull in the changes just fine, however if you chain multiple commands like “rake db:drop db:create db:migrate” it does not include migrations from the engine.

    But you should never do “rake db:create db:migrate”. Migrations are only to change an existing database without losing data. If you’ve just created the database, then you should always do “rake db:schema:load” (or “rake db:structure:load”) instead. If you can’t do that, then your migrations are improperly written (and probably contain seed data that you should put in seeds.rb instead).

  20. Ben Smith:

    My gut tells me that if you’re gemifying and distributing your engine, you should use the default rake task to copy the migrations from the engine into the wrapping Rails app. The main reason being the case where you remove the dependency on the engine at some point in the future… this would leave your database in a strange state.

    I don’t think you’re quite right here. Removing the engine would leave the *migrations* in a strange state, but wouldn’t have any issues with the DB.

    Besides, if I’m using an engine, I think I probably don’t want to clutter up my app with its migration files. If I remove the engine, I can copy the files at that time. Till then, YAGNI.

    I do agree with Jez’s concern about running lots of migrations to install an engine instead of loading a schema file, though. I’m not sure where that leaves us practically speaking.

Post a Comment

Your Information (Name required. Email address will not be displayed with comment.)

* Copy This Password *

* Type Or Paste Password Here *