Close
Glad You're Ready. Let's Get Started!

Let us know how we can contact you.

Thank you!

We'll respond shortly.

Brian Takita

Posts By

LABS
RR 0.6.0 Released

I'm pleased to announce the 0.6.0 version of RR. The changes include:

  • Declaring Double subject objects without having to pass it in via the mock!, stub!, dont_allow!, instance_of!, and proxy! methods
  • Revised Double chaining API
  • satisfy matcher
  • hash_including matcher

Declaring Double Subjects (The bang methods)

In previous versions of RR, you always needed to pass in the subject of the double. For example:

subject = Object.new

mock(subject).does_something {:and_returns_me}

subject.does_something # :and_returns_me

Now you can have RR automatically create the subject object for you by using the ! method:

subject = mock!.does_something {:and_returns_me}.subject

subject.does_something # :and_returns_me

Now the bang methods by themselves don't really add a whole lot, but when used in the context of Double chaining, they become a powerful addition.

Double Chaining

Nick Kallen presented the use case for Double chaining and contributed a patch for the 0.5.0 release of RR. It has proved useful and is now more fully incorporated into RR. Now you can pass in your subject or use the subject provided by RR by using the ! method. Here are some examples of Double Chaining:

mock(subject).first(1) {mock(Object.new).second(2) {mock(Object.new).third(3) {4}}}

subject.first(1).second(2).third(3) # 4



mock(subject).first(1) {mock!.second(2) {mock!.third(3) {4}}}

subject.first(1).second(2).third(3) # 4



mock(subject).first(1).mock!.second(2).mock!.third(3) {4}

subject.first(1).second(2).third(3) # 4

Of course you have access to the proxy facilities:

mock.proxy(User).find('1').mock.proxy!.children.mock.proxy!.find_all_by_group_id(10)

User.find('1').children.find_all_by_group_id(10) # Makes verifications pass and returns the actual children

You can also do branched Double chaining:

mock(subject).first do

  mock! do |expect|

    expect.branch1.mock!.branch11 {11} # or expect.branch1 {mock!.branch11 {11}}

    expect.branch2.mock!.branch22 {22} # or expect.branch2 {mock!.branch22 {22}}

  end

end

o = subject.first

o.branch1.branch11 # 11

o.branch2.branch22 # 22

Satisfy Matcher

Matthew O'Conner submitted a patch that added the satisfy matcher. This adds the ability to add arbitrary argument expectation matchers.

mock(object).foobar(satisfy {|arg| arg.length == 2})

object.foobar("xy")

Hash Including Matcher

Matthew O'Conner also submitted a patch that added the hash_including matcher. This adds a convenient way to assert that the passed-in hash includes certain key/value pairs.

mock(object).foobar(hash_including(:red => "#FF0000", :blue => "#0000FF"))

object.foobar({:red => "#FF0000", :blue => "#0000FF", :green => "#00FF00"})

Mailing list

RR has a mailing lists at:

Also, RR's rubyforge page is at http://rubyforge.org/projects/double-ruby and of course the github page is at http://github.com/btakita/rr.

Yes, and there is more to come

There are many interesting ideas floating around. Joseph Wilk has been playing around with adding Spies into RR. I'm also thinking about adding Double validation scoping into RR. Also, I'm impressed by Mocha's warning of unused stubs. Josh Susser also proposed having a mode where a warning would occur if a mocked method is not implemented on the subject being mocked.

If you have any feature requests, please send an email to the mailing list or add it to the rubyforge tracker.

LABS
Build Your Own Rails Plugin Platform with Desert

While it is easy to include plugins in your Rails projects, it isn't easy to extend and customize the plugin for your own application. Desert solves that limitation/complication by making it just as easy to extend or modify a plugin class as it would be with any other class. In this post we will go over how Desert provides an easy way to manage and extend your plugins.

At Pivotal, we offer an integrated platform of Rails plugins named Socialitis.

Socialitis is an internal project that grew out of the observation that many of our start-up clients needed to build the same non-differentiating features; user management, friends/contacts, activity feeds, on-site messaging, etc.

The Socialitis platform is broken up into number of plugins that extend the Rails app in specific ways. These plugins may have dependencies on other plugins.

One of the major design goals of Socialitis is easy, drop-in, integration into existing Rails apps. This means using convention over configuration and removing as much integration responsibility from the user of the plugin as possible.

Another design goal was to provide sensible defaults and make each plugin easy to customize for your app.

We used Desert to achieve these goals, and so can you for your own platform.

The major features that Desert provides are:

  • Defining a Rail's like directory structure into your plugin (models, views, controllers, helpers)
  • Plugin dependencies
  • Seamless overriding of classes and modules defined by parent plugins
  • Plugin migrations
  • Plugin routing

Desert provides a similar feature set to the Radient plugin system and the now defunct Appable Plugins framework.

For a simple example, lets say you have two plugins, name User and Messaging. The User plugin provides basic authentication and login features, and the Messaging plugin allows Users to send Messages to each other. The Message plugin depends on the User plugin.

The directory structure of the full Rails app looks like:


  |-- app

  |   |-- controllers

  |   |   |-- application.rb

  |   |   `-- blogs_controller.rb

  |   |-- helpers

  |   |   |-- application_helper.rb

  |   |   `-- blogs_helper.rb

  |   |-- models

  |   |   `-- user.rb

  |   `-- views

  |       |-- blogs

  |       |-- layouts

  |       |   `-- users.html.erb

  |       `-- users

  |           |-- index.html.erb

  |           `-- show.html.erb

  |-- db

  |   `-- migrate

  |       `-- 001_migrate_users_to_001.rb

  |-- lib

  |   `-- current_user.rb

  |-- spec

  |   |-- controllers

  |   |   `-- blogs_controller_spec.rb

  |   |-- fixtures

  |   |-- models

  |   |-- spec_helper.rb

  |   `-- views

  |       `-- blogs

  `-- vendor

      `-- plugins

          `-- user

              |-- app

              |   |-- controllers

              |   |   |-- logins_controller.rb

              |   |   `-- users_controller.rb

              |   |-- helpers

              |   |   |-- logins_helper.rb

              |   |   `-- users_helper.rb

              |   |-- models

              |   |   |-- login.rb

              |   |   `-- user.rb

              |   `-- views

              |       |-- logins

              |       |   |-- edit.html.erb

              |       |   |-- index.html.erb

              |       |   |-- new.html.erb

              |       |   `-- show.html.erb

              |       `-- users

              |           |-- edit.html.erb

              |           |-- index.html.erb

              |           |-- new.html.erb

              |           `-- show.html.erb

              |-- config

              |   `-- routes.rb

              |-- db

              |   `-- migrate

              |       `-- 001_create_users.rb

              |-- init.rb

              |-- lib

              |   `-- current_user.rb

              |-- spec

              |   |-- controllers

              |   |   `-- user_controller_spec.rb

              |   |-- fixtures

              |   |   `-- users.yml

              |   |-- models

              |   |   `-- user.rb

              |   |-- spec_helper.rb

              |   `-- views

              |       `-- users

              `-- tasks

          `-- message

              |-- app

              |   |-- controllers

              |   |   `-- message_controller.rb

              |   |-- helpers

              |   |   |-- message_helper.rb

              |   |   `-- user_helper.rb

              |   |-- models

              |   |   |-- message.rb

              |   |   `-- user.rb

              |   `-- views

              |       `-- messages

              |           |-- edit.html.erb

              |           |-- index.html.erb

              |           |-- new.html.erb

              |           `-- show.html.erb

              |-- config

              |   `-- routes.rb

              |-- db

              |   `-- migrate

              |       `-- 001_create_messages.rb

              |-- init.rb

              |-- spec

              |   |-- controllers

              |   |   |-- message_controller_spec.rb

              |   |   `-- user_controller_spec.rb

              |   |-- fixtures

              |   |   |-- messages.yml

              |   |   `-- users.yml

              |   |-- models

              |   |   |-- message_spec.rb

              |   |   `-- user_spec.rb

              |   |-- spec_helper.rb

              |   `-- views

              |       `-- messages

              `-- tasks



The User plugin introduces the various User and Login Rails objects. The Message plugin introduces its respective Message objects. Notice that the Message plugin also reopens some of the User objects to insert functionality.

For example, vendor/plugins/users/app/models/user.rb looks something like:

class User < ActiveRecord::Base

  has_many :logins

end

The Message plugin would then reopen User in vendor/plugins/message/app/models/user.rb:

 class User < ActiveRecord::Base

   has_many :messages_received

   has_many :messages_sent

 end

Meanwhile, the main application can also reopen User in app/models/user.rb

 class User < ActiveRecord::Base

   def custom_app_method

     # custom app logic #

   end

 end

Desert allows you to utilize Ruby's ability to repoen classes to layer on functionality in your plugins and application. At Pivotal, we have had success in sharing code across multiple client applications using this technique.

Another thing to note is normally the Message plugin would be loaded before the User plugin. Desert allows you to create plugin dependencies. So in vendor/plugins/message/init.rb:

require_plugin 'user'

require_plugin 'will_paginate'

This means you no longer need to define plugin load order inside of environment.rb. Your plugins can take care of that. Desert works with practically all plugins. That means you can have a plugin dependency on any existing Rails plugin.

To see more examples & documentation, take a look at the Desert project at http://github.com/pivotal/desert.

LABS
Introducing RR

I'm pleased to introduce a new Test Double (or mock) framework named RR, which is short for Double Ruby.

Why a Double framework and not a Mock framework?

A mock is a type of test double. Since RR supports mocks, stubs, and proxies, it makes sense to refer to RR as a double framework. The proxy is a new usage pattern that I will introduce later in this article, and in more detail in future articles.

Unfortunately, the terminology over doubles has been contradictory depending on the framework. RR's terminology tries to be as faithful as possible to Gerald Meszaros' definition of test doubles. You can read more about test doubles in XUnit Test Patterns and Martin Fowler's article, Mocks aren't Stubs. Regretfully, this does mean that RR will have slightly different terminology than other double frameworks.

How does RR compare to other Mock frameworks?

Most double frameworks focus mainly on mocks (hence the categorization "mock framework"). RR's focus is on enabling more double test patterns in a terse and readable syntax.

RR also does not have dedicated mock objects. It primarily uses the technique called 'double injection'. Names that other frameworks use are 'stub injection', 'mock object injection', 'partial mocking', or 'stubbing'. The term I'll use for this is a double injection, since one or many doubles are being injected into an object's method.

I'll use trivial Rails examples to highlight the syntactical differences between RR, Mocha, Rspec's mocks, and Flexmock. They may or may not be appropriate situations for mocks. The right situations for mocks is an entirely different discussion.

If there is better way to do any of the examples, please post a comment and I will gladly replace it.

Mocks

Here are the ways to mock the User.find method. The expectation is the User class object will receive a call to #find with the argument '99' once and will return the object represented by the variable user.

RR
mock(User).find('99') { user }

Mocha
User.expects(:find).with('99').returns(user)

spec/mocks
User.should_receive(:find).with('99').and_return(user)

Flexmock
flexstub(User).should_receive(:find).with('99').and_return(user).once

Stubs

Here are the ways to stub the User.find method. When the User class object receives a call to find with the argument '99' it will return user1. When User receives find with any other arg, it returns user2.

RR
stub(User).find('99') { user1 }

stub(User).find { user2 }

Mocha
User.stubs(:find).with(anything).returns(2)

User.stubs(:find).with('99').returns(1)

spec/mocks
users = {

  '99' => user1,

  'default' => user2

}

User.stub!(:find).and_return do |id|

  users[id] || users['default']

end

Flexmock
users = {

  '99' => user1,

  'default' => user2

}

flexstub(User).should_receive(:find).and_return do |id|

  users[id] || users['default']

end

Proxy

A proxy used with a mock or stub causes the real method to be called. Expectations can be placed on the invocation and the return value can be intercepted. The main rationales are test clarity and you can ensure that the methods are being called correctly, even after you refactor your code. I will delve more into proxies and their usage patterns in my next article.

Mock Proxy

The following examples set an expectation that User.find('99') will be called once. The actual user is returned.

RR
mock.proxy(User).find('99')

Mocha

You cannot implement this in Mocha. You can do an approximation in this situation however. This technique is not always the solution you need, though.

user = User.find('99')

User.expects(:find).with('99').returns(user)

spec/mocks
find_method = User.method(:find)

User.should_receive(:find).with('99').and_return(&find_method)

Flexmock
find_method = User.method(:find)

User.should_receive(:find).with('99').and_return(&find_method)

Stub Proxy

The following examples intercept the return value of User.find('99') and stub out valid? to return false.

RR
stub.proxy(User).find('99') do |user|

  stub(user).valid? {false}

  user

end

Mocha

Again, this is an approximation, since you cannot use proxies in Mocha.

user = User.find('99')

user.stubs(:valid?).returns(false)

User.stubs(:find).with('99').returns(user)

spec/mocks
find_method = User.method(:find)

User.stub!(:find).with('99').and_return do |id|

  user = find_method.call(id)

  user.stub!(:valid?).and_return(false)

  user

end

Flexmock
find_method = User.method(:find)

flexstub(User).should_receive(:find).with('99').and_return do |id|

  user = find_method.call(id)

  flexstub(user).should_receive(:valid?).and_return(false)

  user

end

instance_of

instance_of is method sugar than allows you to mock or stub instances of a particular class. The following examples mock instances of User to expect valid? with no arguments to be called once and return false.

RR
mock.instance_of(User).valid? {false}

Mocha
User.any_instance.expects(:valid?).returns(false)

spec/mocks
new_method = User.method(:new)

User.stub!(:new).and_return do |*args|

  user = new_method.call(*args)

  user.should_receive(:valid?).and_return(false)

  user

end

Flexmock
new_method = User.method(:new)

flexstub(User).should_receive(:new).and_return do |*args|

  user = new_method.call(*args)

  flexmock(user).should_receive(:valid?).and_return(false)

  user

end

More to come

This concludes the introduction to RR. RR enables some techniques, like proxying, that will make your tests clearer and less brittle. In the next article I will describe into patterns and techniques that will make mocks a more feasible tool for more situations.

LABS
Cacheable Flash 0.1.4 — Test Helpers

I just released Cacheable Flash 0.1.4. This version includes test helpers so you can easily test your cache messages. It works by allowing you to make assertions on the flash cookie.

Here is a test/unit example:


  require "cacheable_flash/test_helpers"



  class TestController < ActionController::Base

    def index

      flash["notice"] = "In index"

    end

  end



  class ControllerTest < Test::Unit::TestCase

    include CacheableFlash::TestHelpers



    def setup

      @controller = TestController.new

      @request = ActionController::TestRequest.new

      @response = ActionController::TestResponse.new

    end



    def test_cacheable_flash_action

      get :index

      asset_equal "In index", flash_cookie["notice"]

    end

  end

Here is a rspec example:


  require "cacheable_flash/test_helpers"



  class TestController < ActionController::Base

    def index

      flash["notice"] = "In index"

    end

  end



  describe TestController, "#index" do

    include CacheableFlash::TestHelpers



    it "writes to the flash cookie" do

      get :index

      flash_cookie["notice"].should == "In index"

    end

  end

You can install Cacheable Flash by running:

ruby script/plugin install svn://rubyforge.org/var/svn/pivotalrb/cacheable_flash/trunk

See the Cacheable Flash blog post, Show Flash Messages on Cached Pages, and the README for more information.

LABS
Cacheable Flash

Page caching is an easy way to get massive performance and scalability increases with little up front effort. Of course, when page caching, a number of design changes are necessary. For example, server side session data cannot be used to render data on cached pages.

Rails provides the flash hash to easily render alert messages and errors. In the controller you can write to the flash hash:

flash[:error] = "You cannot go there"

# or

flash[:notice] = "Welcome to eternity"

and render the flash hash in the view. Typically the rendering happens on a layout:

<div id="error_div_id" class="flash flash_error"><%= flash[:error] %></div>

<div id="notice_div_id" class="flash flash_notice"><%= flash[:notice] %></div>

There are some strange quirks with using flash such as needing to use FlashHash#now when rendering the response without redirecting.

Everything works great until you need to page cache the landing page.

For example, lets say you page cache your home page. After logging in, you are redirected to the home page with the flash notice "Logged in successfully". When page caching, this solution does not work because the request is responded to by the Web Server (i.e. Apache) and does not reach the Rails App server (i.e. Mongrel).

This means the view does not get a chance to render the flash error and notice.

There are a couple of solutions to this problem.

  • Do an AJAX request back to the server to render the flash error and notice (i.e. using RJS)
  • Send the flash error and notice from the server with cookies and render it during page load

Introducing Cacheable Flash

To solve the problem using cookies on Peer to Patent, we wrote the Cacheable Flash plugin. The plugin allows you to set the flash hash as normal on the controller. It handles converting the flash hash into cookies. All you need to do is include the CacheableFlash module into your controller.

class ApplicationController < ActionController::Base

  include CacheableFlash

  # ...

end

On the view side, you will need to add some javascript.

<div id="error_div_id" class="flash flash_error"></div>

<div id="notice_div_id" class="flash flash_notice"></div>

<script type="text/javascript">

  Flash.transferFromCookies();

  Flash.writeDataTo('error', $('error_div_id'));

  Flash.writeDataTo('notice', $('notice_div_id'));

</script>

The Flash.transferFromCookies method:

  • Grabs the flash data from the cookies and saves it to Flash.data
  • Deserializes the flash hash from JSON to Javascript
  • Erases the flash data from the cookies

The Flash.writeDataTo method:

  • Writes data from the passed in key in Flash.data to the passed in element or element id

Here is how the life cycle of the login works:

  1. The user enters the correct login information
  2. Rails handles the web request. In the Login controller, flash[:notice] is written to

      if current_user
    
        flash[:notice] = "Welcome to Eternity"
    
      end
    
    
  3. An after filter serializes contents of the Flash Hash as JSON into cookies

  4. The after filter clears the flash hash
  5. The cached page is rendered
  6. The client side receives and clears the flash cookie data
  7. The client side javascript renders the flash messages

There is also a side benefit -- you don't have to use FlashHash#now because storing the flash in cookies to be rendered and erased by the client makes this unnecessary.

The Cacheable Flash plugin is on the Pivotal RB project page (http://rubyforge.org/projects/pivotalrb). You can install it by running:

ruby script/plugin install svn://rubyforge.org/var/svn/pivotalrb/cacheable_flash/trunk

It will copy flash.js, cookie.js, and json.js if you do not already have these files.

Happy Page Caching!!

Thanks to Josh Susser for pairing with me on this.

LABS
Sake for Gems Downloads List

I have a few gems on Rubyforge and I want to track how many of them were downloaded. I found Firefox's search tools lacking to find my gem rr.

To fix this issue, I made a sake task, named gems:downloads:list, that prints the gem downloads in text.

The source is on caboo.se.

You can install it by using:


sudo gem install sake

sake -i http://pastie.caboo.se/79547.txt gems:downloads:list

sake gems:downloads:list | less

This will give an output like:


------------------------------------------------

|                              Gem | Downloads |

------------------------------------------------

|                            rails |   1194471 |

|                     activerecord |   1121778 |

|                       actionpack |   1054718 |

|                    activesupport |    990851 |

|                     actionmailer |    960759 |

|                 actionwebservice |    948640 |

|                             rake |    860824 |

|                            mysql |    593476 |

|                             fcgi |    230394 |

|                          mongrel |    220370 |

|                          daemons |    167443 |

|                          rmagick |    164537 |

|                       gem_plugin |    153505 |

|                         RedCloth |    147182 |

|                  rubygems-update |    119615 |

|                          net-ssh |    114369 |

|                     sqlite3-ruby |    105796 |

|                       fastthread |     95534 |

|            cgi_multipart_eof_fix |     95399 |

|                           needle |     87718 |

Sake is way cool. It was just too easy to implement and deploy this. Have fun making your own sake tasks.

LABS
Rails, Slashdotted: no problem

By Steve Conover and Brian Takita

Peer-to-Patent, one of Pivotal Labs' clients, got Slashdotted last week, and we had no trouble handling the load. The site was just as responsive as it always is, and we didn't come close to having a scale problem.

Moral of the story: the technology for serving static web pages is old, boring, and extremely scalable. If you have the type of site that can be page-cached, do so aggressively, starting with the front page and any pages likely to be linked to. We got a huge payoff for the engineering time that we invested in our page-caching strategy.

Highlights:

  • We moved away from Rails page-caching and developed our own "holeless cache", which uses a symlink trick (see below) to instantly and "holelessly" switch to a new version of a cached page. (The cache "hole" is the time between the expiration or purge of a cached page and the time when it's regenerated. The danger is that in that time your Mongrels can be saturated with requests - something we proved to ourselves could easily happen.)
  • Here's our symlink trick, using the front page as an example:

    1. Have index.html point to index.html.current
    2. If (index.html.current is >= 20 minutes old)
      1. Copy index.html.current to index.html.old
      2. Point index.html to index.html.old
      3. Rewrite index.html.current by asking Rails for the page (using the process method)
      4. Repoint index.html back at index.html.current
    3. Repeat step 2 every minute using a cron job.
  • For cache expiration that's model-based, we make a call from the model observer class to our holeless cache routine, instead of using Rails cache sweepers. So, instead of just deleting the cached page we regenerate it in place.

  • It was important to write tests that proved that the HTML we generated for cached pages looked exactly the same in different "modes" (user logged in vs not, for example). This forced us to push modal decision logic out of Markaby templates and into JavaScript, meaning that view-oriented Rspec tests asserting modal differences became useless. We rewrote them as Selenium tests.

  • Performance/load testing: we tried several tools and approaches and found that a simple Ruby script that launches wget requests (that write to /dev/null) in many separate threads worked best for us.

  • We send down exactly one .js and one .css file. If you are sending down more than one of each of these to the browser, you have a performance problem. Fix it with asset packager.

Update: one clarification about the cron job: we deploy this "automatically" using capistrano.

LABS
Redefining Constants

We all like a good oxymoron, like redefining constants. There are times where we need to redefine a constant to test an edge case in the application code. Before I go into this example, please note that redefining constants is generally not a good way to have maintainable software. If you find yourself needing to redefine a constant, it may be an indication that refactoring is needed.

Given that, lets get into an example where you may need to redefine a constant. Lets say an app has does file uploads to Amazon's S3 service. A common practice to upload to a real S3 account made for the production, development, or demo environment.

When in the test environment, a fake S3 service would be used instead. The fake service is useful to keep your tests fast and running predictably.

To get a different File Upload service object in each of your environments, one can have the S3 configuration in the environment files:

test.rb


STORAGE_SERVICE = FakeStorageService.new

development.rb


STORAGE_SERVICE = S3StorageService.new("development_service", "access_key", "secret_access_key")

production.rb


STORAGE_SERVICE = S3StorageService.new("production_service", "access_key", "secret_access_key")

The File Upload service objects can be set to constants in the environment file. This works great when testing the logic of the objects that use the File Upload service. However it is a good idea to run an integration test that does a real upload.

Since the tests are running in the test environment, a fake File Upload service is being used. Well now we want to use a real service that points to a test S3 account. An easy trick is to redefine the constant to the S3 service in setup and then redefine the constant back to the fake service on teardown.

There are a few ways of doing this...

Just Reset the Constant


context "A real S3 call" do

  setup do

    STORAGE_SERVICE = S3StorageService.new("test_service", "access_key", "secret_access_key")

  end

  teardown do

    STORAGE_SERVICE = FakeStorageService.new

  end

end

This is the simplest approach, but it produces an error:

warning: already initialized constant STORAGE_SERVICE
###Use silence_warnings

context "A real S3 call" do

  setup do

    silence_warnings do

      STORAGE_SERVICE = S3StorageService.new("test_service", "access_key", "secret_access_key")

    end

  end

  teardown do

    silence_warnings do

      STORAGE_SERVICE = FakeStorageService.new

    end

  end

end

This solution removes the warning, but now a certain section of your code will not have warning at all. Also, one could argue that you lose semantic meaning. It also feels like a hack.

Redefine the Constant


class Module

  def redefine_const(name, value)

    __send__(:remove_const, name) if const_defined?(name)

    const_set(name, value)

  end

end



context "A real S3 call" do

  setup do

    Object.redefine_const(

      :STORAGE_SERVICE,

      S3StorageService.new("test_service", "access_key", "secret_access_key")

    )

  end

  teardown do

    Object.redefine_const(

      :STORAGE_SERVICE,

      STORAGE_SERVICE = FakeStorageService.new

    )

  end

end

Calling redefining the constant does not generate a warning. Also it does provide semantic value because you are actively declaring that you are redefining the constant. If there are other warnings, you will also see them.

Its all Dirty

Redefining constants is a non-standard tatic, especially for those new to Ruby. Since this is unconventional and is often contrary to assumptions, it may lead to unpredictable behavior.

Maybe the storage service can be an attribute that can be changed for individual tests.