Close
Glad You're Ready. Let's Get Started!

Let us know how we can contact you.

Thank you!

We'll respond shortly.

LABS
Logically Negating an ActiveRecord scope

If you simply want to know how to negate an ActiveRecord scope, and you don’t care how it works, here’s the TL;DR:

scope(:not), ->(scope) { where(scope.where_values.reduce(:and).not) }

Arel is the powerful library that powers the Rails ActiveRecord library. It’s also black magic (and largely undocumented black magic at that). It was created in isolation from Rails, and later retrofitted into ActiveRecord. Prepare yourself for a crash course in the dark arts of Arel.

Imagine you’re creating a blog, and you needed a scope that would show you all of the posts that were recently tagged:

class Post < ActiveRecord::Base   scope :recently_tagged, -> {
    where(
      arel_table[:tagged_at].gt(10.days.ago)
    ).
    where(arel_table[:tags].not_eq(nil))
  }
end

We’ve chained two `where` clauses together with ActiveRecord, which will join them together with `AND`. In other words, a post that has tags and has had its tags changed within the last 10 days is “recently tagged”. Freed of the syntactic trappings of ActiveRecord and Arel, it can be simply represented with the following boolean expression:

tagged_at > 10.days.ago && tags != nil

Now suppose you’d like to find the opposite: the list of all posts that were not recently tagged (either because their tags haven’t changed in the last 10 days, or because all of their tags were removed). In boolean logic, this is simply the logical negation of the “recently_tagged” scope:

!(tagged_at > 10.days.ago && tags != nil)

which can be simplified to:

tagged_at <= 10.days.ago || tags == nil

The question is, how do we tell ActiveRecord / Arel to negate a scope? The answer is more difficult than you might expect.

Let’s start with looking at what a scope is:

> Post.recently_tagged.class
 => ActiveRecord::Relation::ActiveRecord_Relation_Post

That seems slightly odd. Apparently ActiveRecord generates a class for a scope. Let’s look at what this class is made of:

ActiveRecord::Relation::ActiveRecord_Relation_Post.ancestors
 => [ActiveRecord::Relation::ActiveRecord_Relation_Post, ActiveRecord::Delegation::ClassSpecificRelation, ActiveRecord::Relation, ActiveRecord::Relation::DeprecatedMethods, …...........]

Not suprisingly, it looks like ActiveRecord_Relation_Post is a type of ActiveRecord::Relation. If you peek inside ActiveRecord::Relation, you’ll see it’s a sort of wrapper around Arel (along with a few other ActiveRecord classes and modules). At the top of the ActiveRecord::Relation class file, there are several constants defined, including the following:

MULTI_VALUE_METHODS  = [
  :includes,
  :eager_load,
  :preload,
  :select,
  :group,
  :order,
  :joins,
  :where,
  :having,
  :bind,
  :references,
  :extending
]

This constant, along with the SINGLE_VALUE_METHODS constant, describes all of the possible constituents of an ActiveRecord scope. In our case, we want to peel off the where values of a scope so that we can logically negate them. Searching for usages of the ‘MULTI_VALUE_METHODS’ will lead you to this juicy bit of metaprogramming in ActiveRecord::QueryMethods:

Relation::MULTI_VALUE_METHODS.each do |name|
  class_eval <<-CODE, __FILE__, __LINE__ + 1
    def #{name}_values                   # def select_values
      @values[:#{name}] || []            #   @values[:select] || []
    end                                  # end
                                         #
    def #{name}_values=(values)          # def select_values=(values)
      raise ImmutableRelation if @loaded #   raise ImmutableRelation if @loaded
      @values[:#{name}] = values         #   @values[:select] = values
    end                                  # end
  CODE
end

ActiveRecord creates accessor methods for each of the scope’s constituent parts; since we’re interested in the where values, let’s see what `where_values` returns (note that you could also call values[:where]):

> Post.recently_tagged.where_values => [#<Arel::Nodes::GreaterThan:0x007fa759f263b0 @left=#<struct Arel::Attributes::Attribute relation=#<Arel::Table:0x007fa75d011ba0 @name="posts", @engine=Post(id: integer, tagged_at: datetime, tags: string, created_at: datetime, updated_at: datetime), @columns=nil, @aliases=[], @table_alias=nil, @primary_key=nil>, name=:tagged_at>, @right=Fri, 28 Jun 2013 05:17:39 UTC +00:00>, #<Arel::Nodes::NotEqual:0x007fa759f26018 @left=#<struct Arel::Attributes::Attribute relation=#<Arel::Table:0x007fa75d011ba0 @name="posts", @engine=Post(id: integer, tagged_at: datetime, tags: string, created_at: datetime, updated_at: datetime), @columns=nil, @aliases=[], @table_alias=nil, @primary_key=nil>, name=:tags>, @right=nil>]

Huzzah! We’ve now found the mysterious Arel. Although these class inspections don’t make it entirely obvious, these Arel nodes actually form the nodes of a tree representing boolean expressions in SQL.

The first node is:

         >
        / 
       /   
      /     
     /       
:tagged_at  10.days.ago

And the second is simply:

        !=
        / 
       /   
    :tags  nil

Now that we have these where values, what can we do with them? Well, we’d like to AND them back together, and then negate them. Fortunately, there are methods for both of these things on any Arel::Nodes::Node:

> Post.recently_tagged.values[:where].first.method(:not)
 #<Method: Arel::Nodes::GreaterThan(Arel::Nodes::Node)#not>
> Post.recently_tagged.values[:where].first.method(:and)
  #<Method: Arel::Nodes::GreaterThan(Arel::Nodes::Node)#and>

As you would expect, calling either of these methods will return a new Arel::Nodes::Node. In our specific case, we could call these methods directly:

> Post.recently_tagged.values[:where].first.and(Post.recently_tagged.values[:where].last).not

The `not` at the end of this expression is what we’ve been searching for. However, this returns an Arel::Nodes::Node (or more specifically, an Arel::Nodes::Not), but we’re using ActiveRecord – we need an ActiveRecord::Relation. Luckily, that’s exactly what `where` returns, and it can take, as an argument, an Arel::Nodes::Node.

> Post.where(Post.recently_tagged.values[:where].first.and(Post.recently_tagged.values[:where].last).not)

And this will, in fact, return what we’ve been searching for: all posts not recently tagged. It’s of course, incredibly ugly and not very reusable; luckily, we can create a generic negation scope with a little help from Ruby’s Enumerable#reduce method:

class Post
  #....
  scope :not, ->(scope_name) { where(send(scope_name).where_values.reduce(:and).not) }
end

And now, you could use it whenever you like:

Post.not(:recently_tagged)

Comments
  1. David says:

    would the following not be nicer as it would allow chaining:

    “`
    class Post
    #….
    def self.not
    where_values = current_scope.where_values
    current_scope.where_values = []
    where(where_values.reduce(:and).not)
    end
    end

    Post.recently_tagged.not.from_user(x)
    “`

    * could not get this to working using just scopes as the current_scope object was not set inside the scope method.

  2. Mark Johnson says:

    Perhaps I’m missing something but if a post’s tagged_at is greater than 10 days ago then how is that recently tagged? It’s early in the morning so perhaps I’m reading this all wrong.

  3. One of them says “from_now” where it should say “ago” ;)

    Future tags?!

  4. Mark Johnson says:

    Ha yeah sorry about that. I tried to delete it as soon as I hit send. I was half asleep.

  5. Tim Scott says:

    Maybe I am misunderstanding something. This returns a string where scope is an ActiveRecord::Relation.

    scope.where_values.reduce(:and)

    Which explains the error I get:

    undefined method `not’ for “foo.bar is not null or bar.foo is not null”:String

    I’m using 3.2.14. Thoughts?

  6. Ім'я says:

    This apparently doesn’t work when you mix other tables into the game

  7. Matty says:

    I’ve found this won’t work if you’ve used where(string). In this case the where value is a string and won’t reduce. It seems to work in you map the string where values to Arel::Nodes::SqlLiteral before reducing.

  8. Mike says:

    I changed this to a Rails3 syntax, but it doesn’t seem to work

    scope :not, where(scope.where_values.reduce(:and).not)

    >>
    ArgumentError: wrong number of arguments (0 for 1)
    from /home/mike/.rvm/gems/ruby-1.9.3-p547/gems/activerecord-3.2.11/lib/active_record/scoping/named.rb:174:in `scope’

  9. Daniel says:

    This post is awesome. Thank you for sharing.

  10. Sebastian says:

    This helped me a lot but didnt work with chaining scopes. it would invert the hole scope so if you did something like not(:foo).not(:bar) you would end with NOT(NOT foo). With `unscoped` before send it works however:
    scope :invert, ->(scope_name) { where(unscoped.send(scope_name).where_values.reduce(:and).not) }

Post a Comment

Your Information (Name required. Email address will not be displayed with comment.)

* Copy This Password *

* Type Or Paste Password Here *