Number of Assertions In Integration Tests

At the end of Writing Integration Tests video we got two assertions:

  1. On the correct todo displayed for the user.
  2. Assertion that the wrong todo won’t be displayed for the signed in user.

What is the rule of thumb for these kind of tests? It’s clear that we’d like to test this logic but sometimes it could be pretty complicated and I tend to put 3,4,5 and more assertions of this sort. Is that a good idea or it should be tested somewhere else, say in controller tests, not in integration? But at the same time it makes sense because this is the logic for displaying things to the user.

1 Like

I imagine @joshclayton will have his own opinions on this, but I’d be concerned if I felt the need to include 5 assertions in a test.

I don’t think you should never have more than 1 assertion, but if you have many, I think it’s a good idea to ask yourself some questions:

  1. Why do I need to assert so many things to be confident this is working?
  2. If I have this many assertions, isn’t it likely that I’m re-testing something that’s already tested elsewhere?
  3. If I tested this functionality with a different kind of spec, would it be more focused?

Like most things in development, there are few rules that you should follow all the time. I’d consider these questions, weigh your options, and make the best decision you can.

Hope this helps!

Thx, this makes sense and I guess because I already tried to ask myself these questions I’m still struggling to figure out if I’d cover this somewhere else. For example, say I have a list of todos and I assert in integration tests that I see the right todos for the user(it’s already 2 assertions as above). And also I assert that I have them in the right order(sorting of some kind) - it’s also 1 or 2 so I got 4 already.

If I were testing two different things (that the user sees the todos and that they’re sorted), I’d write two different tests. I’d put the related assertions in their appropriate tests.

But you still think that sorting assertion should be in the integration tests?

Depends on where the sorting is happening.

I would prefer that sorting happens as part of a scope on a model. In that case, I’d test that in the model’s unit tests.

@alexbush, I don’t mind putting multiple assertions in acceptance tests because you’re testing the behavior of the application and not just one “unit” (hence the use of unit tests for the very fine-grained aspects). In unit tests, I’ll adhere to “one assertion per test” more strictly because testing one unit (typically a method) is often asserting one output given a known input. Example:

class User < ActiveRecord::Base
  def full_name
    [first_name, last_name].compact.join(' ')
  end
end

describe User, '#full_name' do
  it 'joins both first and last name' do
    user = User.new(first_name: 'Jane', last_name: 'Doe')
    expect(user.full_name).to eq 'Jane Doe'
  end

  it 'returns first name if no last name is present' do
    user = User.new(first_name: 'Jane')
    expect(user.full_name).to eq 'Jane'
  end

  it 'returns last name if no first name is present' do
    user = User.new(last_name: 'Doe')
    expect(user.full_name).to eq 'Doe'
  end
end

In those three assertions, the input (permutations of first_name/last_name with and without a value) are all tested against a known output, so one assertion per test makes a lot of sense.

In the example todo app, I’m writing two assertions against the same data displayed on the page, but a similar test would be to make one assertion that ONLY the todos I expect to show up on the page are. Example:

todo_titles = page.all('.todos li .title').map(&:text)
expect(todo_titles).to eq ['Todo I Own']

This would assert the only todos that show up are ones I own, effectively rolling two assertions into one.

This can’t always happen, however. For example, in an app I’ve been working on recently, I wanted to write acceptance tests surrounding applying and removing filters (which live-updated results displayed on the page); I would apply a filter, assert the data was correct, and then either apply another filter or disable the filter. I feel like this sort of assertion block is an integral part of acceptance testing because only doing one assertion would start to fall into the “unit” category (in this example, we already had Javascript unit tests to ensure the Backbone was filtering correctly).

One thing we often forget is that acceptance tests with RSpec + Capybara are doing assertions all the time, even if we aren’t writing expect statements. Using Capybara’s find and within methods will raise a Capybara::ElementNotFound exception if something isn’t on the page, for example. Form interactions will outright break in a similar fashion if not on the correct page. The nature of acceptance tests is to write expectations across many levels of the application from the standpoint of the user/web browser.

All that said, I do think breaking things out into smaller, more focused pieces is going to be a big win. If you have an object using a state machine and a view that renders gobs of different information based on the object state, I’d test that with view specs asserting that the markup is correct and write an acceptance test that briefly touches the markup generated by the view(s) to make sure that it’s displayed to the user. Same thing goes for validations: I’d rather write unit tests making sure the validations are in place on a model versus writing an acceptance test that fills out a form incorrectly and asserts that each and every error message is displayed. Instead, I’d just test submitting the form with one invalid value and assert that errors are displayed (without getting specific about the error message). Unit tests have the benefit of (hopefully) being blazing fast because they touch so little of the application.

Finally, I’d still test sorting at an integration level in the generic “view whatever” scenario if ordering is a large part of the business. A simple example would be blog posts; even though my model contains the ordering logic (most likely created_at DESC) and would be tested at the unit level, I’d still ensure that the user SEES that scope in action on the #index page. A blog’s primary goal is to display relevant information in the correct order, so I’d consider that a must-test from an acceptance level because the wrong order would completely and utterly confuse users.

Like @benorenstein mentioned, there’s no hard and fast rule for testing things in different scenarios, but those are the experiences I’ve had personally. Good luck!

2 Likes

Thank you both @joshclayton and @benorenstein for such a comprehensive answers!