The different kinds of tests

Testing
1

I’m trying to learn TDD with Rails and so far I think I have a pretty good idea of how to write unit tests for model code with Rspec. What confuses me is what I should look into next though.

I find it very overwhelming that there are so many different kinds of tests which I don’t feel are very well defined. So far I have heard people mention unit, feature, integration, acceptance, regression, and smoke test. Could some explain or lead me to a resources explaining these different kinds of testing and maybe give me some guidance in where to start learning all of these things, and the tools used to write them in Rails.

2

The big three:

Integration Tests: Testing that everything works together.
Functional Tests: Testing a single controller
Unit Tests: Testing a model

I’ve never knowingly used a regression or a smoke test, I have no idea what they’re for! A feature is generally going to be something that’s tested in an integration test.

For example: here’s a feature being tested in an integration test using Rspec and Capybara:

feature 'Sign in as a user' do
  let(:user) { Fabricate :user }
  scenario 'with an email address and password' do
    visit sign_in_path
    fill_in 'email', with: user.email
    fill_in 'password', with: 'password'
    click_button 'sign in'
    expect(page).to have_content "Welcome #{user.first_name}
  end
end

You can see that the above example isn’t testing just that a User is valid (that would be a unit test) and it’s not testing a specific controller (that would be a functional test,) it’s actually testing an entire range of “stuff” that’s happening: that there’s a sign_in route, that the user is a valid user and that the user gets some kind of “result” from the actions, in this case, they see a welcome message. It’s testing that the user is signed in and that tests whether or not a session was created for the User.

A Unit test in the same context would test things like: does this user have a unique email, is the user password long enough, is the email in a valid format. Everything dealing with specifically a User object would be tested with a Unit test.

An acceptance test is really just a test to determine if a particular piece of functionality is acceptable to the client (or boss/project manager, etc.) For me, the integration tests are generally also the acceptance tests. Meaning: if the integration tests pass, that feature “works” and is therefore accepted.

As far as resources – the best one, given your question would be the TDD with Rails online workshop that Thoughtbot has. You’re already a Prime subscriber, so you’re good to go. It walks you through the types of test in a very practical way – trust me, you go through this course, you’ll understand the different types of testing easily. The Michael Hartl Rails Tutorial stuff is also useful, but I think the Thoughtbot TDD online course will make the answers to your question even more clear.

I am not the definitive expert on this (or anything for that matter,) so someone correct me or elaborate if I’m wrong!

3

Just to expand on @briandear’s answer a little:

Your list of test types mixes two different categories: Unit and integration (and possibly feature) describe what is being tested; acceptance, regression and smoke describe the purpose of the test.

What’s being tested

There’s a great discussion about how these types of tests interact in the workshop Brian mentioned, and in this Byte: When should I write an isolated vs an integrated test?

Why it’s being tested

  • Acceptance tests are tests that tell you when you’re finished; they tell you the feature is ready to be accepted (as @briandear said, these are usually integration tests).
  • Regression tests are tests that prevent bugs creeping into features that used to work. Every time you re-run the tests for features you’re not currently working on you’re performing a regression test.
  • Smoke testing refers to a method of testing physical machines by pumping smoke through a system of pipes so that you can spot any leaks. In software terms it usually refers to doing a quick manual test to make sure that you really do see what you expect and you’ve not made a mistake in your acceptance tests.
1 Like