← Back to Upcase

Integration vs Unit Testing

Joe and Ben discuss the reasons for testing, and how integration and unit testing each serve different reasons. Dive into the benefits of drawbacks of testing at each point in the isolation/integration spectrum by looking through examples.
This is a companion discussion topic for the original entry at https://thoughtbot.com/upcase/videos/integration-vs-unit-testing

is the code for those tests / classes available as a github repo? thx

Off-topic
@jferris have you moved to a thinkpad?
If yes what OS are you running?

Yeah, I switched a couple months ago. I’m running Arch Linux.

May I ask why you switched?

My history with OS X is a complex and sorry story, but here’s a summary:

I used Linux for seven or eight years before getting frustrated with it in 2007. I was tired of fighting video cards, Wifi adapters, and chasing down new hardware glitches with system updates. There was also a lot of conflict in the community around things like compositing window managers and cross-desktop preferences. It seemed like KDE4 would never be released and every other system felt like it was moving faster. I switched to OS X in 2007 out of the hopes that it would be an easier system to maintain.

Although it did have excellent hardware support, I found that it required no less effort to maintain after updates. Developer tools like GCC, Ruby, and Vim, tmux debuggers were broken for me between most major releases. This was made more frustrating by the fact that Apple was clearly dedicating most of its resources towards mobile development, so people using OS X as a web development platform had their bugs and request de-prioritized.

After OS X proved to be no easier to use (for a developer) than Linux, I decided to switch back. If you’re going to spend time maintaining a system, it may as well be open source.

One nice thing about being away from Linux for almost a decade is that most of the things that bugged me before have been fixed in the mean time, and most of the conflicts I remembered have been resolved and forgotten by most of the community. Leaving some of the tools I loved has also given me a greater appreciation for them now that I have them back. Although I doubt I’ll ever switch back to OS X, I’m glad that I used it for a while. I think trying another system is good for you.

4 Likes

@benorenstein what vim snippet were you using for capybara? Anywhere I can download that? =)

@jferris You say that you shouldn’t mock out a collaborator and just test the message because there’s no test to make sure that the message is actually valid - why not? What’s your opinion on contract tests?

I think it’s important to have at least one test where the collaborator isn’t mocked out, even if it just gets exercised and not verified. This is mostly what I use integration test for. All the details and most of the expectations are in unit tests where I stub at will, but I like to have a UI-driven test where nothing is stubbed out to make sure the methods actually exist.

This is necessary in Ruby and JavaScript because there’s nothing in place to enforce that contract. It’s very easy during refactoring to end up with green, stub-based tests where nothing actually works.

1 Like

@jferris Is it fair to say that the missing link is contract verification? I’ve been using http://github.com/psyho/bogus to pretty good effect.

Things like bogus or “strong mocking” (now supported by RSpec and mocha) will help reduce problems. However, they’re not good enough to replace integration tests.

For example, RSpec will warn you if you’re stubbing a non-existent method. You can also set up arity checks to make sure that the correct number of arguments is used. However, Ruby is dynamic, and there’s no way to know that the expected interfaces of all passed objects, returned objects, and composed objects will match.

Here’s an example where bogus provides no safety:

require "bogus/rspec"

class Greeter
  def hello(name)
    "Hello, #{name.capitalize}!"
  end
end

class Composer
  def initialize(greeter)
    @greeter = greeter
  end

  def greet(name)
    @greeter.hello(name)
  end
end

describe Composer do
  fake(:greeter)

  describe "#greet" do
    it "uses its greeter" do
      composer = Composer.new(greeter)

      composer.greet(["billy"])

      expect(greeter).to have_received.hello(["billy"])
    end
  end
end

Although an Array will never work, bogus happily verifies that you’ve called the fake with an Array. Similarly, dynamic methods like send or instance_eval will slip underneath most layers of detection.

A dynamically typed language can’t really provide any guarantees that something works until run time, so you need to use actual objects working together at some point if you want to know they work.

1 Like

Thank you @jferris. I actually just ran into a similar problem - despite my best efforts verifying contracts, I ended up with a broken system largely due to Bogus’ inability to verify objects against fakes.

However, your example has a small problem - Greeter should have a test file, no? At the top of that file will be something like verify_contract :greeter. Then, if your Greeter instance in that test file (describe Greeter) doesn’t test #hello with the argument [“billy”], it’ll throw an error at the end of your test suite.

What can happen is that test which accepts [“billy”] can be useless, but it still stops the problem you mention and is the other half of a contract test.

https://www.relishapp.com/bogus/bogus/v/0-1-6/docs/contract-tests

Documentation is there - it’s pretty neat, I’d take a look.