Integration vs Unit Testing

Testing
1
Joe and Ben discuss the reasons for testing, and how integration and unit testing each serve different reasons. Dive into the benefits of drawbacks of testing at each point in the isolation/integration spectrum by looking through examples.
This is a companion discussion topic for the original entry at https://thoughtbot.com/upcase/videos/integration-vs-unit-testing
2

is the code for those tests / classes available as a github repo? thx

3

Off-topic
@jferris have you moved to a thinkpad?
If yes what OS are you running?

4

Yeah, I switched a couple months ago. Iā€™m running Arch Linux.

5

May I ask why you switched?

6

My history with OS X is a complex and sorry story, but hereā€™s a summary:

I used Linux for seven or eight years before getting frustrated with it in 2007. I was tired of fighting video cards, Wifi adapters, and chasing down new hardware glitches with system updates. There was also a lot of conflict in the community around things like compositing window managers and cross-desktop preferences. It seemed like KDE4 would never be released and every other system felt like it was moving faster. I switched to OS X in 2007 out of the hopes that it would be an easier system to maintain.

Although it did have excellent hardware support, I found that it required no less effort to maintain after updates. Developer tools like GCC, Ruby, and Vim, tmux debuggers were broken for me between most major releases. This was made more frustrating by the fact that Apple was clearly dedicating most of its resources towards mobile development, so people using OS X as a web development platform had their bugs and request de-prioritized.

After OS X proved to be no easier to use (for a developer) than Linux, I decided to switch back. If youā€™re going to spend time maintaining a system, it may as well be open source.

One nice thing about being away from Linux for almost a decade is that most of the things that bugged me before have been fixed in the mean time, and most of the conflicts I remembered have been resolved and forgotten by most of the community. Leaving some of the tools I loved has also given me a greater appreciation for them now that I have them back. Although I doubt Iā€™ll ever switch back to OS X, Iā€™m glad that I used it for a while. I think trying another system is good for you.

4 Likes
7

@benorenstein what vim snippet were you using for capybara? Anywhere I can download that? =)

8

@jferris You say that you shouldnā€™t mock out a collaborator and just test the message because thereā€™s no test to make sure that the message is actually valid - why not? Whatā€™s your opinion on contract tests?

9

I think itā€™s important to have at least one test where the collaborator isnā€™t mocked out, even if it just gets exercised and not verified. This is mostly what I use integration test for. All the details and most of the expectations are in unit tests where I stub at will, but I like to have a UI-driven test where nothing is stubbed out to make sure the methods actually exist.

This is necessary in Ruby and JavaScript because thereā€™s nothing in place to enforce that contract. Itā€™s very easy during refactoring to end up with green, stub-based tests where nothing actually works.

1 Like
10

@jferris Is it fair to say that the missing link is contract verification? Iā€™ve been using GitHub - psyho/bogus: Fake library for Ruby to pretty good effect.

11

Things like bogus or ā€œstrong mockingā€ (now supported by RSpec and mocha) will help reduce problems. However, theyā€™re not good enough to replace integration tests.

For example, RSpec will warn you if youā€™re stubbing a non-existent method. You can also set up arity checks to make sure that the correct number of arguments is used. However, Ruby is dynamic, and thereā€™s no way to know that the expected interfaces of all passed objects, returned objects, and composed objects will match.

Hereā€™s an example where bogus provides no safety:

require "bogus/rspec"

class Greeter
  def hello(name)
    "Hello, #{name.capitalize}!"
  end
end

class Composer
  def initialize(greeter)
    @greeter = greeter
  end

  def greet(name)
    @greeter.hello(name)
  end
end

describe Composer do
  fake(:greeter)

  describe "#greet" do
    it "uses its greeter" do
      composer = Composer.new(greeter)

      composer.greet(["billy"])

      expect(greeter).to have_received.hello(["billy"])
    end
  end
end

Although an Array will never work, bogus happily verifies that youā€™ve called the fake with an Array. Similarly, dynamic methods like send or instance_eval will slip underneath most layers of detection.

A dynamically typed language canā€™t really provide any guarantees that something works until run time, so you need to use actual objects working together at some point if you want to know they work.

1 Like
12

Thank you @jferris. I actually just ran into a similar problem - despite my best efforts verifying contracts, I ended up with a broken system largely due to Bogusā€™ inability to verify objects against fakes.

However, your example has a small problem - Greeter should have a test file, no? At the top of that file will be something like verify_contract :greeter. Then, if your Greeter instance in that test file (describe Greeter) doesnā€™t test #hello with the argument [ā€œbillyā€], itā€™ll throw an error at the end of your test suite.

What can happen is that test which accepts [ā€œbillyā€] can be useless, but it still stops the problem you mention and is the other half of a contract test.

Documentation is there - itā€™s pretty neat, Iā€™d take a look.