Let’s say you faced this situation on the large codebase
What is the strategy for investigating inconsistently failing tests?
- Be able to re-run the tests in the same order as when it failed (default is random)
- Database snapshot at the point of failure (already have DatabaseCleaner in place)?
Thank you in advance!
I find that inconsistently failing tests are usually caused by one of two things:
- State leaking between tests
- Asynchronous timing problems
The first is usually easier to solve.
If you’re using RSpec, you can provide a seed value to the
rspec command which will cause the tests to be run in the same order each time:
Also, RSpec 3.3 added a
bisect command which will repeatedly run your tests until it can determine which combinations are causing a failure:
Being able to inspect the state of the app the point of the failure is certainly useful - I’d suggest using the
pry-rescue gem for this:
Nice talk on the subject: