I use Rack::Timeout on my Heroku apps to cut off requests before Heroku kills the request. I like being able to get the stacktrace of what was going on in the app. However, I occasionally get a Rack::Timeout cutoff that appears to just be network slowness on Heroku’s part. These exceptions generate a report in Airbrake, and then I inevitably get notified by an alarm by PagerDuty.
Here’s my question: I don’t want to just not get these exceptions as reports, because if there are many of them it might point to an application problem, but a random cut off connection here or there isn’t something I have much control over. What are thoughtbot engineers doing with applications they monitor to strike a balance between attentiveness to exceptions and avoiding a (mostly actionless) alarm at 3am?