I’m doing a project where I have to pull some data from an external API very frequently, possibly every 30 seconds or so. Heroku’s scheduler only allows you to schedule jobs every 10 minutes. Is there a Thoughtbot-approved™ strategy for dealing with this kind of issue?
One of the advantages of the Heroku scheduler is that if a particular job is running behind schedule, I don’t believe it will kick off the same job on top of it. Currently, I’m thinking about having a daily process that seeds the delayed_jobs table with a job to be run every thirty seconds for the upcoming day, and then letting delayed_job handle this task. I’m concerned that if for some reason one of the updates takes longer than expected that they will collide.
Any ideas on how to handle this? Moving off of Heroku is not possible at the current time, so if there are solutions that involve that move, I can consider them, but only as a long-term solution.