← Back to Upcase

A Tale of Two Workers (Sidekiq workers)


(Jason Pierce) #1

So I have these two sidekiq workers, both should connect to and save data to a database. The specific database can change based on the Account.

Here is the one that works: (there are other methods in the worker)

class GaDataloadProcessor < GaWorker
  attr_reader :profile, :dataload, :connection, :results

  def perform(id)
    @dataload = Dataload::GoogleAnalytics.find(id)
    connector = dataload.google_analytics_connector
    with_session(connector.username, connector.password) do
      load_profile(connector.account, dataload.profile)

      run_report

      setup_db_connection

      import_results
    end
  end

  private

  def setup_db_connection
    @connection = ActiveRecord::RdsDb.get_connection(dataload.account)
  end

I start this worker in the console with a valid ID and I get:

irb(main):004:0> ga_worker.perform(2)
  Dataload::GoogleAnalytics Load (3.0ms)  SELECT "dataload_google_analytics".* FROM "dataload_google_analytics" WHERE "dataload_google_analytics"."id" = $1 LIMIT 1  [["id", 2]]
  Connector::GoogleAnalytics Load (0.7ms)  SELECT "connector_google_analytics".* FROM "connector_google_analytics" WHERE "connector_google_analytics"."id" = 7 LIMIT 1
  Account Load (1.2ms)  SELECT "accounts".* FROM "accounts" WHERE "accounts"."id" = 16 LIMIT 1
   (76.1ms)  insert into google_analytics_test2 (visitors,date) values ('4','20131018')

Awesome, works as expected. This worker does not work (bad worker): (and there are other methods in the worker)

class CmpUpdateWorker < BaseWorker
  attr_reader :dataload, :connection

  def perform(dataload_id)
    @dataload = DataloadMailchimp.find(dataload_id)
    @gibbon = Gibbon.new(@dataload.api_key)
   
    setup_db_connection

    run_report
    import_results
end

private

  def setup_db_connection
    @connection = ActiveRecord::RdsDb.get_connection(dataload.account)
  end

And when I execute this worker from console I get this:

irb(main):002:0> cmp_worker.perform(5)
  DataloadMailchimp Load (68.0ms)  SELECT "dataload_mailchimps".* FROM "dataload_mailchimps" WHERE "dataload_mailchimps"."id" = $1 LIMIT 1  [["id", 5]]
NoMethodError: undefined method `rds_username' for 16:Fixnum

16 is the correct account_id and it is in the database. That record also has the correct rds_username and other db connection values.

Both models referenced in the workers have belongs_to :account. I have been testing and trying to narrow this down but am stuck. I know there is a lot more code than I have posted here. I am happy to add anything that may help in solving this to a gist or access to the repo.

I appreciate any advice or direction. Thanks.


(Jason Pierce) #2

I got the CmpUpdateWorker to run by updating setup_db_connection to:

  def setup_db_connection
    @connection = ActiveRecord::RdsDb.get_connection(@dataload.account)
  end

Why would changing dataload from an instance to local variable make this work? Especially considering that it works as an instance variable for the other worker.