AF
Ash Framework•2w ago
pikdum

AshOban scheduled action crashing Oban.Plugins.Cron

We have something like this: 
  oban do
    domain Foobar.Translations

    scheduled_actions do
      schedule :create_nightly_global_batch_request, "0 0 * * *" do
        worker_module_name Foobar.Translations.BatchTranslationsRequest.AshOban.ActionWorker.CreateNightlyGlobalBatchRequest
      end
    end

    triggers do
      trigger :create_batch_translations_jobs do
        action :create_batch_translations_jobs
        scheduler_cron "* * * * *"
        where expr(status == :pending)

        worker_module_name Foobar.Translations.BatchTranslationsRequest.AshOban.Worker.CreateBatchTranslationsJobs

        scheduler_module_name Foobar.Translations.BatchTranslationsRequest.AshOban.Scheduler.CreateBatchTranslationsJobs
      end
    end
  end
  oban do
    domain Foobar.Translations

    scheduled_actions do
      schedule :create_nightly_global_batch_request, "0 0 * * *" do
        worker_module_name Foobar.Translations.BatchTranslationsRequest.AshOban.ActionWorker.CreateNightlyGlobalBatchRequest
      end
    end

    triggers do
      trigger :create_batch_translations_jobs do
        action :create_batch_translations_jobs
        scheduler_cron "* * * * *"
        where expr(status == :pending)

        worker_module_name Foobar.Translations.BatchTranslationsRequest.AshOban.Worker.CreateBatchTranslationsJobs

        scheduler_module_name Foobar.Translations.BatchTranslationsRequest.AshOban.Scheduler.CreateBatchTranslationsJobs
      end
    end
  end
It looks like that :create_nightly_global_batch_request is crashing Oban.Plugins.Cron at midnight, though. (Log too large, will add as a comment) It seems to have started after we upgraded from ash_oban 0.4.12 to 0.5.1 and Elixir from 1.18.4 OTP 27 to 1.19.2 OTP 28 (among other things). Any ideas? Thanks!
5 Replies
pikdum
pikdumOP•2w ago
pikdum
pikdumOP•2w ago
Error log copied from cloudwatch json logs, so formatting could be a slightly messed up. After a bit more research, this specifically seemed to happen first the night after we upgraded from elixir 1.18.4 otp 27 to elixir 1.19.2 otp 28.1.1. Not sure if it's an elixir 1.19 thing or an otp 28 thing, though. (Or actually totally unrelated? lol) And a bit more information - that happens to be the oban block we had to add domain to for it to compile under Elixir 1.19. Removing it gets this error: 
Erlang/OTP 28 [erts-16.1] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [jit:ns]

Compiling 169 files (.ex)

== Compilation error in file lib/foobar/translations/batch_translations_request.ex ==
** (RuntimeError) Exception in transformer AshOban.Transformers.DefineActionWorkers on Foobar.Translations.BatchTranslationsRequest:

No configuration for `domain` present on Foobar.Translations.BatchTranslationsRequest
    (ash_oban 0.5.1) /home/redacted/foobar/deps/spark/lib/spark/info_generator.ex:249: AshOban.Info.oban_domain!/1
    (ash_oban 0.5.1) lib/transformers/define_action_workers.ex:44: AshOban.Transformers.DefineActionWorkers.define_worker/4
    (ash_oban 0.5.1) lib/transformers/define_action_workers.ex:23: anonymous fn/3 in AshOban.Transformers.DefineActionWorkers.transform/1
    (elixir 1.19.2) lib/enum.ex:2520: Enum."-reduce/3-lists^foldl/2-0-"/3
    (ash_oban 0.5.1) lib/transformers/define_action_workers.ex:19: AshOban.Transformers.DefineActionWorkers.transform/1
    (spark 2.3.14) lib/spark/dsl/extension.ex:732: anonymous fn/4 in Spark.Dsl.Extension.run_transformers/4
    (elixir 1.19.2) lib/enum.ex:5023: Enumerable.List.reduce/3
    /home/redacted/foobar/lib/foobar/translations/batch_translations_request.ex:1: (file)
Erlang/OTP 28 [erts-16.1] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [jit:ns]

Compiling 169 files (.ex)

== Compilation error in file lib/foobar/translations/batch_translations_request.ex ==
** (RuntimeError) Exception in transformer AshOban.Transformers.DefineActionWorkers on Foobar.Translations.BatchTranslationsRequest:

No configuration for `domain` present on Foobar.Translations.BatchTranslationsRequest
    (ash_oban 0.5.1) /home/redacted/foobar/deps/spark/lib/spark/info_generator.ex:249: AshOban.Info.oban_domain!/1
    (ash_oban 0.5.1) lib/transformers/define_action_workers.ex:44: AshOban.Transformers.DefineActionWorkers.define_worker/4
    (ash_oban 0.5.1) lib/transformers/define_action_workers.ex:23: anonymous fn/3 in AshOban.Transformers.DefineActionWorkers.transform/1
    (elixir 1.19.2) lib/enum.ex:2520: Enum."-reduce/3-lists^foldl/2-0-"/3
    (ash_oban 0.5.1) lib/transformers/define_action_workers.ex:19: AshOban.Transformers.DefineActionWorkers.transform/1
    (spark 2.3.14) lib/spark/dsl/extension.ex:732: anonymous fn/4 in Spark.Dsl.Extension.run_transformers/4
    (elixir 1.19.2) lib/enum.ex:5023: Enumerable.List.reduce/3
    /home/redacted/foobar/lib/foobar/translations/batch_translations_request.ex:1: (file)
Okay, I've found a few interesting things. * the presence of that scheduled_actions block is what requires domain to be present - if I remove scheduled_actions I can also remove domain and compile without errors * the queue for that that schedule was being set to nil instead of auto-generated, if I manually set the queue I'm hoping this will fix my issues These seem to be Elixir 1.19 issues, as far as I can tell 1.18 works fine. Best guess is the async compiler changes? Oh, this also looks reproducible with the current ash_oban tests using Elixir 1.19.2 and OTP 28. :)
ZachDaniel
ZachDaniel•2w ago
strange, please open an issue, will look into it. It is likely some kind of missing Code.ensure_compiled due to changes in 1.19
pikdum
pikdumOP•2w ago
yeah, that's my guess too. ran into one of those in our codebase will plan on making a ticket tomorrow
ZachDaniel
ZachDaniel•2w ago
I pushed something to main to try

Did you find this page helpful?