Topics

Ash Framework•4mo ago

Bulk update through AshAdmin is skipping 200 records, not sure why

I have a bulk action wired up like this:

    update :backfill_code do
      change fn changeset, _ ->
        Ash.Changeset.change_attribute(
          changeset,
          :code,
          Foobar.Utils.CodeGenerator.generate_code(changeset.data.id)
        )
      end

      require_atomic? false
    end

    action :backfill_codes, :map do
      run fn _input, context ->
        {:ok,
         Ash.bulk_update!(Foobar.Baz, :backfill_code, %{},
           actor: context.actor,
           resource: Foobar.Baz,
           strategy: [:stream]
         )}
      end
    end

    update :backfill_code do
      change fn changeset, _ ->
        Ash.Changeset.change_attribute(
          changeset,
          :code,
          Foobar.Utils.CodeGenerator.generate_code(changeset.data.id)
        )
      end

      require_atomic? false
    end

    action :backfill_codes, :map do
      run fn _input, context ->
        {:ok,
         Ash.bulk_update!(Foobar.Baz, :backfill_code, %{},
           actor: context.actor,
           resource: Foobar.Baz,
           strategy: [:stream]
         )}
      end
    end

I'm triggering this through AshAdmin, since I don't have access to iex in our environments. I've noticed that after running this, there are always 200 records that still don't have codes. If I go into AshAdmin and run :backfill_code for that specific record, it'll get a code. I was thinking maybe it was policies on the read action or something, but that's always(). Any ideas? Maybe it's timing out in AshAdmin or something?

37 Replies

pikdumOP•4mo ago

Looking at it more, I'm thinking timeout. Think by default batches are 100, so a consistent 200 remaining would make sense. That and this in the console:

phx-GEY9YTxv3Ab2a3VB timeout: received timeout while communicating with server. Falling back to hard refresh for recovery - undefined

ZachDaniel•4mo ago

🤔 unlikely to be timing out. It may be a bug in our streaming code. oh, interesting Then yeah it could be that? Try and do it in iex and see it works

pikdumOP•4mo ago

setup makes that a bit complicated, lol my guess is it's hitting a 30s live view timeout and reloading the page, killing the execution

ZachDaniel•4mo ago

Is it very expensive?

pikdumOP•4mo ago

nah my setup is just not great to reproduce prod data locally right now thoughts on making generic action execution async in ash admin if that's the issue i'm running into? could probably make a PR

ZachDaniel•4mo ago

Is there a 30s liveview timeout?

pikdumOP•4mo ago

Failed to push form submit Object { timeout: true }

https://github.com/phoenixframework/phoenix_live_view/blob/06fc6d0f24af0e9e86a6167075e5bc6cf1594f54/assets/js/phoenix_live_view/view.js#L1784

pikdumOP•4mo ago

not sure if it's 30 seconds or not, but found this ticket that seems to reference that: https://github.com/phoenixframework/phoenix_live_view/issues/2305

GitHub

LiveView doesn't wait for long running process to complete and relo...

Environment Elixir version (elixir -v): 1.14.1 Phoenix version (mix deps): current ("6e23d1a1120a241e3bf3e41343a43e2df8a831f2") Phoenix LiveView version (mix deps): current ("52f2cab...

ZachDaniel•4mo ago

Gotcha. I'd want to make it some kind of setting in that case Alternatively, you can make your generic action send to a genserver and/or use Task.async

pikdumOP•4mo ago

wouldn't want to use https://hexdocs.pm/phoenix_live_view/1.0.12/Phoenix.LiveView.html#module-arbitrary-async-operations ?

ZachDaniel•4mo ago

we could, but we wouldn't want to do it for all forms

pikdumOP•4mo ago

could do it just for generic actions?

ZachDaniel•4mo ago

🤔 thats a possibility But I think better to add a DSL config for it

admin do
  long_runing_actions [:foo, :bar]
end

admin do
  long_runing_actions [:foo, :bar]
end

pikdumOP•4mo ago

okay, i was mixing two issues * there is a timeout, but it doesn't actually cancel the action or anything, but does prevent you from getting the output back * my action was erroring, presumably preventing the last two batches from running so i could make a pr to use async for that first issue and to improve on the ux a bit

ZachDaniel•4mo ago

I think we'd still want to do it on an action-by-action basis

pikdumOP•4mo ago

hm not sure there are any downsides though?

ZachDaniel•4mo ago

If something goes wrong the async task exits Its a different way to handle errors I don't want to opt into that for the entire admin UI we can start with it being opt-in and if we decide its best and don't find any gotchas we can just default to it everywhere

pikdumOP•4mo ago

https://github.com/ash-project/ash_admin/pull/320 this is sort of what i was thinking isolated to generic actions, rather than the whole UI error handling basically the same, but also catching if the task crashed

GitHub

feat: make generic actions async by pikdum · Pull Request #320 · ...

Makes generic actions async: fixes timeouts that prevent results from being displayed disable run button while action is running change text to &#39;Running...&#39; while running for visua...

ZachDaniel•4mo ago

🤔 I get it, and I get why you want it, but we still have to remember that the admin is like a major cross cutting concern, and you're proposing changing all generic action behavior for everyone who upgrades. We have a bunch of clients using it etc. and I really don't want to change all form submissions even just for generic actions when most cases don't need it At a minimum it should be opt in via config or something but I think to start, from the standpoint of not like...accidentally breaking a critical function for a bunch of people who rely on it, we should make it opt-in via action name. its not hard to do that 🙂

pikdumOP•4mo ago

i'm kinda viewing it as a bug though bit tricky since there's no test coverage for generic actions could build out a test harness for generic actions, then split into a sync + async flow and tests for each but that's a lot more effort than fixing a bug

ZachDaniel•4mo ago

I'm not trying to be an ass, but we're going to have to gradually adopt it if we adopt it at all 🙂

pikdumOP•4mo ago

=/

ZachDaniel•4mo ago

I'm not going to accept a PR that opts into that automatically for actions I'm going to be the one that has to deal with issues that arise, so it shouldn't be a problem to add this in an optional fashion to start We don't automatically start processes and do things async anywhere else in Ash

pikdumOP•4mo ago

not sure if there's a way to fix the bug without having it be async though

ZachDaniel•4mo ago

I just explained that you can make it opt in with resource config I'm not saying we can't ever do it I'm just saying the user needs to select which actions should be submitted aysnc for now

pikdumOP•4mo ago

i could, but there's no tests for the existing behavior or anything so not a straightforward way to prevent regressions either way

ZachDaniel•4mo ago

another reason not to change how everything works

pikdumOP•4mo ago

lol

ZachDaniel•4mo ago

This is just a precursor to what you want Its a very common way to make changes start w/ opt-in so that people who need it now can vet it in production and then when we've fleshed it out and know its not causing issues, we can turn it on for everything by default

pikdumOP•4mo ago

even with opt-in i'd need to change code that has no tests but yeah i get what you're saying

ZachDaniel•4mo ago

There are tests for form submission as of recently by virtue of file uploads (another example of something we don't know works with async stuff BTW 😄 ) I don't know if you can consume uploads from another process

pikdumOP•4mo ago

¯\_(ツ)_/¯

ZachDaniel•4mo ago

any form can contain :file inputs so we'd have to find out if that works with async form submission at a minum to make it automatic But if you make it opt-in on an action by action basis, we have no such issues you can just test that it doesn't blow up for the one action you care about and we're good

pikdumOP•4mo ago

would the most straightforward way to make a totally separate AshAdmin.Components.Resource.GenericActionAsync or similar with everything copied over? or should i try and handle both in the same module

ZachDaniel•4mo ago

Yeah, I'd try and handle it in the same module

pikdumOP•4mo ago

thoughts on these ways to configure it? application config:

config :ash_admin, :async_generic_actions?: true

config :ash_admin, :async_generic_actions?: true

vs per domain boolean dsl config:

admin do
  show? true
  async_generic_actions? true
end

admin do
  show? true
  async_generic_actions? true
end

or list specific actions within domain config:

admin do
  show? true
  async_generic_actions [:foo, :bar]
end

admin do
  show? true
  async_generic_actions [:foo, :bar]
end

thinking the first is probably the easiest to wire up, but I'm fine with whichever was thinking async_generic_actions instead of long_running_actions for clarity on what it's changing, but can do either

ZachDaniel•4mo ago

If it's opt in, I'm okay actually with doing it for any action type So just async_actions

Ash Framework Join

The Elixir backend framework for unparalleled productivity. Declarative tools that let you stop wasting time. Use with Phoenix LiveView or build APIs in minutes for your front-end of choice.

2KMembers

View on Discord

Did you find this page helpful?