AE
Ash Elixir•2d ago
mylanconnolly

Flaky Ash Generator

After adding a number of generators to my application's test suite I started noticing that a resource in particular is starting to get flaky. Sometimes it generates valid data but other times it seemingly returns empty data since all validations fail on the resource. The relevent generator code is: 
defmodule Atlas.Generator do
  # ...

  def tracking_entity_type(opts \\ []) do
    tenant = opts[:tenant] || default_tenant()
    module_id = opts[:module_id] || once(:default_module_id, fn -> generate(module()).id end)

    changeset_generator(EntityType, :create,
      defaults: [
        parent_id: nil,
        module_id: module_id,
        name: sequence(:name, &"Entity Type #{&1}"),
        slug: sequence(:slug, &"entity_type_#{&1}"),
        plural_name: sequence(:plural_name, &"Entity Types #{&1}"),
        plural_slug: sequence(:plural_slug, &"entity_types_#{&1}"),
        label_template: "{{ id }}",
        select_template: "{{ id }}",
        icon: :info
      ],
      tenant: tenant,
      overrides: opts,
      authorize?: false
    )
  end

  # ...

  defp default_tenant do
    once(:default_tenant, fn -> generate(client()) end)
  end
end
defmodule Atlas.Generator do
  # ...

  def tracking_entity_type(opts \\ []) do
    tenant = opts[:tenant] || default_tenant()
    module_id = opts[:module_id] || once(:default_module_id, fn -> generate(module()).id end)

    changeset_generator(EntityType, :create,
      defaults: [
        parent_id: nil,
        module_id: module_id,
        name: sequence(:name, &"Entity Type #{&1}"),
        slug: sequence(:slug, &"entity_type_#{&1}"),
        plural_name: sequence(:plural_name, &"Entity Types #{&1}"),
        plural_slug: sequence(:plural_slug, &"entity_types_#{&1}"),
        label_template: "{{ id }}",
        select_template: "{{ id }}",
        icon: :info
      ],
      tenant: tenant,
      overrides: opts,
      authorize?: false
    )
  end

  # ...

  defp default_tenant do
    once(:default_tenant, fn -> generate(client()) end)
  end
end
An example error message is attached (I think due to the size of it). The resource itself is also attached. I have defined a few extensions to abstract away attribution (inserted by and updated by attributes / relationships) and a tree structure using LTREEs. I didn't include it since I have other resources that are working fine with them but I can certainly include them if it would help. Let me know if I need to provide any additional information.

validation_errors.tx...

entity_type.ex

Solution:
I think I solved it for myself. I didn't realize that Generator was supplying lists of empty maps for an argument children which allowed me to generate child entity types at the same time as the parent. Because the maps were empty, they were failing validation. once I started supplying a default value of an empty list, the flakiness seems to have gone away.
Jump to solution
1 Reply
Solution
mylanconnolly
mylanconnolly•2d ago
I think I solved it for myself. I didn't realize that Generator was supplying lists of empty maps for an argument children which allowed me to generate child entity types at the same time as the parent. Because the maps were empty, they were failing validation. once I started supplying a default value of an empty list, the flakiness seems to have gone away.

Did you find this page helpful?