mix ash_postgres.gen.resources is slow for large codebases
We're investigating using Ash with our existing codebase, which is quite large. We're looking to generate resources for existing Ecto schemas using mix ash_postgres.gen.resources, but we're experiencing slow performance with that command. It seems others have experienced this also (see https://elixirforum.com/t/integrating-ash-into-an-existing-project/66927/6).
I've added some debug logging in ash_postgres, and it seems that the majority of the time is being spent in Ash.Resource.Igniter.list_resources(igniter):
Generating resources from [OgatApp.Repo]
[PERF] Migrations: false
[PERF] Igniter.compose_task("ash.gen.domain", ...): 14947ms
[PERF] list_resources: 607837ms
[PERF] filter resources with Code.ensure_loaded?: 16ms
[PERF] include_all_elixir_files: 0ms
[PERF] Generating for 1 repo(s)
[PERF] table_specs query: 236ms
[PERF] build_attributes(pupils): 101ms
[PERF] Processing 1 tables
[PERF] add_foreign_keys(pupils): 18ms
[PERF] add_indexes(pupils): 13ms
[PERF] add_check_constraints(pupils): 3ms
[PERF] tables(OgatApp.Repo): 1645ms
[PERF] Spec.tables(OgatApp.Repo): 1657ms
[PERF] Spec.tables for all repos: 1667ms
[PERF] Adding relationships for 1 specs
[PERF] do_add_relationships(OgatApp.Repo, 1 specs): 2ms
[PERF] add_relationships (total): 2ms
[PERF] Spec.add_relationships: 2ms
[PERF] Generating 1 resources
[PERF] build resource template(pupils): 0ms
[PERF] add_resource_reference(pupils): 17100ms
[PERF] create_module(pupils): 173ms
[PERF] table_to_resource(pupils): 17273ms
[PERF] table_to_resource for all specs: 17273ms
[PERF] generate (total): 626795ms
[PERF] AshPostgres.ResourceGenerator.generate(...): 626795ms
[PERF] mix task igniter/1 (total): 641748ms
It looks like this might be scanning all existing files to determine which are resources. Is there anything that can be done to speed this up? Happy to help with a PR.
I've added some debug logging in ash_postgres, and it seems that the majority of the time is being spent in Ash.Resource.Igniter.list_resources(igniter):
Generating resources from [OgatApp.Repo]
[PERF] Migrations: false
[PERF] Igniter.compose_task("ash.gen.domain", ...): 14947ms
[PERF] list_resources: 607837ms
[PERF] filter resources with Code.ensure_loaded?: 16ms
[PERF] include_all_elixir_files: 0ms
[PERF] Generating for 1 repo(s)
[PERF] table_specs query: 236ms
[PERF] build_attributes(pupils): 101ms
[PERF] Processing 1 tables
[PERF] add_foreign_keys(pupils): 18ms
[PERF] add_indexes(pupils): 13ms
[PERF] add_check_constraints(pupils): 3ms
[PERF] tables(OgatApp.Repo): 1645ms
[PERF] Spec.tables(OgatApp.Repo): 1657ms
[PERF] Spec.tables for all repos: 1667ms
[PERF] Adding relationships for 1 specs
[PERF] do_add_relationships(OgatApp.Repo, 1 specs): 2ms
[PERF] add_relationships (total): 2ms
[PERF] Spec.add_relationships: 2ms
[PERF] Generating 1 resources
[PERF] build resource template(pupils): 0ms
[PERF] add_resource_reference(pupils): 17100ms
[PERF] create_module(pupils): 173ms
[PERF] table_to_resource(pupils): 17273ms
[PERF] table_to_resource for all specs: 17273ms
[PERF] generate (total): 626795ms
[PERF] AshPostgres.ResourceGenerator.generate(...): 626795ms
[PERF] mix task igniter/1 (total): 641748ms
It looks like this might be scanning all existing files to determine which are resources. Is there anything that can be done to speed this up? Happy to help with a PR.
