Ash Elixir•2y ago

AshUUID: extension for using UUID v4 and v7, with encoding and prefixing support

Hi there, already released a new Ash extension for enhancing Ash UUID fields support. AshUUID allows usage for UUID v4 and the new v7 draft (official paper: https://www.ietf.org/archive/id/draft-peabody-dispatch-new-uuid-format-04.html#name-uuid-version-7, more about: https://blog.devgenius.io/analyzing-new-unique-identifier-formats-uuidv6-uuidv7-and-uuidv8-d6cc5cd7391a). You can use public shortened version of UUIDs through base62 encoding while keeping them stored as native raw uuid column on Postgres (with all the indexing / querying benefits). AshUUID allows using public shortened version with prefix at the beginning (configurable, default to resource name) like Stripe does for better APIs (more about: https://dev.to/stripe/designing-apis-for-humans-object-ids-3o5a). New AshPostgres feature (https://github.com/ash-project/ash_postgres/pull/162) allows Ash extensions to easily ship Postgres functions extensions through the new AshPostgres.CustomExtension behaviour (example: https://github.com/zoonect-oss/ash_uuid/blob/main/lib/ash_uuid/postgres_extension.ex). With AshPostgres v1.3.41 release, AshUUID provides Postgres-side UUIDv7 generation (through uuid_generate_v7 sql function) for better db integrity. AshUUID adoption: - add {:ash_uuid, "~> 0.2"} to your mix.exs project deps; - add AshUUID.PostgresExtension to your app Repo's installed_extensions and set AshUUID config migration_default?: true if Postgres-side UUIDs generation is needed; - use the extension in your resources use Ash.Resource, data_layer: AshPostgres.DataLayer, extensions: [AshUUID] - simply use that for your fields uuid_attribute :id v0.2.2 is released on Hex: feel free to open issues/PRs or to reply to this thread, feedbacks are welcome 🙂 Happy for my first contribution to the Ash community 💪🏼 https://github.com/zoonect-oss/ash_uuid

72 Replies

jart•2y ago

This is awesome work!

Blibs•2y ago

Great extension man! Quick question, I disabled prefixes globally, but when relationships are loaded, they all come with "id_" as the prefix, how can I remove that?

moisselaOP•2y ago

Thank you! Try to force recompilation of the extension with: mix deps.compile ash_uuid --force

Blibs•2y ago

Nah, it still shows up as "id_" for every relationship (belongs_to) that I load when reading a resource

moisselaOP•2y ago

mmm ok @Blibs: I'll check and try to fix that as soon as possible. Could you open an issue on GH for this? @Blibs I did a quick test on an open project using ash_uuid and it seems to work fine 🤷🏼‍♂️ I realized this could probably be something not well described in the readme: the configuration is described like this config :ash_uuid, :ash_uuid, prefixed?: false but the first :ash_uuid should match your project's application otp name, for example config :example_app, :ash_uuid, prefixed?: false

Blibs•2y ago

Ah, yes, I'm aware of that, it did confuse me the first time tbh, but I'm using it the correct way now:

config :feedback_cupcake, :ash_uuid,
  version: 7,
  encoded?: true,
  prefixed?: false,
  migration_default?: true

config :feedback_cupcake, :ash_uuid,
  version: 7,
  encoded?: true,
  prefixed?: false,
  migration_default?: true

But I still have the problem above. I will try to create a small example that triggers the issue and will create a new issue in GH (I already created one there about resources that have relationships it themselves btw).

moisselaOP•2y ago

thanks! I'll check both together

Blibs•2y ago

@moissela https://github.com/zoonect-oss/ash_uuid/issues/2

moisselaOP•2y ago

Hi @Blibs I've released v0.3.0 fixing both 🤞🏼

Blibs•2y ago

Amazing! I will be testing it right now 🚀 @moissela I just tested it, resources with relationships with themselves seems to be working great, but many_tomany relationships still returns the ids with the default prefix (`id`). Both when creating the resource or when retrieving it (like the example I gave in the GH issue ticket)

Blibs•2y ago

moisselaOP•2y ago

Have you added AshUUID like I wrote here https://github.com/zoonect-oss/ash_uuid/issues/2#issuecomment-1675939636?

You've to include AshUUID extension also in the BlibsBlobs resource to fix.

Blibs•2y ago

Ah.. I totally missed that comment 🤦‍♂️ Working great after adding AshUUID in the resource. Thanks again for the help and great extension! Quick question, seems like the extension doesn't work when a resource doesn't have AshPostgres.Datalayer data_layer, is that by design?

moisselaOP•2y ago

No, not by design 😅 I'll check that too

Blibs•2y ago

I can create a new issue ticket in GH if you prefer

moisselaOP•2y ago

Yep, it would be great

Blibs•2y ago

There you go: https://github.com/zoonect-oss/ash_uuid/issues/3

moisselaOP•2y ago

Thanks, I'll check that as soon as possible Hi @Blibs, v0.4.0 released with volatile and embedded resources support 😉

Blibs•2y ago

Amazing, working like a charm! Hey @moissela I believe I found another bug when using AshUUID with embedded resources, I created a ticket describing the issue: https://github.com/zoonect-oss/ash_uuid/issues/4

moisselaOP•2y ago

Hi @Blibs, v.0.5.0 just released should fix that 🙂

kernel•2y ago

I'm having the occasional issue

kernel•2y ago

this only happening when I try and load something related on the Queue

kernel•2y ago

i.e:

kernel•2y ago

and get_by_id is just a code_interface with get_by: [:id] it seems to think it's initially in integer format 🤔

kernel•2y ago

tags are a many_to_many on queue

kernel•2y ago

hmm it could be my UUID's aren't actually v7 uuids I might have missed some data conversion

kernel•2y ago

🤡 how the hell do I have V2 UUID's in my DB 😆

kernel•2y ago

okay all the data is v7 now

kernel•2y ago

but now I get this

kernel•2y ago

because it seems to think the prefix should be id, instead of queue

kernel•2y ago

feels like Ash or whatever doesn't look at the prefix which is set automatically on the Resource when trying to build the belongs_to relationship

kernel•2y ago

right, so the 'fix' for this is to do this

kernel•2y ago

add the constraints to the uuid arg so that ash_uuid knows what it is expecting, makes me think maybe we shouldn't fail so hard when it's an unknown prefix hope you all enjoyed being my rubber ducks 🙂 so I've set prefixed? to false, as I don't really need it, but I'm still getting something trying to be prefixed, I've deleted the build dir numerous times so no idea why something is still trying to add prefixes I'm trying to delete a related item - it had worked before adding ash uuid, but now I see that its trying to add an `id` prefix infront of the id for the changeset - so the changeset fails as it will create a new record instead of deleting what there is

kernel•2y ago

:thinkies:

kernel•2y ago

thats the id from my event handler at the top, and then the changeset contains id_ :thinkies:

kernel•2y ago

need to add prefixed? false on the constraint, I swear it's not reading my config.exs defaults

kernel•2y ago

okay this whole thing is tripping hard I have prefixed?: false on my app config, but occasionally cast_input/2 still gets called with prefix set to "id" and prefixed? set to true @moissela what's the recommended way forward for this then? all I can think of is modifying the cast_input to check the resource's definition I don't think it does that currently and Ash just sets the defaults on it I honestly feel like it's a bug with Ash, as it's only in very specific cases

kernel•2y ago

these are just the defaults from the type, not the configured types on the app so is this a fix in ash or in this? @Zach Daniel pinging you for input as this probably should have been in a support thread 😬

moisselaOP•2y ago

Hi @kernel, I can't well understand what is the current issue by your messages, if it's one or plus related issues and if it concern to AshUUID (I bet on that 😅) or Ash. Can you put together some code to reproduce the bug and submit an issue on github so I can try to help? Have you tried to run a mix deps.compile ash --force after editing the config.exs file? Sometimes it helps 🤷🏼‍♂️ Here it should be argument :tag_id, AshUUID.UUID, allon_nil?: false I think, because :uuid is the Ash uuid standard type

ZachDaniel•2y ago

Yeah, a small reproduction would be the next step here 🙂 And to make sure you’re on the latest ash and ash postgres

kernel•2y ago

I'll summarise here 1) I override :uuid with AshUUID.UUID (although this wouldn't affect what I see as the 'bug') 2) when I have an action argument (and I'll assume elsewhere) that uses :uuid (or AshUUID.UUID), the action gets built with the default constraints from AshUUID.UUID instead of the constraints that I define in the config.exs file 3) this means I need to explicitly override the constraints on each argument to disable the prefixes - otherwise AshUUID will add a prefix etc to the ID, and things will start acting weird I think 2 is just due to how Ash works, I don't think there is a way for it to know what the constraints should be in an agnostic way 4) other than the bug described in #2, I have had issues at times where AshUUID.UUID is too strict with parsing the ids - i.e: I have a read action that wants an ID which I'll use to load a resource which is prefixed with a custom prefix, obviously there is no way for anything to know what I mean to use that id for, but AshUUID will explode - because by default it wants "id" prefix, and I might pass it a "fish" prefix, I don't necessarily care about what the prefix is, all I want is for the type to drop the prefix and decode the encoded UUID to get around #4 you need to be explicit, I don't really consider it a bug, #2 I consider being a bug, if there is a default set which gets applied to the uuid_attribute macro, then I'd expect that default to be used elsewhere AshUUID is used I'm always on the latest 😜

ZachDaniel•2y ago

@kernel hat do you mean "constraints that you define in the config.exs file"? I haven't used AshUUID myself yet, so maybe I'm missing something

kernel•2y ago

in the config.exs you set some defaults, i.e: enable prefixing, enabled encoding, what version of uuid to use. these defaults get used in the uuid_attribute macro, but not in anything else that uses AshUUID.UUID

config :vmx, :ash_uuid,
  version: 7,
  encoded?: true,
  prefixed?: false,
  migration_default?: true

config :vmx, :ash_uuid,
  version: 7,
  encoded?: true,
  prefixed?: false,
  migration_default?: true

and this is the action argument not respecting the setup I have in config.exs

moisselaOP•2y ago

I think that I can add an uuid_argument macro for using configured defaults while keeping that overridable also in arguments

kernel•2y ago

🙏🏿 would be amazing, I looked through ash source code and didn't see a way to hook it easily

moisselaOP•2y ago

yes, without another macro it could be difficult I will try to fix 1) 2) 3) with a new uuid_argument macro tomorrow and I will add a new "strict: true | false" config for 4) both as default and as override

ZachDaniel•2y ago

We should talk about how this is being done and where those config values are used If you pull those in and make them defaults in the constraints function on the type, then you can apply them for any usage of the type

kernel•2y ago

I was messing about with that but I'm honestly not very good at macro / precompilation type stuff currently the constraints function just returns a nimble opts stuct so I wasn't sure of how you'd make that dynamic based on application env 🙂

ZachDaniel•2y ago

You could set the defaults to the values from the config You'd do something like thisL

def constraints() do
  [
    version: [
      ...,
      default: get_config(:version, ...)
    ]
  ]
end

def constraints() do
  [
    version: [
      ...,
      default: get_config(:version, ...)
    ]
  ]
end

Then each can be ovverriden per type, but will take the defaults from the config

moisselaOP•2y ago

@Zach Daniel Is there already an example of configurable extension so that I can take a look at code? which module the get_config fn came from? is your only an example?

kernel•2y ago

only as an example

moisselaOP•2y ago

ah no! you're referencing mine AshUUID.Config.get_config/1 @Zach Daniel , right?

kernel•2y ago

nope

moisselaOP•2y ago

so, based on your advice @Zach Daniel I think I now understand something that I didn't understand when I developed AshUUID and I would like you to confirm it for me Ash will call CustomType.constraints/0 for getting constraints defaults and merge them with user's overrides? If so, yes @kernel : I can change constraints/0 for using configured defaults

kernel•2y ago

I have this so far

  @impl true
  def constraints() do
    Enum.map(@constraints, fn {key, value} ->
      {key, Keyword.replace(value, :default, AshUUID.Config.get_config([]) |> Map.get(key, ""))}
    end)
  end

  @impl true
  def constraints() do
    Enum.map(@constraints, fn {key, value} ->
      {key, Keyword.replace(value, :default, AshUUID.Config.get_config([]) |> Map.get(key, ""))}
    end)
  end

just playing with it I think we could make prefix nillable also either nil or string

moisselaOP•2y ago

@kernel can you submit an issue with a small resource and an action with an AshUUID argument, describing your actual problem? So that I've a reproducible example of your case that I can add to tests I will try to release a fix for that (and the new strict mode for 4) today

kernel•2y ago

I won't have time to do it for a while unfortunately 😦 have just tested this snippet of code above and it works I can make an issue with some basic code in it but won't be able to do set up test cases etc i.e: config.ex .... resource.ex ..... expected output ..... actual output ..... okay snippet doesn't fix #4 , but it fixes #2

moisselaOP•2y ago

I think we can remove the global configured prefix default because it make no sense. If a project needs prefixed ids is because that ids should clarify which resource belongs to, so having a global unique prefix make no sense.

kernel•2y ago

yup, especially considering you already precompute it when using the macro I've never seen an "id_" as everything has a pregenerated prefix already

moisselaOP•2y ago

For the issue I need only the example case, I'll take care of test cases on myself yes, per-resource prefix default generation through macro came next the global configured default that was present from beginning and now I think we can clear that config 🙂 I think #4 will be resolved through removing the global configured prefix and the new "non-strict mode" config that I'll add, can you fill an issue also for #4 ?

kernel•2y ago

yup

moisselaOP•2y ago

thank you! I've some time to work on AshUUID this afternoon (italian time zone)

kernel•2y ago

I don't think #4 is a bug to be honest I think #4 can be left alone (apart from removing the global configured prefix) because if I am using prefixes then I should be explicit with my actions to say what type of resource it's expecting right

moisselaOP•2y ago

Ok, I agree Hi @kernel and sorry for the delay, I've been very busy. Just released v0.6.0 with configurable defaults on standard arguments, new uuid_argument macro (for prefixed mode when automatic prefix based on resource name is needed), some bug fixes and the new deactivable strict mode. Let me know 😉

kernel•2y ago

installed it and will play with it tonight'ish

Gaming

Programming

AshUUID: extension for using UUID v4 and v7, with encoding and prefixing support

Did you find this page helpful?