AE
Ash Elixir•3y ago
rohan

Check if an identity exists in a before_action

I'm not quite sure how to accomplish this in Ash: I'm uploading files to GCS and want to make sure if the hash of the file matches one I already have, I don't upload it again. The upload happens in a before_action on create right now. The hash of the file is an identity, and I thought eager checking would prevent the upload but it doesn't seem to. I'd like the create_with_binary to return the original record if it exists (without uploading) or upload and create a new record if necessary. So basically find_or_create. Here's the code so far:
defmodule Scribble.EmailHandler.AudioFile do
use Ash.Resource, data_layer: AshPostgres.DataLayer

postgres do
table "audio_files"
repo(Scribble.Repo)
end

attributes do
uuid_primary_key :id
attribute :hash, :string, allow_nil?: false
attribute :url, :string, allow_nil?: false
end

identities do
identity :hash, [:hash], eager_check_with: Scribble.EmailHandler
end

code_interface do
define_for Scribble.EmailHandler

define :create_with_binary, args: [:audio_file]
end

actions do
defaults [:read]

create :create_with_binary do
argument :audio_file, :map, allow_nil?: false
accept []

change fn changeset, _context ->
Ash.Changeset.before_action(changeset, fn changeset ->
audio_file = Ash.Changeset.get_argument(changeset, :audio_file)
filename = audio_file_hash(audio_file)

{:ok, ^filename} =
Scribble.Recording.store(%{filename: filename, binary: audio_file.binary})

url = Scribble.Recording.url(filename)
Ash.Changeset.change_attributes(changeset, %{url: url, hash: filename})
end)
end
end
end

defp audio_file_hash(%{binary: binary, extension: extension}) do
hash = Scribble.Scribe.audio_hash(binary)
"#{hash}#{extension}"
end
end
defmodule Scribble.EmailHandler.AudioFile do
use Ash.Resource, data_layer: AshPostgres.DataLayer

postgres do
table "audio_files"
repo(Scribble.Repo)
end

attributes do
uuid_primary_key :id
attribute :hash, :string, allow_nil?: false
attribute :url, :string, allow_nil?: false
end

identities do
identity :hash, [:hash], eager_check_with: Scribble.EmailHandler
end

code_interface do
define_for Scribble.EmailHandler

define :create_with_binary, args: [:audio_file]
end

actions do
defaults [:read]

create :create_with_binary do
argument :audio_file, :map, allow_nil?: false
accept []

change fn changeset, _context ->
Ash.Changeset.before_action(changeset, fn changeset ->
audio_file = Ash.Changeset.get_argument(changeset, :audio_file)
filename = audio_file_hash(audio_file)

{:ok, ^filename} =
Scribble.Recording.store(%{filename: filename, binary: audio_file.binary})

url = Scribble.Recording.url(filename)
Ash.Changeset.change_attributes(changeset, %{url: url, hash: filename})
end)
end
end
end

defp audio_file_hash(%{binary: binary, extension: extension}) do
hash = Scribble.Scribe.audio_hash(binary)
"#{hash}#{extension}"
end
end
37 Replies
ZachDaniel
ZachDaniel•3y ago
create :create_with_binary do
upsert? true
upsert_identity :hash

...rest of your action
end
create :create_with_binary do
upsert? true
upsert_identity :hash

...rest of your action
end
I think that might be all you need
rohan
rohanOP•3y ago
i wasn't able to find many docs on the upsert - does that prevent the before_action from running?
ZachDaniel
ZachDaniel•3y ago
eager_check_with will do its work before before_action hooks are run, which is why you probably saw no change on that Nope Ah, I see
rohan
rohanOP•3y ago
i think i might need to manually do a read of the database and then block the rest of the action from executing, returning the thing that was read but not sure how to do that in the before_action
ZachDaniel
ZachDaniel•3y ago
you need to make sure not to upsert if one exists already.
create :create_with_binary do
argument :audio_file, :map, allow_nil?: false
accept []

change fn changeset, _context ->
Ash.Changeset.before_action(changeset, fn changeset ->
audio_file = Ash.Changeset.get_argument(changeset, :audio_file)
filename = audio_file_hash(audio_file)
case <get_by_hash> do
{:ok, result} ->
Ash.Changeset.set_result(changeset, result)
{:error, _} ->
do_the_upload
end
end)
end
end
create :create_with_binary do
argument :audio_file, :map, allow_nil?: false
accept []

change fn changeset, _context ->
Ash.Changeset.before_action(changeset, fn changeset ->
audio_file = Ash.Changeset.get_argument(changeset, :audio_file)
filename = audio_file_hash(audio_file)
case <get_by_hash> do
{:ok, result} ->
Ash.Changeset.set_result(changeset, result)
{:error, _} ->
do_the_upload
end
end)
end
end
Ash.Changeset.set_result lets you have a sort of "conditional" manual action, where sometimes the result is computed by you and other times you let Ash execute the action.
rohan
rohanOP•3y ago
ooh that's really cool haha trying it now silly thing but is there a way to call get on the AudioFile and not the associated API like rather than: Scribble.EmailHandler.get(AudioFile, hash: filename)
ZachDaniel
ZachDaniel•3y ago
Have you looked into the code interface yet?
code_interface do
define_for Your.Api

define :get_by_hash, action: :read, get_by: [:hash], args: [:hash]
end
code_interface do
define_for Your.Api

define :get_by_hash, action: :read, get_by: [:hash], args: [:hash]
end
will get you
Scribble.EmailHandler.get_by_hash(hash)
Scribble.EmailHandler.get_by_hash(hash)
rohan
rohanOP•3y ago
oh yes I've done that but I meant rather than having to use Scribe.EmailHandler.get_by_hash or Scribe.EmailHandler.get I'd want to do : Scribe.EmailHandler.AudioFile.get or something
ZachDaniel
ZachDaniel•3y ago
Oh, interesting
rohan
rohanOP•3y ago
it's a small thing but it feels like an AudioFile should know how to get itself in case I move it somewhere else later
ZachDaniel
ZachDaniel•3y ago
Nothing to do that currently, although you can always do
def get(fields, opts) do
YourApi.get(fields, opts)
end
def get(fields, opts) do
YourApi.get(fields, opts)
end
in your resource One thing you may need to do is lock the record in the database to prevent multiple of these actions happening side by side. Ash doesn't currently support that natively, so you may want to do something like this:
read :get_and_lock_by_hash do
get? true

argument :hash, :string, allow_nil?: false
filter expr(hash == ^arg(:hash))
modify_query &lock_for_update/2
end

...

defp lock_for_update(_, query) do
{:ok, Ecto.Query.lock(query, "FOR UPDATE")}
end

code_interface do
read :get_and_lock_by_hash, args: [:hash]
end
read :get_and_lock_by_hash do
get? true

argument :hash, :string, allow_nil?: false
filter expr(hash == ^arg(:hash))
modify_query &lock_for_update/2
end

...

defp lock_for_update(_, query) do
{:ok, Ecto.Query.lock(query, "FOR UPDATE")}
end

code_interface do
read :get_and_lock_by_hash, args: [:hash]
end
rohan
rohanOP•3y ago
it's a little confusing to me that :
code_interface do
define_for Scribble.EmailHandler

define :create_with_binary, args: [:audio_file]
define :get_by_hash, action: :read, get_by: [:hash]
end
code_interface do
define_for Scribble.EmailHandler

define :create_with_binary, args: [:audio_file]
define :get_by_hash, action: :read, get_by: [:hash]
end
this defines functions on Scribble.EmailHandler.<Resource> when it says define_for Scribble.EmailHandler i like that behavior I just don't quite get the syntax
ZachDaniel
ZachDaniel•3y ago
Yes, define_for is which api the generated functions will call under the hood because all action invocations always go through an api module
rohan
rohanOP•3y ago
ah so that way any extensions in the API apply to that resource
ZachDaniel
ZachDaniel•3y ago
Yep, and there are certain rules like authorization configuration/timeouts and things like that that can be configured at the api level
rohan
rohanOP•3y ago
i'd love a video on how ash works under the hood whenever you get to it šŸ™‚ always helps me understand how to use something properly when I get the mental model behind it
ZachDaniel
ZachDaniel•3y ago
For sure šŸ™‚ We're also working to make that code generally more understandable by using Ash.Flow Ash.Flow can auto generate flow charts for a given operation, and so ideally we can create an Ash.Flow for the various parts of our system, and then explain any given thing by generating the diagram of it or at least allow for a way to see that behavior laid out without having to read the code but that will probably be a while šŸ˜„
rohan
rohanOP•3y ago
this didn't seem to work I'm getting the error that :url is required, :hash is required
ZachDaniel
ZachDaniel•3y ago
oh interesting... See if this helps actually hang on... is there a stacktrace there? There is probably a check we are doing that we shouldn't do if set_result has been used šŸ™‚
rohan
rohanOP•3y ago
change fn changeset, _context ->
Ash.Changeset.before_action(changeset, fn changeset ->
audio_file = Ash.Changeset.get_argument(changeset, :audio_file)
filename = audio_file_hash(audio_file)

case Scribble.EmailHandler.AudioFile.get_by_hash(filename) do
{:ok, audio_file} ->
IO.puts("EXISTS")
IO.inspect(audio_file)
Ash.Changeset.set_result(changeset, audio_file) |> IO.inspect()

{:error, _} ->
{:ok, ^filename} =
Scribble.Recording.store(%{filename: filename, binary: audio_file.binary})

url = Scribble.Recording.url(filename)
Ash.Changeset.change_attributes(changeset, %{url: url, hash: filename})
end
end)
end
change fn changeset, _context ->
Ash.Changeset.before_action(changeset, fn changeset ->
audio_file = Ash.Changeset.get_argument(changeset, :audio_file)
filename = audio_file_hash(audio_file)

case Scribble.EmailHandler.AudioFile.get_by_hash(filename) do
{:ok, audio_file} ->
IO.puts("EXISTS")
IO.inspect(audio_file)
Ash.Changeset.set_result(changeset, audio_file) |> IO.inspect()

{:error, _} ->
{:ok, ^filename} =
Scribble.Recording.store(%{filename: filename, binary: audio_file.binary})

url = Scribble.Recording.url(filename)
Ash.Changeset.change_attributes(changeset, %{url: url, hash: filename})
end
end)
end
that's my code right now no stacktrace it's returning an {:error, %Ash.Error.Invalid...}
ZachDaniel
ZachDaniel•3y ago
can you use the ! version of the function you're calling? should see some stacktraces there
rohan
rohanOP•3y ago
↳ anonymous fn/3 in Ash.Changeset.with_hooks/3, at: lib/ash/changeset/changeset.ex:1573
** (Ash.Error.Invalid) Input Invalid

* attribute url is required
(ash 2.6.29) lib/ash/changeset/changeset.ex:1425: anonymous fn/2 in Ash.Changeset.require_values/4
(elixir 1.14.3) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
(ash 2.6.29) lib/ash/actions/create.ex:383: anonymous fn/8 in Ash.Actions.Create.as_requests/5
(ash 2.6.29) lib/ash/changeset/changeset.ex:1865: Ash.Changeset.run_around_actions/2
(ash 2.6.29) lib/ash/changeset/changeset.ex:1575: anonymous fn/2 in Ash.Changeset.with_hooks/3
(ecto_sql 3.9.2) lib/ecto/adapters/sql.ex:1203: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
(db_connection 2.4.3) lib/db_connection.ex:1611: DBConnection.run_transaction/4
(ash 2.6.29) lib/ash/changeset/changeset.ex:1573: anonymous fn/3 in Ash.Changeset.with_hooks/3
(ash 2.6.29) lib/ash/changeset/changeset.ex:1689: Ash.Changeset.transaction_hooks/2
(ash 2.6.29) lib/ash/actions/create.ex:319: anonymous fn/10 in Ash.Actions.Create.as_requests/5
(ash 2.6.29) lib/ash/engine/request.ex:1048: Ash.Engine.Request.do_try_resolve_local/4
(ash 2.6.29) lib/ash/engine/request.ex:282: Ash.Engine.Request.do_next/1
(ash 2.6.29) lib/ash/engine/request.ex:211: Ash.Engine.Request.next/1
(ash 2.6.29) lib/ash/engine/engine.ex:683: Ash.Engine.advance_request/2
(ash 2.6.29) lib/ash/engine/engine.ex:589: Ash.Engine.fully_advance_request/2
(ash 2.6.29) lib/ash/engine/engine.ex:530: Ash.Engine.do_run_iteration/2
(elixir 1.14.3) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
(ash 2.6.29) lib/ash/engine/engine.ex:271: Ash.Engine.run_to_completion/1
* attribute hash is required
(ash 2.6.29) lib/ash/changeset/changeset.ex:1425: anonymous fn/2 in Ash.Changeset.require_values/4
(elixir 1.14.3) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
(ash 2.6.29) lib/ash/actions/create.ex:383: anonymous fn/8 in Ash.Actions.Create.as_requests/5
(ash 2.6.29) lib/ash/changeset/changeset.ex:1865: Ash.Changeset.run_around_actions/2
(ash 2.6.29) lib/ash/changeset/changeset.ex:1575: anonymous fn/2 in Ash.Changeset.with_hooks/3
(ecto_sql 3.9.2) lib/ecto/adapters/sql.ex:1203: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
(db_connection 2.4.3) lib/db_connection.ex:1611: DBConnection.run_transaction/4
(ash 2.6.29) lib/ash/changeset/changeset.ex:1573: anonymous fn/3 in Ash.Changeset.with_hooks/3
(ash 2.6.29) lib/ash/changeset/changeset.ex:1689: Ash.Changeset.transaction_hooks/2
(ash 2.6.29) lib/ash/actions/create.ex:319: anonymous fn/10 in Ash.Actions.Create.as_requests/5
(ash 2.6.29) lib/ash/engine/request.ex:1048: Ash.Engine.Request.do_try_resolve_local/4
(ash 2.6.29) lib/ash/engine/request.ex:282: Ash.Engine.Request.do_next/1
(ash 2.6.29) lib/ash/engine/request.ex:211: Ash.Engine.Request.next/1
(ash 2.6.29) lib/ash/engine/engine.ex:683: Ash.Engine.advance_request/2
(ash 2.6.29) lib/ash/engine/engine.ex:589: Ash.Engine.fully_advance_request/2
(ash 2.6.29) lib/ash/engine/engine.ex:530: Ash.Engine.do_run_iteration/2
(elixir 1.14.3) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
(ash 2.6.29) lib/ash/engine/engine.ex:271: Ash.Engine.run_to_completion/1
(ash 2.6.29) lib/ash/error/error.ex:463: Ash.Error.choose_error/2
(ash 2.6.29) lib/ash/error/error.ex:218: Ash.Error.to_error_class/2
(ash 2.6.29) lib/ash/actions/create.ex:126: Ash.Actions.Create.do_run/4
(ash 2.6.29) lib/ash/actions/create.ex:38: Ash.Actions.Create.run/4
(ash 2.6.29) lib/ash/api/api.ex:1476: Ash.Api.create!/3
iex:36: (file)
↳ anonymous fn/3 in Ash.Changeset.with_hooks/3, at: lib/ash/changeset/changeset.ex:1573
** (Ash.Error.Invalid) Input Invalid

* attribute url is required
(ash 2.6.29) lib/ash/changeset/changeset.ex:1425: anonymous fn/2 in Ash.Changeset.require_values/4
(elixir 1.14.3) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
(ash 2.6.29) lib/ash/actions/create.ex:383: anonymous fn/8 in Ash.Actions.Create.as_requests/5
(ash 2.6.29) lib/ash/changeset/changeset.ex:1865: Ash.Changeset.run_around_actions/2
(ash 2.6.29) lib/ash/changeset/changeset.ex:1575: anonymous fn/2 in Ash.Changeset.with_hooks/3
(ecto_sql 3.9.2) lib/ecto/adapters/sql.ex:1203: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
(db_connection 2.4.3) lib/db_connection.ex:1611: DBConnection.run_transaction/4
(ash 2.6.29) lib/ash/changeset/changeset.ex:1573: anonymous fn/3 in Ash.Changeset.with_hooks/3
(ash 2.6.29) lib/ash/changeset/changeset.ex:1689: Ash.Changeset.transaction_hooks/2
(ash 2.6.29) lib/ash/actions/create.ex:319: anonymous fn/10 in Ash.Actions.Create.as_requests/5
(ash 2.6.29) lib/ash/engine/request.ex:1048: Ash.Engine.Request.do_try_resolve_local/4
(ash 2.6.29) lib/ash/engine/request.ex:282: Ash.Engine.Request.do_next/1
(ash 2.6.29) lib/ash/engine/request.ex:211: Ash.Engine.Request.next/1
(ash 2.6.29) lib/ash/engine/engine.ex:683: Ash.Engine.advance_request/2
(ash 2.6.29) lib/ash/engine/engine.ex:589: Ash.Engine.fully_advance_request/2
(ash 2.6.29) lib/ash/engine/engine.ex:530: Ash.Engine.do_run_iteration/2
(elixir 1.14.3) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
(ash 2.6.29) lib/ash/engine/engine.ex:271: Ash.Engine.run_to_completion/1
* attribute hash is required
(ash 2.6.29) lib/ash/changeset/changeset.ex:1425: anonymous fn/2 in Ash.Changeset.require_values/4
(elixir 1.14.3) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
(ash 2.6.29) lib/ash/actions/create.ex:383: anonymous fn/8 in Ash.Actions.Create.as_requests/5
(ash 2.6.29) lib/ash/changeset/changeset.ex:1865: Ash.Changeset.run_around_actions/2
(ash 2.6.29) lib/ash/changeset/changeset.ex:1575: anonymous fn/2 in Ash.Changeset.with_hooks/3
(ecto_sql 3.9.2) lib/ecto/adapters/sql.ex:1203: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
(db_connection 2.4.3) lib/db_connection.ex:1611: DBConnection.run_transaction/4
(ash 2.6.29) lib/ash/changeset/changeset.ex:1573: anonymous fn/3 in Ash.Changeset.with_hooks/3
(ash 2.6.29) lib/ash/changeset/changeset.ex:1689: Ash.Changeset.transaction_hooks/2
(ash 2.6.29) lib/ash/actions/create.ex:319: anonymous fn/10 in Ash.Actions.Create.as_requests/5
(ash 2.6.29) lib/ash/engine/request.ex:1048: Ash.Engine.Request.do_try_resolve_local/4
(ash 2.6.29) lib/ash/engine/request.ex:282: Ash.Engine.Request.do_next/1
(ash 2.6.29) lib/ash/engine/request.ex:211: Ash.Engine.Request.next/1
(ash 2.6.29) lib/ash/engine/engine.ex:683: Ash.Engine.advance_request/2
(ash 2.6.29) lib/ash/engine/engine.ex:589: Ash.Engine.fully_advance_request/2
(ash 2.6.29) lib/ash/engine/engine.ex:530: Ash.Engine.do_run_iteration/2
(elixir 1.14.3) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
(ash 2.6.29) lib/ash/engine/engine.ex:271: Ash.Engine.run_to_completion/1
(ash 2.6.29) lib/ash/error/error.ex:463: Ash.Error.choose_error/2
(ash 2.6.29) lib/ash/error/error.ex:218: Ash.Error.to_error_class/2
(ash 2.6.29) lib/ash/actions/create.ex:126: Ash.Actions.Create.do_run/4
(ash 2.6.29) lib/ash/actions/create.ex:38: Ash.Actions.Create.run/4
(ash 2.6.29) lib/ash/api/api.ex:1476: Ash.Api.create!/3
iex:36: (file)
what's interesting is I'm inspecting the changeset after set_result and the data does look like it's nil for some reason
#Ash.Changeset<
api: Scribble.EmailHandler,
action_type: :create,
action: :create_with_binary,
attributes: %{id: "6523c5c6-d18c-4867-8a21-d4ba7805d97a"},
relationships: %{},
arguments: %{audio_file: %{binary: <<255>>, extension: ".mp3"}},
errors: [],
data: #Scribble.EmailHandler.AudioFile<
__meta__: #Ecto.Schema.Metadata<:built, "audio_files">,
id: nil,
hash: nil,
url: nil,
aggregates: %{},
calculations: %{},
__order__: nil,
...
>,
context: %{actor: nil, authorize?: false},
valid?: true
>
#Ash.Changeset<
api: Scribble.EmailHandler,
action_type: :create,
action: :create_with_binary,
attributes: %{id: "6523c5c6-d18c-4867-8a21-d4ba7805d97a"},
relationships: %{},
arguments: %{audio_file: %{binary: <<255>>, extension: ".mp3"}},
errors: [],
data: #Scribble.EmailHandler.AudioFile<
__meta__: #Ecto.Schema.Metadata<:built, "audio_files">,
id: nil,
hash: nil,
url: nil,
aggregates: %{},
calculations: %{},
__order__: nil,
...
>,
context: %{actor: nil, authorize?: false},
valid?: true
>
ZachDaniel
ZachDaniel•3y ago
Yeah, thats fine šŸ™‚ The way we set the result is into the context key which is hidden there well, trimmed down can you try main? I just pushed something up that ought to fix it
rohan
rohanOP•3y ago
yup trying it now hmm did something change between main and 2.6 about how code_interfaces work my case clause is failing
ZachDaniel
ZachDaniel•3y ago
šŸ¤” don't think so
rohan
rohanOP•3y ago
does it no longer return {:ok, ...}
ZachDaniel
ZachDaniel•3y ago
If so its a bug did you add a ! to the call? that will cause it not to return {:ok,
rohan
rohanOP•3y ago
nope
ZachDaniel
ZachDaniel•3y ago
šŸ¤” so get_by_hash is in your code interface, and its returning %Record{}?
rohan
rohanOP•3y ago
yes but on failure it's returning an {:error, _} here's the case:
case Scribble.EmailHandler.AudioFile.get_by_hash(filename) do
{:error, _} ->
{:ok, ^filename} =
Scribble.Recording.store(%{filename: filename, binary: audio_file.binary})

url = Scribble.Recording.url(filename)
Ash.Changeset.change_attributes(changeset, %{url: url, hash: filename})

audio_file ->
Ash.Changeset.set_result(changeset, audio_file)
end
case Scribble.EmailHandler.AudioFile.get_by_hash(filename) do
{:error, _} ->
{:ok, ^filename} =
Scribble.Recording.store(%{filename: filename, binary: audio_file.binary})

url = Scribble.Recording.url(filename)
Ash.Changeset.change_attributes(changeset, %{url: url, hash: filename})

audio_file ->
Ash.Changeset.set_result(changeset, audio_file)
end
^ this works
ZachDaniel
ZachDaniel•3y ago
šŸ¤”
rohan
rohanOP•3y ago
and here's the code interface:
code_interface do
define_for Scribble.EmailHandler

define :create_with_binary, args: [:audio_file]
define :get_by_hash, action: :read, get_by: [:hash]
end
code_interface do
define_for Scribble.EmailHandler

define :create_with_binary, args: [:audio_file]
define :get_by_hash, action: :read, get_by: [:hash]
end
ZachDaniel
ZachDaniel•3y ago
oh, actually are you sure its your case statement? I think its because you need to do Ash.Changeset.set_result(changeset, {:ok, audio_file}) I forgot about that
rohan
rohanOP•3y ago
that worked!
ZachDaniel
ZachDaniel•3y ago
Yeah, so it was actually the case statement in the create function
rohan
rohanOP•3y ago
cool everything's working now šŸ™‚ thanks!
ZachDaniel
ZachDaniel•3y ago
šŸ‘

Did you find this page helpful?