Topics

Ash Elixir•3y ago

Porting `cloak`-managed fields from Ecto schema to Ash

One of the Ecto schemas in my project uses Cloak to encrypt a value to a column on write, then decrypt the column on read and expose that on a virtual field on the schema. For example, the schema has these two fields:

field :encrypted_meetup, GF.Encrypted.Map
field :meetup, :map, virtual: true

field :encrypted_meetup, GF.Encrypted.Map
field :meetup, :map, virtual: true

GF.Encrypted.Map is a very small module tells cloak about the details of encrypting and decrypting the data. cloak will then perform its magic in the background. On database write, encrypted_meetup is expected to be an unencrypted value, and cloak will encrypt that value to the underlying column as a Postgres bytea type. On database read, cloak will decrypt the value, and put it in encrypted_meetup, and it will also cast that value to a struct, and put it in meetup. How should I approach porting this to an Ash resource? Should I also set up two attributes in the Ash resource, just like in the Ecto schema? Should I define a custom Ash type for transparently encrypting and decrypting the value?

GitHub

GitHub - danielberkompas/cloak: Elixir encryption library designed ...

Elixir encryption library designed for Ecto. Contribute to danielberkompas/cloak development by creating an account on GitHub.

32 Replies

ZachDaniel•3y ago

You can see how I do it in ash_hq here: https://github.com/ash-project/ash_hq/blob/main/lib/ash_hq/accounts/resources/user/user.ex

GitHub

ash_hq/user.ex at main · ash-project/ash_hq

The Ash Framework homepage and documentation site. - ash_hq/user.ex at main · ash-project/ash_hq

ZachDaniel•3y ago

The general pattern is a change that encrypts incoming values, and a calculation for the decrypted value

ZachDaniel•3y ago

https://github.com/ash-project/ash_hq/blob/main/lib/ash_hq/accounts/resources/user/user.ex#L237

GitHub

ash_hq/user.ex at main · ash-project/ash_hq

The Ash Framework homepage and documentation site. - ash_hq/user.ex at main · ash-project/ash_hq

ZachDaniel•3y ago

That is the change

ZachDaniel•3y ago

https://github.com/ash-project/ash_hq/blob/main/lib/ash_hq/accounts/resources/user/user.ex#L244

GitHub

ash_hq/user.ex at main · ash-project/ash_hq

The Ash Framework homepage and documentation site. - ash_hq/user.ex at main · ash-project/ash_hq

ZachDaniel•3y ago

and those are the calculations

moxleyOP•3y ago

👏 Nice, I will check these out!

ZachDaniel•3y ago

The benefit of that pattern is that fields are never decrypted unless explicitly asked for, and in calculations you have access to things like the user and tenant so you could pretty easily audit log access to encrypted data, for example

moxleyOP•3y ago

@Zach Daniel I'm still trying to get this to work. I created a test for exercising the database read and decrypt. The test setup uses the Ecto schema to write the record to the database with the encrypted column. Then the test queries the record using Ash:

ash_org = Org |> Ash.Query.filter(id == ^org.id) |> GF.Ash.read_one!()

dbg(ash_org.encrypted_meetup)
dbg(ash_org.meetup)

ash_org = Org |> Ash.Query.filter(id == ^org.id) |> GF.Ash.read_one!()

dbg(ash_org.encrypted_meetup)
dbg(ash_org.meetup)

The resulting decrypted field value is #Ash.NotLoaded<:calculation>, and my Decrypt Calculation module isn't being called. What needs to happen to trigger the calculation? The documentation for calculate/3 says:

To ensure that the necessary fields are selected:

1.) Specifying the select option on a calculation in the resource. 2.) Define a select/2 callback in the calculation module 3.) Set always_select? on the attribute in question

To ensure that the necessary fields are selected:

1.) Specifying the select option on a calculation in the resource. 2.) Define a select/2 callback in the calculation module 3.) Set always_select? on the attribute in question

I'm pretty sure I'm doing #2. I don't understand #1 or #3.

ZachDaniel•3y ago

You need to load those values Org |> Ash.Query.filter(id == ^org.id) |> Ash.Query.load([:encrypted_meetup]) |> GF.Ash.read_one!() they aren't loaded by default You load calculations/aggregates/relationships all using load

moxleyOP•3y ago

load! Okay, today I learn about load.

ZachDaniel•3y ago

And you can do Api.load to load on things that you already have i.e Org |> .... |> GF.Ash.read_one!() |> GF.Ash.load(:calc)

moxleyOP•3y ago

Hmm, I'm not following that. Things I already have?

ZachDaniel•3y ago

Like records you've already retrieved I mean you can load calculations on the query, or after the fact on records

moxleyOP•3y ago

Ok, got it. Already-loaded records, versus something specified inside the query.

ZachDaniel•3y ago

yep!

moxleyOP•3y ago

Okay, I think I'm out of the woods with regards to reading and decrypting. Now focusing on encrypting and writing... The relevant parts of the Org resource:

actions do
  create :create do
    argument :meetup, :map
    change set_attribute(:encrypted_meetup, arg(:meetup))
  end
end

changes do
  change {GF.Ash.Encrypt, fields: [:encrypted_meetup]}
end

actions do
  create :create do
    argument :meetup, :map
    change set_attribute(:encrypted_meetup, arg(:meetup))
  end
end

changes do
  change {GF.Ash.Encrypt, fields: [:encrypted_meetup]}
end

The test:

test "encrypt meetup" do
  attrs = Map.put(@attrs, :meetup, %{group_id: "test-group-id"})

  ash_org =
    Org
    |> Ash.Changeset.for_create(:create, attrs)
    |> GF.Ash.create!()
end

test "encrypt meetup" do
  attrs = Map.put(@attrs, :meetup, %{group_id: "test-group-id"})

  ash_org =
    Org
    |> Ash.Changeset.for_create(:create, attrs)
    |> GF.Ash.create!()
end

Running the test gives this error:

    ** (Ash.Error.Invalid) Input Invalid
     
     * Invalid value provided for encrypted_meetup: is invalid.
     
     %{group_id: "test-group-id"}

    ** (Ash.Error.Invalid) Input Invalid
     
     * Invalid value provided for encrypted_meetup: is invalid.
     
     %{group_id: "test-group-id"}

Unlike your example in lib/ash_hq/accounts/resources/user/user.ex, which is a string both for the encrypted and decrypted values, I have a map for the decrypted value, and a binary for the encrypted value. The most important parts of my Decrypt module are not getting executed.

ZachDaniel•3y ago

Yeah, okay that makes sense so you're going to need something a bit more robust since you can't just set the attribute and then encrypt it and TBH I should probably do something similar

{GF.Ash.Encrypt, fields: [meetup: :encrypted_meetup]}

{GF.Ash.Encrypt, fields: [meetup: :encrypted_meetup]}

moxleyOP•3y ago

The anon function in the Ash.Changeset.before_action() call isn't getting called.

ZachDaniel•3y ago

Yeah, its that change set_attribute/1 because its trying to set a map to a string field, and then expecting your change to encrypt it later but that first part doesn't work So if you do a key value of source -> destination (unencrypted input -> encrypted attribute) you can do

opts[:fields]
|> Enum.reduce(changeset, fn {source, destination}, changeset -> 
  case Ash.Changeset.fetch_argument_or_change(changeset, source) do
    {:ok, value} -> Ash.Changeset.force_change_attribute(changeset, destination, encrypt(value))
    :error ->
      changeset
  end
end}

opts[:fields]
|> Enum.reduce(changeset, fn {source, destination}, changeset -> 
  case Ash.Changeset.fetch_argument_or_change(changeset, source) do
    {:ok, value} -> Ash.Changeset.force_change_attribute(changeset, destination, encrypt(value))
    :error ->
      changeset
  end
end}

then you remove the change set_attribute/1 part

moxleyOP•3y ago

I notice you're not calling Ash.Changeset.before_action(), like you do in your user.ex example. What's going on there?

ZachDaniel•3y ago

you mean in the example I just gave above? Just leaving it out for brevity

moxleyOP•3y ago

Should I use Ash.Changeset.before_action(), like you do in user.ex?

ZachDaniel•3y ago

Yes, you most likely should before_action says "do this when the action is actually run", and ensures that it only happens once, for example Whereas if you just do it in the change callback, a changeset might be validated many times (like in the case of an AshPhoenix.Form)

moxleyOP•3y ago

If I wrap it in Ash.Changeset.before_action(changeset, fn changeset -> ... end), the anonymous function doesn't get called. The Invalid value provided for encrypted_meetup: is invalid. error happens first. If I don't wrap it, eventually Ash.Changeset.force_change_attribute(changeset, destination, new_value) gets called with the correct values, but the error still happens.

ZachDaniel•3y ago

before_action hooks don't get called on invalid changesets Did you remove the change set_attribute ? Something in your action is setting a value for encrypted_meetup And its setting it to an invalid value

moxleyOP•3y ago

No, I didn't remove it. Sorry, I missed that part!

ZachDaniel•3y ago

👍 Changes get called (by default) on valid and invalid changes but before_action hooks are saved until just before the valid action is being executed

moxleyOP•3y ago

It's working!

ZachDaniel•3y ago

so thats why you saw your stuff happening when it wasn't in a before_action hook I mean

moxleyOP•3y ago

Okay. That's starting to make sense.

ZachDaniel•3y ago

The action lifecycle can be a lot to absorb 🙂 but it is laid out here: https://ash-hq.org/docs/guides/ash/latest/topics/actions specifically here: https://ash-hq.org/docs/guides/ash/latest/topics/actions#changesets-for-actions

Ash Elixir Join

1.6KMembers

View on Discord

Did you find this page helpful?