AE
Ash Elixir•3y ago
tommasop#2001

Filter fields to be returned on an Ash.Query and an AshJsonApi route

I'm having a hard time finding out how can I filter returned fields from an Ash.Query this is my code: 
    read :quantities do
      description "Product Quantities available in MMS"

      prepare fn query, _ ->
        query
        |> Ash.Query.select([
          :sku_code,
          :quantity
        ])
      end
    end
    read :quantities do
      description "Product Quantities available in MMS"

      prepare fn query, _ ->
        query
        |> Ash.Query.select([
          :sku_code,
          :quantity
        ])
      end
    end
And I'm supposing it to return only :sku_code and :quantity but it returns all the fields as nil and only the two fields with real data. Is it possible to filter the returned fields? Thanks a lot
25 Replies
ZachDaniel
ZachDaniel•3y ago
So the elixir structus will always contain all fields, but in terms of doing it for AshJsonApi, do you want those fields to be hidden for all requests against that resource? Or just one specific field?
tommasop#2001
tommasop#2001OP•3y ago
Just one specific request
ZachDaniel
ZachDaniel•3y ago
Gotcha. So I don't think we have a way to do that (but the idea is that we would once someone asked :D). The main consideration is that with json:api the idea is that the caller can ask for the fields that they want. fields[type]=sku_code,quantity Are you limiting fields for authorization purposes? Or just to cut out unnecessary information?
tommasop#2001
tommasop#2001OP•3y ago
just to cut info out but I can do it like this fields[type]=sku_code,quantity didn't think about it so what does the Ash.Query.select is used for if I got all the fields back?
ZachDaniel
ZachDaniel•3y ago
Well, select limits the data that comes back from the data layer its an interesting point, though, that we could potentially just infer what fields to show in the api, i.e hide them if they weren't selected instead of showing nil
tommasop#2001
tommasop#2001OP•3y ago
it would be a behavior more similar to other data layers for sure
ZachDaniel
ZachDaniel•3y ago
When calling a read action, you can ask for the "ultimate query" and we could get back and see what was/wasn't selected and hide the fields that weren't.
tommasop#2001
tommasop#2001OP•3y ago
what If I need some fields to be hidden in a json_api request?
ZachDaniel
ZachDaniel•3y ago
We could also just ask each record what was/wasn't selected Well, you can hide fields with private?: true on the attribute
tommasop#2001
tommasop#2001OP•3y ago
but that will work for every request
ZachDaniel
ZachDaniel•3y ago
Yep! So we'd need to add an option to do that. It would be very easy to add. 
index :foo, hide_fields: [...]
index :foo, hide_fields: [...]
tommasop#2001
tommasop#2001OP•3y ago
How can I access this "ultimate query" data?
ZachDaniel
ZachDaniel•3y ago
Well, in the context of AshJsonApi you can't But when calling a read action yourself return_query?: true is an option We could also add a more low level helper like index :foo, renderer: Module I'd rather not do renderer though because that will break open api stuff So I'd say LMK when you need it, we can reason about why you need it, and adding it should be easy Not saying I'd argue with why you need it, just saying that we want to have a concrete use case before deciding how it should work.
tommasop#2001
tommasop#2001OP•3y ago
for me it will be fine to have the exclude_fields option we have several legacy application and we are translating data between them
ZachDaniel
ZachDaniel•3y ago
the use case does matter though, for example, if the client wants to ask for those fields anyway, should they be allowed to? i.e exclude_fields: [:foo] and fields[type]=foo In which case we're really talking about default_fields "foo,bar,baz" whereas exclude_fields would always hide well, probably default_fields [:foo, :bar, :baz]
tommasop#2001
tommasop#2001OP•3y ago
we have a Product resource coming from CRM which is synced with a Magento shop in two steps: 1. The product data itself (which must be stripped down of Ash id, and foreign keys) 2. the product quantity which is updated several times a day and that must have its own endpoint returning [:external_id, :quantity] I can return all the data and let the syncing app do the filtering and cleaning job but it seems awkward to me actually we translate data between several legacy apps so the same thing will be needed for Order and their shipping etc.
ZachDaniel
ZachDaniel•3y ago
defmodule SimpleProduct do
  use Ash.Resource,
    extensions: [AshJsonApi.Resource]

  json_api do
    type "product"
    routes do
      base "/products"
      index :read
    end
  end

  attributes do
    attribute :id, :string, allow_nil?: false
    attribute :external_id, :string
    attribute :quantity, :string
  end

  actions do
    read :read do
      prepare fn query, _ -> 
        Ash.Query.before_action(query, fn query -> 
          data = 
            Product
            |> Api.read!()
            |> Enum.map(&clean/1)

          Ash.DataLayer.Simple.set_data(query, data)
        end)
      end
    end
  end
end
defmodule SimpleProduct do
  use Ash.Resource,
    extensions: [AshJsonApi.Resource]

  json_api do
    type "product"
    routes do
      base "/products"
      index :read
    end
  end

  attributes do
    attribute :id, :string, allow_nil?: false
    attribute :external_id, :string
    attribute :quantity, :string
  end

  actions do
    read :read do
      prepare fn query, _ -> 
        Ash.Query.before_action(query, fn query -> 
          data = 
            Product
            |> Api.read!()
            |> Enum.map(&clean/1)

          Ash.DataLayer.Simple.set_data(query, data)
        end)
      end
    end
  end
end
What about something like that? clean/1 would need to go from Product -> SimpleProduct You can define simpler/more trimmed down versions of your resources and expose those over APIs, and then source their data from your other resources.
tommasop#2001
tommasop#2001OP•3y ago
for my use case it can be used because we will have maximum one trimmed down version per resource but yes even if I have more of them it is an interesting approach
ZachDaniel
ZachDaniel•3y ago
Its definitely more verbose than hide_fields but also creates a clear separation between your primary domain logic and the stuff you're doing to backport. And you can do things like test/interact with those resources, i.e Api.read(SimpleProduct) Is it a read only api?
tommasop#2001
tommasop#2001OP•3y ago
the trimmed down version will always be read only
ZachDaniel
ZachDaniel•3y ago
So I think what you could do is edit this to add an option for exclude_fields that is {:list_of, :atom} https://github.com/ash-project/ash_json_api/blob/main/lib/ash_json_api/resource/resource.ex#L2
GitHub
ash_json_api/resource.ex at main · ash-project/ash_json_api
A JSON:API extension for the Ash Framework. Contribute to ash-project/ash_json_api development by creating an account on GitHub.
ZachDaniel
ZachDaniel•3y ago
And then modify this logic: https://github.com/ash-project/ash_json_api/blob/main/lib/ash_json_api/serializer.ex#L582
GitHub
ash_json_api/serializer.ex at main · ash-project/ash_json_api
A JSON:API extension for the Ash Framework. Contribute to ash-project/ash_json_api development by creating an account on GitHub.
ZachDaniel
ZachDaniel•3y ago
Actually, much simpler if you start off just saying default_fields Then this: 
fields = Map.get(request.fields || %{}, resource) || request.route.default_fields || default_attributes(resource)
fields = Map.get(request.fields || %{}, resource) || request.route.default_fields || default_attributes(resource)
Would do it
tommasop#2001
tommasop#2001OP•3y ago
Thank you so much for the detailed explanation. I'll try to whip up a PR to implement this behavior and will get back to the internal stakeholders to understand if the JSON:API way can be used to avoid workarounds
tommasop#2001
tommasop#2001OP•3y ago

Did you find this page helpful?