Topics

Ash Elixir•3y ago

Limiting action sort and filter

I'm going through and starting to audit my setup for sorting and filtering. Ideally, it would only be possible to sort and filter on a limited set of attributes/relationships. I can block anything I don't want with filtering_on policies, but I'd like to get it out of my GraphQL schema and JSON:API altogether. Is there a way to define at the action and/or resource level which attributes and relationships are allowed to be sorted and filtered on? If there is a way to do this, is there also a flag that would set these to [] by default globally? I'd prefer to explicitly allow rather than deny. This seems to be in-line with the planned changes for Ash 3.0, re: accept defaulting to [].

47 Replies

\ ឵឵឵OP•3y ago

FWIW I didn't see a policy equivalent of filtering_on for sort, does such a thing exist? I suppose it would be partially covered by selecting, but I'd really rather not have folks sort on columns that aren't indexed, for example.

ZachDaniel•3y ago

A filtering_on does not exist in that way, no. One could be written, but I'd probably avoid it. you can set filterable? false on attributes/aggregates/calculations Although...that probably isn't what you want, come to think of it, as that would prevent it fully at the resource layer. Do you think that would be sufficient? If not, there are some things you can do

graphql do
  ...
  derive_filter? false
end

graphql do
  ...
  derive_filter? false
end

That will hide the auto generated filter logic entirely you'd then add it by adding optional arguments to your actions that are exposed over graphql We don't have a similar thing for ash_json_api but one could be added

\ ឵឵឵OP•3y ago

Ok, I can work with setting filterable? false on my attributes. Does this also apply to relationships? Seems that AshGraphql by default will let me filter all down the relationship, which is fantastic when I need it, but also a risk in the average case. With filterable? false, will it also apply to AshJsonApi?

ZachDaniel•3y ago

It should, yes.

\ ឵឵឵OP•3y ago

Ok, perfect. filterable? and sortable? it is. Thanks mate!

ZachDaniel•3y ago

LMK if you run into any issues on that front, thats an area that I think may need some love, not sure.

\ ឵឵឵OP•3y ago

Will do.

ZachDaniel•3y ago

I think we may actually want to define it at the api layer in reality

\ ឵឵឵OP•3y ago

Brings to mind that it would be nice to set defaults for all of these at the resource and API level.

ZachDaniel•3y ago

graphql do
  filterable [:foo, :bar, :baz]
end

json_api do
  filterable [:foo, :bar, :baz]
end

graphql do
  filterable [:foo, :bar, :baz]
end

json_api do
  filterable [:foo, :bar, :baz]
end

\ ឵឵឵OP•3y ago

Yeah, that would be handy, as well as being able to define it the same way at the action level would be fantastic. That should cover most cases. I think I'll be able to get by defining it directly on the attributes etc. for now, but maybe something for the future. I actually wondered the same thing about limit... Is there a way to enforce query/pagination limits at the API/resource level? It would be fine for actions to override these limits where there's necessity, but it would be great to enforce a global default (and separately maximum, which cannot be overridden, perhaps).

defmodule App.Api do
  default_filterable :all # :none, etc.
  default_sortable :indexed # would be nice, but crosses boundaries
end
    
defmodule App.AllGood do
  use Ash.Resource
  
  filterable [:a, :b, :c, :d]
  sortable [:a, :d, :rel_a, rel_b: [:a, :b, :c]]
  
  actions do
    read :read do
      filterable [:a, :b]
      sortable [:a, :c]
    end
  end
  
  graphql do
    sortable [:a, :rel_a, rel_b: [:c]] # rel_a inherits from its primary read action
    filterable [:a, :b]
  end
  
  filterable do
    # validation error perhaps, if mandated that the be a subset of the
    # resource globally-defined filterables  
    filterable [:a, :e]
  end
end

defmodule App.Api do
  default_filterable :all # :none, etc.
  default_sortable :indexed # would be nice, but crosses boundaries
end
    
defmodule App.AllGood do
  use Ash.Resource
  
  filterable [:a, :b, :c, :d]
  sortable [:a, :d, :rel_a, rel_b: [:a, :b, :c]]
  
  actions do
    read :read do
      filterable [:a, :b]
      sortable [:a, :c]
    end
  end
  
  graphql do
    sortable [:a, :rel_a, rel_b: [:c]] # rel_a inherits from its primary read action
    filterable [:a, :b]
  end
  
  filterable do
    # validation error perhaps, if mandated that the be a subset of the
    # resource globally-defined filterables  
    filterable [:a, :e]
  end
end

In that last case regarding validation error, it would be reasonable to introduce an override: true flag. Similar stuff could be done for limit/max_page_size. Some of this stuff may belong in core/AshGraphql/AshJsonApi, other bits could be defined as a transformer/validator extension. I feel like I'm not that bothered about adding this separately to GQL/JSON though, as long as they are able to understand the limits defined at the API/resource/action level.

ZachDaniel•3y ago

Have you looked into pagination? Ash has native support for pagination that should do the limit logic and supports a max page size I'm not sure if I'm a fan of supporting action-specific filterability, but its possible Although I guess it would be fine I think better to do it 1. at the field level and 2. at the api level

\ ឵឵឵OP•3y ago

Yep, pagination is what I was referring to with max_page_size. What I was thinking was that it would be nice to be able to introduce defaults/limits for these at various scopes in the hierarchy (pagination.max_page_size, filterable?, sortable? and I'd throw allow_nil? in there as well. Actions could raise or lower the limit set at the API or resource level by specifying it explicitly, but would need to set override: true if raising it above the level set at the next scope up. I'm not 100% set on that, would also be reasonable to specify default_ and max_ separately—the ability to set them in more scopes and have them inherited is my main thing. The reason I like the override flag is that it's primarily a validation thing in the end. Mostly I want to be able to make devs working on the project aware that they're doing something they should be careful about if they choose to override the limit set higher up the chain.

ZachDaniel•3y ago

Yeah, I see what you mean. It adds a lot of complexity though, especially overrides

\ ឵឵឵OP•3y ago

allow_nil? is a simpler case, for sure, but it seemed related enough to throw it in the mix 😄 Yeah, I mean how rough is the territory when we get into the compile-time dependencies of this 😂

ZachDaniel•3y ago

There are tools for allow_nil? related things already allow_nil_input [:foo] on a create action and require_attributes [:foo] on an update action

\ ឵឵឵OP•3y ago

For allow_nil? I'm thinking specifically about being able to swap the default at an API/resource level. Tbh there are whole applications where I'd prefer allow_nil?: false be the default. For that case, I was wondering if there was an Ash flag to set it globally already.

ZachDaniel•3y ago

ah, no there isn't overrides at the api level are also effectively impossible the same resource can be in multiple APIs

\ ឵឵឵OP•3y ago

Righto, globally, though? And at the resource level would work as well, shouldn't have dependency issues and you can do the override easily. Since this is a single flag, I believe Spark should already allow you to override it simply by declaring again.

ZachDaniel•3y ago

Yes, we could support globally

\ ឵឵឵OP•3y ago

Ash.set_default_filterable? [] or what have you.

ZachDaniel•3y ago

its only realistic to do something like ash_filterable_by_default to set filterable? false by default

\ ឵឵឵OP•3y ago

Yeah, depends, I tend to name identically-purposed attributes the same, but that works in any case. So the override chain would be Ash global -> Resource -> attribute/action/relationship.

ZachDaniel•3y ago

Potentially. All of these things add to the complexity and I'm not really sold on the gain yet

\ ឵឵឵OP•3y ago

I feel like if we wanted to add filterable [...] and sortable [... to GQL/JSON then they should be subsets.

ZachDaniel•3y ago

like when defining an attribute, you have to decide wether or not it is nillable so whatever the default is, its better if you know it because otherwise you have to hunt it down

\ ឵឵឵OP•3y ago

If we had to pick one, it would be the ability to override more things at the resource level, since that is functionally a global default already if you use base resources.

ZachDaniel•3y ago

like attribute :foo, :type, allow_nil?: ...oh whats the default again? do I have to specify it or not? its better if there is just one framework-wide default

\ ឵឵឵OP•3y ago

The reasoning makes sense. At the same time, we can keep the framework-wide default and it's kind of on you if you decide you have a good enough reason to change it. Moreover, most of the defaults now are fully open, so if you decided to change it, the way things stand, it would only be to make it stricter than it already was, therefore (a) you'd introduce no new risk and (b) you'd know about it pretty quick if you thought it was the other way. Personally, I'd be happy if the framework-wide defaults were require/deny everything, but then I suppose there would be pressure to be able to override it the other direction.

ZachDaniel•3y ago

Yeah, I mean ultimately we had to pick one. If we do other defaults I'd be open to the global one but probably no farther than that. Can be done with Application.compile_env in the DSL definition for attributes

\ ឵឵឵OP•3y ago

Not resource-level default?

defmodule App.Defaults do
  use Ash.Resource

  default_filterable? false
  default_sortable? false
end

defmodule App.Defaults do
  use Ash.Resource

  default_filterable? false
  default_sortable? false
end

Along those lines, I started adding in filterable? false to one of my resources. Seems fine, somewhat concerned that this will disable filtering on that attribute for the resource as a whole, if I'm doing backend workloads that need it. But started adding sortable? false and it doesn't seem to exist.

ZachDaniel•3y ago

right you are I guess we just didn't add that yet but yeah, honestly I think we just need to implement it at the api layers thats the best place for what you want to do I think the simplest win for now would be in the dsl filterable_fields [....] and sortable_fields [...] in graphql and json_api

\ ឵឵឵OP•3y ago

That's the interface that I'd like to have, do you think the starting place might be at the action level, though?

ZachDaniel•3y ago

I don't think so, no. It would actually be harder to implement it at the action level right now

\ ឵឵឵OP•3y ago

I definitely understand the value of limiting them for external interfaces like GQL/JSON.

ZachDaniel•3y ago

the filter builder and sort builder aren't aware of which action they are building for we'd have to modify that code a fair amount and also I think as you start adding more actions you don't want to be copying that list around necessarily.

\ ឵឵឵OP•3y ago

Gotcha. Ok, in my case setting limits for all actions in the interface would definitely work pretty well.

ZachDaniel•3y ago

I think we could do it like this:

graphql do
  filterable_fields [...]

  queries do
    list :foo do
      filterable_fields [...] # this is "action level" sort of
    end
  end
end

graphql do
  filterable_fields [...]

  queries do
    list :foo do
      filterable_fields [...] # this is "action level" sort of
    end
  end
end

\ ឵឵឵OP•3y ago

100%, that's what I'm doing now with 500+ attributes XD

ZachDaniel•3y ago

So step 1 is to add an option to the schema here: https://github.com/ash-project/ash_graphql/blob/main/lib/resource/resource.ex#L226

GitHub

ash_graphql/resource.ex at main · ash-project/ash_graphql

An absinthe backed graphql API extension for the Ash Framework - ash_graphql/resource.ex at main · ash-project/ash_graphql

ZachDaniel•3y ago

that will be of type {:array, :atom}

ZachDaniel•3y ago

and then in the two functions here that produce the filter types: https://github.com/ash-project/ash_graphql/blob/main/lib/resource/resource.ex#L1809

GitHub

ash_graphql/resource.ex at main · ash-project/ash_graphql

An absinthe backed graphql API extension for the Ash Framework - ash_graphql/resource.ex at main · ash-project/ash_graphql

ZachDaniel•3y ago

we'd do something like Enum.filter(&(&1.name in filterable)) (or is_nil(filterable)) ash_json_api may be a bit more complex unfortunately, because we aren't currently doing the same level of filter validation (at the api layer, its still properly vetted as input)

\ ឵឵឵OP•3y ago

Running the check Other things vying for my attention 😄 For the second bit, filtering there it looks like the names are already munged:

["CommentFilterId", "CommentFilterText"]
["CompositePrimaryKeyFilterFirst", "CompositePrimaryKeyFilterSecond"]
["CompositePrimaryKeyNotEncodedFilterFirst",
 "CompositePrimaryKeyNotEncodedFilterSecond"]
["DoubleRelRecursiveFilterId", "DoubleRelRecursiveFilterType",
 "DoubleRelRecursiveFilterThis"]
["DoubleRelFilterId", "DoubleRelFilterDummy"]
["MultitenantTagFilterId", "MultitenantTagFilterName"]
["NonIdPrimaryKeyFilterOther"]
["NoObjectFilterId", "NoObjectFilterName"]
["PostFilterId", "PostFilterText", "PostFilterPublished", "PostFilterFoo",
 "PostFilterStatus", "PostFilterStatusEnum", "PostFilterBest",
 "PostFilterScore", "PostFilterIntegerAsStringInApi", "PostFilterText1",
 "PostFilterText2", "PostFilterVisibility", "PostFilterEnumNewType",
 "PostFilterStringNewType", "PostFilterCreatedAt"]
["RelayTagFilterId", "RelayTagFilterName"]
["TagFilterId", "TagFilterName"]
["UserFilterId", "UserFilterName"]

["CommentFilterId", "CommentFilterText"]
["CompositePrimaryKeyFilterFirst", "CompositePrimaryKeyFilterSecond"]
["CompositePrimaryKeyNotEncodedFilterFirst",
 "CompositePrimaryKeyNotEncodedFilterSecond"]
["DoubleRelRecursiveFilterId", "DoubleRelRecursiveFilterType",
 "DoubleRelRecursiveFilterThis"]
["DoubleRelFilterId", "DoubleRelFilterDummy"]
["MultitenantTagFilterId", "MultitenantTagFilterName"]
["NonIdPrimaryKeyFilterOther"]
["NoObjectFilterId", "NoObjectFilterName"]
["PostFilterId", "PostFilterText", "PostFilterPublished", "PostFilterFoo",
 "PostFilterStatus", "PostFilterStatusEnum", "PostFilterBest",
 "PostFilterScore", "PostFilterIntegerAsStringInApi", "PostFilterText1",
 "PostFilterText2", "PostFilterVisibility", "PostFilterEnumNewType",
 "PostFilterStringNewType", "PostFilterCreatedAt"]
["RelayTagFilterId", "RelayTagFilterName"]
["TagFilterId", "TagFilterName"]
["UserFilterId", "UserFilterName"]

Looks like resource_filter_fields is the one.

\ ឵឵឵OP•3y ago

Not convinced this is correct, but gotta leave it for now: https://github.com/ash-project/ash_graphql/pull/69

GitHub

feat: add filterable_fields to limit generated filters by bcksl ·...

Adds the ability to specify which fields a filter should be made available for: graphql do filterable_fields [:name, :age] end Contributor checklist Bug fixes include regression tests Features...

ZachDaniel•3y ago

yeah, you're right I pointed you at the wrong place

\ ឵឵឵OP•3y ago

Under a bit of a time crunch lately, finally got a chance 😄

Ash Elixir Join

1.6KMembers

View on Discord

Did you find this page helpful?