Ash Elixir•4w ago

Is it possible to use LangChain.NativeTools?

I tried creating the following resource to build a simple lookup prompt action. The idea is that someone can put some search term into it and have the LLM look up the NPI registry information using a Google search. The LangChain docs show a usage example here and I tried recreating it with an Ash resource here:

defmodule Prism.NPI.Lookup do
  use Ash.Resource, extensions: [AshAi], domain: Prism.NPI

  actions do
    action :find_person, Prism.NPI.Person do
      description "Look up a person's NPI information for a given query phrase (intended to be a name)"

      argument :query, :string do
        allow_nil? false
      end

      run prompt(
            LangChain.ChatModels.ChatGoogleAI.new!(%{model: "gemini-2.0-flash"}),
            tools: [
              LangChain.NativeTool.new!(%{name: "google_search", configuration: %{}})
            ],
            prompt: """
            Search for NPPES NPI results for the following physician: <%= @input.arguments.query %>

            NPPES NPI information is primarily found on the following website: https://npiregistry.cms.hhs.gov
            """
          )
    end
  end
end

defmodule Prism.NPI.Lookup do
  use Ash.Resource, extensions: [AshAi], domain: Prism.NPI

  actions do
    action :find_person, Prism.NPI.Person do
      description "Look up a person's NPI information for a given query phrase (intended to be a name)"

      argument :query, :string do
        allow_nil? false
      end

      run prompt(
            LangChain.ChatModels.ChatGoogleAI.new!(%{model: "gemini-2.0-flash"}),
            tools: [
              LangChain.NativeTool.new!(%{name: "google_search", configuration: %{}})
            ],
            prompt: """
            Search for NPPES NPI results for the following physician: <%= @input.arguments.query %>

            NPPES NPI information is primarily found on the following website: https://npiregistry.cms.hhs.gov
            """
          )
    end
  end
end

This seems to fail the Spark validation here. Is there a way we can support something like this or am I just missing how to do it? I'd be happy to work on adding it but I'm not sure if this is something you'd want to support and I'm not super familiar with defining Spark schemas (though I'd be happy to try!). Thanks!

GitHub

ash_ai/lib/ash_ai.ex at main · ash-project/ash_ai

Structured outputs, vectorization and tool calling for your Ash application - ash-project/ash_ai

Solution:

It's slightly more verbose but converting it to a generic action wasn't so bad: ```elixir defmodule Prism.NPI.Lookup do use Ash.Resource, extensions: [AshAi], domain: Prism.NPI...

Jump to solution

4 Replies

barnabasj•4w ago

Looking at the code, tools only takes a list of atoms atm. Could you open an issue on gh with a feature request please. In the meantime, you can just call Langchain yourself in a generic action instead of going through prompt

Mylan ConnollyOP•4w ago

Thanks, I'll open a feature request and refactor to a generic action for now

Solution

Mylan Connolly•4w ago

It's slightly more verbose but converting it to a generic action wasn't so bad:

defmodule Prism.NPI.Lookup do
  use Ash.Resource, extensions: [AshAi], domain: Prism.NPI

  alias LangChain.Chains.LLMChain
  alias LangChain.Message
  alias LangChain.NativeTool
  alias LangChain.ChatModels.ChatGoogleAI
  alias LangChain.MessageProcessors.JsonProcessor

  actions do
    action :find_person, Prism.NPI.Person do
      description "Look up a person's NPI information for a given query phrase (intended to be a name)"

      argument :query, :string do
        allow_nil? false
      end

      run fn input, _context ->
        model = ChatGoogleAI.new!(%{temperature: 0, stream: false, model: "gemini-2.0-flash"})
        query = input.arguments.query

        {:ok, updated_chain} =
          %{llm: model, verbose: false, stream: false}
          |> LLMChain.new!()
          |> LLMChain.add_message(
            Message.new_user!("""
            Search for NPPES NPI results for the following physician: #{query}

            NPPES NPI information is primarily found on the following website: https://npiregistry.cms.hhs.gov

            IMPORTANT INSTRUCTIONS:
            You MUST respond with valid JSON that matches the following schema:

            ``json
            {
              "id": "1234567890",
              "last_name": "DOE",
              "first_name": "JOHN",
              "primary_fax_number": "+14075981501",
              "taxonomy": ["taxonomy"]
            }
            ``
            """)
          )
          |> LLMChain.add_tools(NativeTool.new!(%{name: "google_search", configuration: %{}}))
          |> LLMChain.message_processors([JsonProcessor.new!(~r/``json(.*?)``/s)])
          |> LLMChain.run()

        case updated_chain do
          %{last_message: %{processed_content: json}} ->
            Ash.Type.cast_input(Prism.NPI.Person, json)

          _ ->
            {:ok, nil}
        end
      end
    end
  end
end

defmodule Prism.NPI.Lookup do
  use Ash.Resource, extensions: [AshAi], domain: Prism.NPI

  alias LangChain.Chains.LLMChain
  alias LangChain.Message
  alias LangChain.NativeTool
  alias LangChain.ChatModels.ChatGoogleAI
  alias LangChain.MessageProcessors.JsonProcessor

  actions do
    action :find_person, Prism.NPI.Person do
      description "Look up a person's NPI information for a given query phrase (intended to be a name)"

      argument :query, :string do
        allow_nil? false
      end

      run fn input, _context ->
        model = ChatGoogleAI.new!(%{temperature: 0, stream: false, model: "gemini-2.0-flash"})
        query = input.arguments.query

        {:ok, updated_chain} =
          %{llm: model, verbose: false, stream: false}
          |> LLMChain.new!()
          |> LLMChain.add_message(
            Message.new_user!("""
            Search for NPPES NPI results for the following physician: #{query}

            NPPES NPI information is primarily found on the following website: https://npiregistry.cms.hhs.gov

            IMPORTANT INSTRUCTIONS:
            You MUST respond with valid JSON that matches the following schema:

            ``json
            {
              "id": "1234567890",
              "last_name": "DOE",
              "first_name": "JOHN",
              "primary_fax_number": "+14075981501",
              "taxonomy": ["taxonomy"]
            }
            ``
            """)
          )
          |> LLMChain.add_tools(NativeTool.new!(%{name: "google_search", configuration: %{}}))
          |> LLMChain.message_processors([JsonProcessor.new!(~r/``json(.*?)``/s)])
          |> LLMChain.run()

        case updated_chain do
          %{last_message: %{processed_content: json}} ->
            Ash.Type.cast_input(Prism.NPI.Person, json)

          _ ->
            {:ok, nil}
        end
      end
    end
  end
end

Mylan ConnollyOP•4w ago

I had to replace the markdown code fences with double backticks from triple backticks so that the Markdown rendering isn't confused (you'd replace them with triple backticks in the actual code), but otherwise this example should work with Gemini models, at least. I'll mark this as the solution for now since it works. Thanks for the help Barnabas!

Gaming

Programming

Is it possible to use LangChain.NativeTools?

Did you find this page helpful?