Ash Framework•4mo ago

AshAI tool calling: framework does not automatically run the tool?

I'm working through my first tool calling experience in a LiveBook as a learning exercise. AshAI is adding a %LangChain.Function{} to the chain's tools list, and the LLM responds with :assistant role Message containing the %LangChain.Message.ToolCall{} 🎉 I suppose I expected the magic to continue, that the framework would intercept that ToolCall, automatically run the tool, and then respond with a :tool result message to continue the chain. I know I can write the logic myself to do all that, but I'm here to ask a) did I miss something and this can actually happen or b) what are the considerations that would make this impossible as a feature request?

Solution:

Yeah, LLMChain.run takes a mode option

Jump to solution

7 Replies

ZachDaniel•4mo ago

That is how it's supposed to work. What does your code look like?

Mike SchellOP•4mo ago

Excited that I will have less code to write once I know how 😉

ash_ai_tool_calling....

ZachDaniel•4mo ago

I can't actually open that on my phone 😂 No apps to open livemd files Can I just see the snippet where you're invoking langchain? I think you just need the "while needs response" logic I forget the function. If you run Ash AI's chat generator you can see an example

Mike SchellOP•4mo ago

alias LangChain.ChatModels.ChatOpenAI
alias LangChain.Chains.LLMChain
alias LangChain.Message

# Build the model
model = ChatOpenAI.new!(%{model: "gpt-4"})

# Start the chain
chain =
  LLMChain.new!(%{llm: model})
  |> LLMChain.add_message(Message.new_user!("What is the square of 8?"))
  |> AshAi.setup_ash_ai(otp_app: :poc_livebook, tools: [:square])

{:ok, result_chain} = LLMChain.run(chain)
result_chain

alias LangChain.ChatModels.ChatOpenAI
alias LangChain.Chains.LLMChain
alias LangChain.Message

# Build the model
model = ChatOpenAI.new!(%{model: "gpt-4"})

# Start the chain
chain =
  LLMChain.new!(%{llm: model})
  |> LLMChain.add_message(Message.new_user!("What is the square of 8?"))
  |> AshAi.setup_ash_ai(otp_app: :poc_livebook, tools: [:square])

{:ok, result_chain} = LLMChain.run(chain)
result_chain

I'll run the generator again, been a couple of weeks and I missed the example in the generated code.

Solution

ZachDaniel•4mo ago

Yeah, LLMChain.run takes a mode option

ZachDaniel•4mo ago

Take a look at the langchain docs for it

Mike SchellOP•4mo ago

Got it. Thanks for pointing the way. For others:

LLMChain.run(chain, mode: :while_needs_response)

LLMChain.run(chain, mode: :while_needs_response)

Gaming

Programming

AshAI tool calling: framework does not automatically run the tool?

Did you find this page helpful?