GenkitG
Genkit•12mo ago
uncertain-scarlet

Context docs not working in prompts

Hi,

There is an issue with the docs parameter on dotprompt calls, it seems like it just isn't passed into the underlying generate calls, this can be seen in the development tools fairly easily. The retriever is executed but the subsequent generate call doesn't have any docs field. (I didn't look under the hood, this is just what shows up in the genkit developer UI)

    const chatbotPrompt= promptRef('chatbot');
    
    ...
    
    const docs = useRetrieverToGetRelevantDocs(question);

    // this works
    const llmResponse1 = await ai.generate({
      docs,
      model,
      prompt: `You are a chatbot responding to... \nAnswer the following question:\n      \n${question}`,
    });

    // this does not
    const llmResponse2 = await chatbotPrompt(
      {
        question,
      },
      {
        model,
        docs,
      }
    );


There is an issue on github, but it doesn't look like its seen much attention and its labelled [Docs] as in documentation.. https://github.com/firebase/genkit/issues/1454. This seems incorrect as it is an issue in the code somewhere, the docs param isn't getting passed onto a lower function call somewhere. To get around this you need to merge the docs into your prompt input and rework your .prompt file to manually insert the docs into the body of the instructions. This seems incorrect that you would have to change the way you supply data from retrievers to your prompts depending on if they are in files or in code.
Was this page helpful?