GenkitG
Genkit•5mo ago
dry-scarlet

Hi, I'm using Genkit and Ollama with

Hi, I'm using Genkit and Ollama with different models.

For my situation I need structured output. I tried getting structured output from different models and none have been consistent for me, some better than others.

In one test, out of the 1008 prompts to 'ollama/qwen3:30b' 262 failed to meet my output structure.

My expected output is simple:
// Model's options
const StigmaOptionsEnumA = z.enum(["Not at all likely", "Not very likely", "Somewhat likely", "Very likely", "Do not know"]);
const StigmaOptionsEnumB = z.enum(["Definitely unwilling", "Probably unwilling", "Probably willing", "Definitely willing", "Do not know"]);

// Output schema for a prompt response
export const StigmaSchema = z.object({
    selection: z.union([StigmaOptionsEnumA, StigmaOptionsEnumB]),
});


The flow is very simple as well:
export const stigmaGenerateFlow = ai.defineFlow(
    {
        name: 'stigmaGenerateFlow',
        inputSchema: StigmaInputSchema,
        outputSchema: StigmaSchema,
    },
    async (input) => {
        // Compose the prompt for the model
        const prompt = input.prompt;

        // Call the model
        const { output } = await ai.generate({
            config: { temperature: 0.0 },
            model: 'ollama/qwen3:30b',
            system: steelManPrompt,
            prompt: prompt,
            output: { schema: StigmaSchema },
        });

        if (!output) throw new Error('Failed to generate prompt response');
        return output;
    }
);


Am I doing something wrong? Any help or suggestions is appreciated.
Was this page helpful?