C#C
C#•3y ago
ægteemil

❔ streaming an IAsyncEnumerable<string> from complex type

hey. i have a minimal api that looks like this:
public static async IAsyncEnumerable<string> AskOpenAi(
    [FromQuery] string question,
    [FromServices] IOpenAIService openAiService)
{
    var completionResult = openAiService.ChatCompletion.CreateCompletionAsStream(new ChatCompletionCreateRequest
    {
        Messages = new List<ChatMessage>
        {
            new(StaticValues.ChatMessageRoles.System, "You are a helpful assistant."),
            new(StaticValues.ChatMessageRoles.User, question),
        },
        Model = Models.ChatGpt3_5Turbo,
        MaxTokens = 150
    });

    await foreach (var completion in completionResult)
        yield return completion.Choices.First().Message.Content;
}

it's registered like this:
app.MapGet("", AskOpenAiEndpoint.AskOpenAi);

but whenever i visit the url, the answer is not being streamed but rather returend as one large string. i tried making a dummy endpoint that works like expected, but im unsure how to modify the above code so it works. this is the test that worked for me:
public static async IAsyncEnumerable<string> Test()
{
    var values = new[] { "value1", "value2", "value3", "value4" };
    foreach (var item in values)
    {
        await Task.Delay(1000);
        yield return item;
    }
}
Was this page helpful?