How to Stream API Responses with Effect's HttpApiBuilder and AiLanguageModel

Hi! Just getting started with Effect. I have an API I've built using HttpApiBuilder, but I want one of my endpoints to return the result of an LLM call (i.e.

AiLanguageModel.generateText

AiLanguageModel.generateText

AiLanguageModel.generateText

AiLanguageModel.generateText). I'd like the response to stream back to the client. I'm not sure how to go about defining my API schema to indicate the result is a stream.

// How to make this expect a streaming result?
export class AdvisorResult extends Schema.Class<AdvisorResult>("AdvisorResult")({
    text: Schema.Array(Schema.String)
}) { }

// How to make this expect a streaming result?
export class AdvisorResult extends Schema.Class<AdvisorResult>("AdvisorResult")({
    text: Schema.Array(Schema.String)
}) { }

// How to make this expect a streaming result?
export class AdvisorResult extends Schema.Class<AdvisorResult>("AdvisorResult")({
    text: Schema.Array(Schema.String)
}) { }

// How to make this expect a streaming result?
export class AdvisorResult extends Schema.Class<AdvisorResult>("AdvisorResult")({
    text: Schema.Array(Schema.String)
}) { }

Is there anything else I have to do in my handler aside from returning an AiResponse to get this to work correctly?

No replies yetBe the first to reply to this messageJoin

How to Stream API Responses with Effect's HttpApiBuilder and AiLanguageModel

Similar Threads

Similar Threads

Similar Threads