Effect CommunityEC
Effect Community•12mo ago•
3 replies
Jambudipa

Streaming from HttpApi

So I am getting somewhere with my HttpApi stuff. Got a basic API working.

Now, I need to support a streaming response.

I have a service that calls my remote LangChain service, and that returns a stream :

  streamTokenizedResponse: () => Effect.gen(function* () {
    const remoteChain = new RemoteRunnable({
      url: `https://mylangchain.com`
    });

    const remoteStream = yield* Effect.tryPromise({
      try: () =>
        remoteChain.stream(
          {
            question: "What's your favourite colour?"
          },
          {
            configurable: {
              session_id: '123abc'
            }
          }
        ),
      catch: (error) =>
        new Error(`Failed to establish a streaming connection: ${error}`)
    });

    return Stream.fromAsyncIterable(
      remoteStream, // Pass the async iterable
      (error) => Effect.fail(new Error('Streaming failed'))
    ) as Stream.Stream<string>;
  })


Have tried a few ways to define my endpoint and implementation:

export const MyApi = HttpApi.make('MyApi').add(
  HttpApiGroup.make('chat')
    .add(
      HttpApiEndpoint.get('ask')`/chat`
        .addSuccess(Schema.Array(Schema.String))
    )
);


export const MyChatLive = HttpApiBuilder.group(MyApi, 'chat', (handlers) =>
  Effect.gen(function* () {
    const searchService = yield* SearchService;

    return handlers.handle(
      'ask',
      () =>
        Effect.succeed(
          searchService.streamTokenizedResponse() as Stream.Stream<string>
        )
    );
  })
);


Does not even transpile. Have had a good go at sorting out the errors:

TS2322: Type Effect<Stream<string, never, never>, never, never> is not assignable to type Effect<readonly string[], HttpApiDecodeError, never>

How can I accomplish this? The samples provide no examples of streaming.
Was this page helpful?