Workers AI: getting internal server error

When doing the following call

const res = await ai.run('@cf/mistral/mistral-7b-instruct-v0.1', { messages, max_tokens: 1000 });
    return c.json({ res })

const res = await ai.run('@cf/mistral/mistral-7b-instruct-v0.1', { messages, max_tokens: 1000 });
    return c.json({ res })

const res = await ai.run('@cf/mistral/mistral-7b-instruct-v0.1', { messages, max_tokens: 1000 });
    return c.json({ res })

const res = await ai.run('@cf/mistral/mistral-7b-instruct-v0.1', { messages, max_tokens: 1000 });
    return c.json({ res })

I'm getting InferenceUpstreamError, httpCode: 504. Any ideas, if i run with the default tokens size (not specifying max_tokens) it seems to work. Btw, running this locally.

No replies yetBe the first to reply to this messageJoin

Workers AI: getting internal server error

Similar Threads

Similar Threads

Similar Threads